[00:08:35] [[Special:Log/newusers]] create 10 * Contemporaryart * (New user account) [00:13:15] Load avg. on ortelius is WARNING: WARNING - load average: 5.50, 12.21, 19.33 [00:21:15] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 43.09, 21.75, 19.66 [00:22:15] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 381444 MB (7% inode=69%): [00:23:15] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [00:23:25] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:24:05] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [00:24:46] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [00:24:55] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:26:55] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 49652 MB (8% inode=99%): [00:27:45] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:27:55] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:40:14] Load avg. on ortelius is WARNING: WARNING - load average: 14.02, 15.58, 19.99 [00:41:15] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 21.54, 17.52, 20.40 [00:46:15] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2045 [00:46:15] Load avg. on ortelius is WARNING: WARNING - load average: 5.33, 15.51, 19.45 [00:53:05] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [01:22:15] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 383088 MB (7% inode=69%): [01:22:54] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1836.000000 [01:23:04] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1837 [01:23:15] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [01:23:15] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3632 [01:23:25] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:24:05] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [01:24:45] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [01:24:55] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:26:55] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 49497 MB (8% inode=99%): [01:27:45] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:27:55] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:31:35] wikidata replag on z-dat-s2-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1902.000000 [01:37:15] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 28.76, 35.53, 31.29 [01:37:34] wikidata replag on z-dat-s2-b is OK: QUERY OK: SELECT ts_rc_age() returned 1770.000000 [01:37:45] s1 replag on thyme is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1880.000000 [01:37:55] MySQL slave on thyme is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1881 [01:38:25] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 69410 MB (7% inode=99%): [01:51:15] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3390 [01:53:05] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [01:56:46] s1 replag on thyme is OK: QUERY OK: SELECT ts_rc_age() returned 1773.000000 [01:56:55] MySQL slave on thyme is OK: Uptime: 5545642 Threads: 8 Questions: 3101613241 Slow queries: 376022 Opens: 130607 Flush tables: 1 Open tables: 689 Queries per second avg: 559.288 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1744 [02:04:15] MySQL slave on z-dat-s2-b is OK: Uptime: 196649 Threads: 9 Questions: 142521516 Slow queries: 2647 Opens: 6458741 Flush tables: 1 Open tables: 64 Queries per second avg: 724.750 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1718 [02:13:45] s1 replag on thyme is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1818.000000 [02:13:55] MySQL slave on thyme is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1816 [02:15:45] s1 replag on thyme is OK: QUERY OK: SELECT ts_rc_age() returned 1794.000000 [02:15:55] MySQL slave on thyme is OK: Uptime: 5546782 Threads: 12 Questions: 3102395225 Slow queries: 376084 Opens: 130609 Flush tables: 1 Open tables: 689 Queries per second avg: 559.314 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1747 [02:22:15] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 381813 MB (7% inode=69%): [02:22:55] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2455.000000 [02:23:05] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2449 [02:23:15] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (3 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.CommunicationLost.desc:S3:oob:S4:2530:S9:ts-array5:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.descri [02:23:25] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:24:05] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:24:44] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [02:24:55] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:26:55] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 49296 MB (8% inode=99%): [02:27:45] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:27:55] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:32:55] s1 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1796.000000 [02:33:05] MySQL slave on rosemary is OK: Uptime: 471069 Threads: 20 Questions: 316820979 Slow queries: 88875 Opens: 35370 Flush tables: 1 Open tables: 3052 Queries per second avg: 672.557 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1786 [02:37:14] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 22.41, 21.93, 27.37 [02:43:05] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2035 [02:43:55] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2044.000000 [02:53:05] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [03:05:55] s1 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1779.000000 [03:06:05] MySQL slave on rosemary is OK: Uptime: 473049 Threads: 16 Questions: 318134191 Slow queries: 89196 Opens: 35513 Flush tables: 1 Open tables: 3054 Queries per second avg: 672.518 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1764 [03:22:15] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 380481 MB (7% inode=69%): [03:23:14] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [03:23:24] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:24:05] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:24:46] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [03:24:55] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:26:56] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 49201 MB (8% inode=99%): [03:27:45] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:27:55] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:37:15] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 20.76, 22.03, 24.05 [03:48:16] Load avg. on ortelius is WARNING: WARNING - load average: 10.62, 15.90, 19.95 [03:52:15] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 33.24, 20.13, 20.32 [03:53:05] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [03:54:15] Load avg. on ortelius is WARNING: WARNING - load average: 16.99, 19.07, 19.93 [04:16:17] Load avg. on ortelius is WARNING: WARNING - load average: 12.43, 14.48, 16.81 [04:22:16] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 380677 MB (7% inode=69%): [04:23:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [04:23:27] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [04:24:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [04:24:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:26:57] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 49107 MB (8% inode=99%): [04:27:17] Load avg. on ortelius is OK: OK - load average: 5.62, 11.73, 14.52 [04:27:58] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:27:58] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:53:06] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [05:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 379410 MB (7% inode=69%): [05:23:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [05:23:27] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:24:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [05:24:58] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:27:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 49050 MB (8% inode=99%): [05:27:17] Load avg. on ortelius is WARNING: WARNING - load average: 9.14, 15.14, 13.29 [05:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:28:17] Load avg. on ortelius is OK: OK - load average: 9.98, 14.48, 13.18 [05:28:58] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:33:57] snag and nagios-wm, yeah. [05:34:06] You can't have a decent IRC experience without ignoring them. [05:34:11] tsnag * [05:34:14] Fucking auto-correct. [05:53:07] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [06:10:17] Load avg. on ortelius is WARNING: WARNING - load average: 24.93, 24.25, 19.44 [06:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 379122 MB (7% inode=69%): [06:23:16] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (3 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.CommunicationLost.desc: [06:23:27] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:24:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [06:24:58] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:25:16] Load avg. on ortelius is OK: OK - load average: 10.36, 12.10, 14.88 [06:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:27:57] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 48980 MB (8% inode=99%): [06:28:58] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:31:07] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [06:31:07] APT on yarrow is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [06:31:57] APT on nightshade is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [06:49:17] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 16.36, 26.40, 20.15 [06:50:17] Load avg. on ortelius is WARNING: WARNING - load average: 16.51, 24.51, 19.87 [06:53:17] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [07:01:17] Load avg. on ortelius is OK: OK - load average: 10.02, 11.96, 14.82 [07:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 375822 MB (7% inode=69%): [07:23:16] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [07:23:27] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [07:24:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [07:24:58] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:27:57] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 48839 MB (7% inode=99%): [07:28:58] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:31:07] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [07:31:07] APT on yarrow is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [07:31:57] APT on nightshade is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [08:01:17] Load avg. on ortelius is CRITICAL: CRITICAL - load average: 11.28, 28.07, 28.40 [08:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 375443 MB (7% inode=69%): [08:23:07] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [08:23:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [08:23:26] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:24:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [08:24:58] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:28:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 48669 MB (7% inode=99%): [08:28:17] Load avg. on ortelius is WARNING: WARNING - load average: 2.66, 12.14, 19.90 [08:28:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:31:07] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [08:31:07] APT on yarrow is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [08:31:57] APT on nightshade is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [08:36:17] Load avg. on ortelius is OK: OK - load average: 5.56, 8.59, 14.75 [09:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 374457 MB (7% inode=69%): [09:23:07] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [09:23:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [09:23:26] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:24:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [09:24:58] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:28:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 48440 MB (7% inode=99%): [09:28:58] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:31:07] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [09:31:07] APT on yarrow is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [09:31:57] APT on nightshade is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [09:54:37] /sql on cassia is WARNING: DISK WARNING - free space: /sql 93630 MB (7% inode=99%): [10:22:16] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 373446 MB (7% inode=69%): [10:23:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [10:23:27] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [10:24:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [10:24:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:28:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:28:58] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 48187 MB (7% inode=99%): [10:31:07] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [10:31:08] APT on yarrow is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [10:31:57] APT on nightshade is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [10:52:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [11:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 371800 MB (6% inode=69%): [11:23:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [11:23:26] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [11:24:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [11:24:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:28:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:28:58] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 47878 MB (7% inode=99%): [11:31:07] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [11:31:08] APT on yarrow is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [11:31:57] APT on nightshade is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [11:53:07] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [12:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 369994 MB (6% inode=69%): [12:23:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (3 errors): null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.CommunicationLost.desc:S3:oob:S4:2530:S9:ts-array5:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.descri [12:23:27] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:24:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [12:24:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:28:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:29:57] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 47480 MB (7% inode=99%): [12:31:07] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [12:31:07] APT on yarrow is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [12:31:57] APT on nightshade is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [12:53:07] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [13:20:17] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2082 [13:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 369462 MB (6% inode=69%): [13:23:16] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [13:23:27] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:24:58] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [13:24:58] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:28:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:29:57] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 47109 MB (7% inode=99%): [13:31:07] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [13:31:07] APT on yarrow is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [13:31:57] APT on nightshade is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [13:38:27] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 70086 MB (7% inode=99%): [13:48:17] MySQL slave on z-dat-s2-b is OK: Uptime: 238892 Threads: 8 Questions: 173389429 Slow queries: 3261 Opens: 8040324 Flush tables: 1 Open tables: 64 Queries per second avg: 725.806 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1748 [13:52:23] hello all [13:52:25] @replag [13:52:25] DaBPunkt: s1-rr-a-wd: 1m 4s [+0.00 s/s]; s1-user-wd: 1m 4s [+0.00 s/s]; s2-rr: 25m 52s [+0.01 s/s]; s2-user: 25m 52s [+0.01 s/s]; s2-user-c: error; s2-user-wd: 1m 4s [-0.08 s/s]; s4-user-wd: 1m 4s [+0.00 s/s]; s5-user-c: 1m 17s [-0.00 s/s] [13:52:26] DaBPunkt: s5-user-wd: 1m 4s [+0.00 s/s]; s6-user-wd: 1m 4s [+0.00 s/s]; s7-user-wd: 1m 4s [+0.00 s/s] [13:52:33] looks good :) [13:53:07] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [13:59:17] hello DaBPunkt [14:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 368874 MB (6% inode=69%): [14:23:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [14:23:27] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [14:24:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [14:24:58] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:28:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:29:57] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 46858 MB (7% inode=99%): [14:31:07] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [14:31:07] APT on yarrow is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [14:31:57] APT on nightshade is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [14:33:37] /sql on cassia is OK: DISK OK - free space: /sql 156897 MB (13% inode=99%): [15:14:07] APT on yarrow is OK: APT OK: 0 packages available for upgrade (0 critical updates). [15:14:57] APT on nightshade is OK: APT OK: 0 packages available for upgrade (0 critical updates). [15:15:07] APT on mayapple is OK: APT OK: 0 packages available for upgrade (0 critical updates). [15:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 367422 MB (6% inode=69%): [15:23:07] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [15:23:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [15:23:27] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:24:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [15:24:58] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:28:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:29:57] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 46558 MB (7% inode=99%): [16:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 366691 MB (6% inode=69%): [16:23:08] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [16:23:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (3 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.CommunicationLost.desc:S3:oob:S4:2530:S9:ts-array5:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.descri [16:23:27] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:24:58] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [16:24:58] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:28:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:29:57] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 46281 MB (7% inode=99%): [17:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 366069 MB (6% inode=69%): [17:23:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [17:23:27] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:24:58] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [17:24:58] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:28:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:29:57] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 46068 MB (7% inode=99%): [17:53:17] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [18:12:02] Yey, Connection to willow is fast now :) [18:18:06] Hersfold * Re: [Toolserver-l] Block mail from dysgo.org? [18:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 366581 MB (6% inode=69%): [18:23:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [18:23:27] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:24:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:24:58] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [18:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:28:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:29:58] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 45728 MB (7% inode=99%): [19:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 366084 MB (6% inode=69%): [19:23:07] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [19:23:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [19:23:27] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:24:06] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:24:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [19:24:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:28:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:29:57] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 45161 MB (7% inode=99%): [20:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 365413 MB (6% inode=69%): [20:23:18] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [20:23:27] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [20:24:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [20:24:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:28:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:29:57] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 44823 MB (7% inode=99%): [20:52:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [21:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 364487 MB (6% inode=69%): [21:23:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [21:23:37] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:24:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [21:24:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:28:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:30:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 44518 MB (7% inode=99%): [21:52:58] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [22:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 363298 MB (6% inode=69%): [22:23:16] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [22:23:37] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [22:24:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [22:24:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:28:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:30:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 44184 MB (7% inode=99%): [22:36:17] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2001 [22:52:57] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [23:22:17] /mnt user-store on rosemary is WARNING: DISK WARNING - free space: /mnt 361486 MB (6% inode=69%): [23:23:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [23:23:37] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:24:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [23:24:57] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [23:24:58] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:27:57] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:28:57] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:30:07] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 43957 MB (7% inode=99%): [23:36:17] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2177 [23:41:17] MySQL slave on z-dat-s2-b is OK: Uptime: 274472 Threads: 10 Questions: 198534925 Slow queries: 3861 Opens: 9512942 Flush tables: 1 Open tables: 65 Queries per second avg: 723.333 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1684 [23:53:07] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL