[00:09:06] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [00:09:48] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [00:09:48] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [00:10:06] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 955952.000000 [00:10:06] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:10:06] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:11:07] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:11:07] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:11:07] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [00:11:07] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:11:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:11:37] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 293354.000000 [00:11:37] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 60455 MB (9% inode=99%): [00:11:47] / on thyme is UNKNOWN: NRPE: Unable to read output [00:11:57] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:11:57] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:12:06] APT on yarrow is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [00:12:06] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:12:06] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [00:12:36] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:12:36] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:12:37] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:12:37] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:12:37] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [00:12:48] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [00:12:48] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 263685 MB (4% inode=62%): [00:12:48] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:12:48] APT on yucca is CRITICAL: APT CRITICAL: 6 packages available for upgrade (5 critical updates). [00:12:57] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [00:13:17] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:13:17] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:14:00] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:14:00] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:14:00] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:14:00] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:14:00] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:14:00] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:14:00] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:14:00] APT on sage is CRITICAL: APT CRITICAL: 8 packages available for upgrade (5 critical updates). [00:14:00] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:14:00] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:14:00] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:14:16] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1061899.000000 [00:15:17] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:15:47] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [00:15:56] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:15:56] APT on nightshade is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [00:16:27] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:16:27] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:16:27] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:16:27] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:16:27] RAID on thyme is UNKNOWN: NRPE: Unable to read output [00:16:27] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 626976.000000 [00:16:27] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:16:28] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:17:17] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:17:17] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [00:17:17] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 26087.000000 [00:17:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:17:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:17:47] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [00:17:48] APT on z-dat-s1-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [00:17:57] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [00:18:37] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [00:18:57] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [00:32:27] MySQL slave on daphne is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 21684 [00:32:57] s4 replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 21714.000000 [00:56:47] / on wolfsbane is WARNING: DISK WARNING - free space: / 6219 MB (20% inode=93%): [00:56:57] /tmp on wolfsbane is WARNING: DISK WARNING - free space: / 6215 MB (20% inode=93%): [01:09:06] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [01:09:36] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [01:09:47] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [01:09:48] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [01:10:06] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 959428.000000 [01:10:06] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:10:06] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:11:06] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:11:06] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:11:07] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [01:11:07] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:11:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:11:37] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 296403.000000 [01:11:37] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 60208 MB (9% inode=99%): [01:11:48] / on thyme is UNKNOWN: NRPE: Unable to read output [01:11:57] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:11:57] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:12:06] APT on yarrow is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [01:12:06] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:12:06] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [01:12:36] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:12:36] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:12:37] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:12:37] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [01:12:37] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:12:48] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [01:12:48] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 266520 MB (5% inode=62%): [01:12:49] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:12:49] APT on yucca is CRITICAL: APT CRITICAL: 6 packages available for upgrade (5 critical updates). [01:12:56] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [01:13:16] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:13:17] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:13:57] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:13:57] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:13:57] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:13:57] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:13:57] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:13:57] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:13:57] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:13:58] APT on sage is CRITICAL: APT CRITICAL: 8 packages available for upgrade (5 critical updates). [01:13:58] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:13:59] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:13:59] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:15:17] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1062081.000000 [01:15:17] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:15:47] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [01:15:57] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:15:57] APT on nightshade is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [01:16:26] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:16:27] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:16:27] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:16:27] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:16:27] RAID on thyme is UNKNOWN: NRPE: Unable to read output [01:16:27] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 628333.000000 [01:16:27] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:16:28] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:17:17] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:17:17] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [01:17:17] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 21685.000000 [01:17:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:17:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:17:56] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [01:18:37] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [01:18:48] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [01:18:48] APT on z-dat-s1-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [01:18:57] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [01:32:57] s4 replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 25315.000000 [01:33:29] MySQL slave on daphne is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 25344 [01:56:47] / on wolfsbane is WARNING: DISK WARNING - free space: / 6127 MB (20% inode=93%): [01:56:56] /tmp on wolfsbane is WARNING: DISK WARNING - free space: / 6127 MB (20% inode=93%): [02:09:06] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [02:09:37] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [02:09:48] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [02:09:48] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [02:10:06] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 962929.000000 [02:10:06] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:10:06] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:11:07] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:11:07] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:11:07] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [02:11:07] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:11:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:11:37] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 299606.000000 [02:11:37] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 58054 MB (9% inode=99%): [02:12:06] APT on yarrow is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [02:12:06] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:12:06] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [02:12:36] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:12:36] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:12:37] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:12:37] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:12:47] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [02:12:48] / on thyme is UNKNOWN: NRPE: Unable to read output [02:12:48] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [02:12:48] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 266470 MB (5% inode=62%): [02:12:48] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:12:48] APT on yucca is CRITICAL: APT CRITICAL: 6 packages available for upgrade (5 critical updates). [02:12:57] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:12:57] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:13:17] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:13:17] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:13:57] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [02:13:57] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:13:57] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:13:57] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:13:57] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:13:57] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:13:57] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:13:58] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:13:58] APT on sage is CRITICAL: APT CRITICAL: 8 packages available for upgrade (5 critical updates). [02:13:59] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:13:59] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:14:00] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:15:17] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1063520.000000 [02:15:17] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:15:47] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [02:15:57] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:15:57] APT on nightshade is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [02:17:17] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:17:17] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:17:17] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 19712.000000 [02:17:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:17:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:17:27] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:17:27] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:17:27] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:17:27] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:17:27] RAID on thyme is UNKNOWN: NRPE: Unable to read output [02:17:27] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 630068.000000 [02:17:28] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:17:28] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [02:17:57] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [02:18:37] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [02:18:47] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [02:19:47] APT on z-dat-s1-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [02:19:57] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [02:32:57] s4 replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 28915.000000 [02:34:27] MySQL slave on daphne is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 29004 [02:56:17] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2185.000000 [02:56:47] / on wolfsbane is WARNING: DISK WARNING - free space: / 6041 MB (20% inode=93%): [02:56:56] /tmp on wolfsbane is WARNING: DISK WARNING - free space: / 6041 MB (20% inode=93%): [03:08:48] / on wolfsbane is OK: DISK OK - free space: / 6704 MB (22% inode=93%): [03:08:56] /tmp on wolfsbane is OK: DISK OK - free space: / 6705 MB (22% inode=93%): [03:09:06] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [03:09:48] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [03:10:06] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 966387.000000 [03:10:06] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:10:06] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:10:47] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [03:10:48] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [03:11:07] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [03:11:07] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:11:07] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:11:07] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:11:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:11:37] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 303120.000000 [03:11:37] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 59934 MB (9% inode=99%): [03:12:06] APT on yarrow is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [03:12:07] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:12:07] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [03:12:47] / on thyme is UNKNOWN: NRPE: Unable to read output [03:12:48] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [03:12:49] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 266216 MB (4% inode=62%): [03:12:49] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:12:49] APT on yucca is CRITICAL: APT CRITICAL: 6 packages available for upgrade (5 critical updates). [03:12:57] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:12:57] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:13:36] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:13:37] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:13:37] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:13:37] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [03:13:37] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:14:16] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:14:16] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:14:57] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [03:14:57] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:14:57] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:14:57] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:14:57] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:14:57] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:14:57] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:14:58] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:14:58] APT on sage is CRITICAL: APT CRITICAL: 8 packages available for upgrade (5 critical updates). [03:14:59] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:14:59] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:15:00] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:15:17] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1066783.000000 [03:15:17] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:15:47] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [03:15:56] APT on nightshade is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [03:15:56] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:17:26] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:17:27] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:17:27] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:17:27] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:17:27] RAID on thyme is UNKNOWN: NRPE: Unable to read output [03:17:27] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:17:27] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:17:28] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 633231.000000 [03:17:56] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [03:18:16] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:18:17] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [03:18:17] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 22023.000000 [03:18:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:18:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:18:48] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [03:19:36] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [03:19:47] APT on z-dat-s1-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [03:19:57] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [03:20:17] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3626.000000 [03:32:57] s4 replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 32516.000000 [03:35:26] MySQL slave on daphne is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 32665 [04:09:06] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [04:10:06] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 969672.000000 [04:10:06] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:10:06] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:10:47] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [04:11:06] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [04:11:48] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [04:12:06] APT on yarrow is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [04:12:06] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:12:06] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:12:06] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:12:07] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [04:12:36] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:12:37] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 306781.000000 [04:12:37] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 59291 MB (9% inode=99%): [04:12:47] / on thyme is UNKNOWN: NRPE: Unable to read output [04:12:48] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [04:12:48] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 265976 MB (4% inode=62%): [04:12:48] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:12:48] APT on yucca is CRITICAL: APT CRITICAL: 6 packages available for upgrade (5 critical updates). [04:12:57] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:12:57] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:13:06] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:13:37] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:13:37] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:13:37] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:13:37] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [04:13:37] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:14:16] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:14:17] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:14:57] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [04:14:57] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:14:57] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:14:57] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:14:57] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:14:57] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:14:57] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:14:58] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:14:58] APT on sage is CRITICAL: APT CRITICAL: 8 packages available for upgrade (5 critical updates). [04:14:59] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:14:59] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:15:00] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:15:17] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1070382.000000 [04:15:17] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:15:47] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [04:15:56] APT on nightshade is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [04:15:56] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:17:26] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:17:27] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:17:27] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:17:28] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:17:28] RAID on thyme is UNKNOWN: NRPE: Unable to read output [04:17:28] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 636831.000000 [04:17:28] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:17:28] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:18:17] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:18:17] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [04:18:17] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 25623.000000 [04:18:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:18:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:18:56] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [04:19:36] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [04:19:48] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [04:19:48] APT on z-dat-s1-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [04:19:56] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [04:20:17] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7225.000000 [04:23:57] /tmp on wolfsbane is WARNING: DISK WARNING - free space: / 6203 MB (20% inode=93%): [04:24:47] / on wolfsbane is WARNING: DISK WARNING - free space: / 6202 MB (20% inode=93%): [04:33:57] s4 replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 36175.000000 [04:35:26] MySQL slave on daphne is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 36265 [04:39:37] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [05:09:06] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [05:10:07] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 972816.000000 [05:10:07] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:10:07] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:10:47] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [05:11:06] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [05:12:06] APT on yarrow is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [05:12:06] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:12:36] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:12:37] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 310381.000000 [05:12:37] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 59119 MB (9% inode=99%): [05:12:47] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [05:12:48] APT on yucca is CRITICAL: APT CRITICAL: 6 packages available for upgrade (5 critical updates). [05:13:07] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:13:07] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:13:07] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:13:07] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [05:13:37] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:13:37] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:13:37] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [05:13:37] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:13:37] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:13:47] / on thyme is UNKNOWN: NRPE: Unable to read output [05:13:47] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [05:13:47] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 265845 MB (4% inode=62%): [05:13:47] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:13:56] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:13:56] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:14:16] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:14:16] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:15:48] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [05:15:56] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [05:15:56] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:15:56] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:15:56] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:15:57] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:15:57] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:15:57] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:15:58] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:15:58] APT on sage is CRITICAL: APT CRITICAL: 8 packages available for upgrade (5 critical updates). [05:15:59] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:15:59] APT on nightshade is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [05:16:00] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:16:00] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:16:01] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:16:16] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1074043.000000 [05:16:17] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:18:17] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:18:17] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 29223.000000 [05:18:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:18:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:18:27] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:18:27] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:18:27] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:18:27] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:18:27] RAID on thyme is UNKNOWN: NRPE: Unable to read output [05:18:27] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 640491.000000 [05:18:27] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:18:28] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:18:56] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [05:19:17] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [05:19:56] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [05:20:17] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10825.000000 [05:20:37] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [05:20:47] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [05:20:47] APT on z-dat-s1-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [05:23:57] /tmp on wolfsbane is WARNING: DISK WARNING - free space: / 6124 MB (20% inode=93%): [05:24:47] / on wolfsbane is WARNING: DISK WARNING - free space: / 6100 MB (20% inode=93%): [05:34:57] s4 replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 39836.000000 [05:35:26] MySQL slave on daphne is CRITICAL: (Return code of 139 is out of bounds) [05:39:37] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [06:09:06] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [06:10:47] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [06:11:06] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 976120.000000 [06:11:06] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:11:06] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:12:06] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [06:12:06] APT on yarrow is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [06:12:06] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:12:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:12:38] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 313982.000000 [06:12:38] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 58925 MB (9% inode=99%): [06:12:47] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [06:12:48] APT on yucca is CRITICAL: APT CRITICAL: 6 packages available for upgrade (5 critical updates). [06:13:06] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:13:07] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:13:07] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:13:07] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [06:13:37] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:13:37] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [06:13:37] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:13:47] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:13:47] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:13:47] / on thyme is UNKNOWN: NRPE: Unable to read output [06:13:48] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [06:13:48] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 265710 MB (4% inode=62%): [06:13:48] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:13:57] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:13:57] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:14:17] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:14:17] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:15:57] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [06:15:57] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:15:57] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:15:57] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:15:57] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:15:58] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:15:58] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:15:59] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:15:59] APT on sage is CRITICAL: APT CRITICAL: 8 packages available for upgrade (5 critical updates). [06:16:00] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:16:00] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:16:01] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:16:01] APT on nightshade is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [06:16:06] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:16:16] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1077643.000000 [06:16:17] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:16:48] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [06:18:18] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:18:18] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 32823.000000 [06:18:18] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:18:18] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:18:25] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:18:26] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:18:27] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:18:27] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:18:57] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [06:19:26] RAID on thyme is UNKNOWN: NRPE: Unable to read output [06:19:26] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:19:26] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:19:26] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 644153.000000 [06:19:56] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [06:20:17] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [06:20:17] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14425.000000 [06:20:37] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [06:20:48] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [06:20:48] APT on z-dat-s1-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [06:23:57] /tmp on wolfsbane is WARNING: DISK WARNING - free space: / 5961 MB (19% inode=93%): [06:24:48] / on wolfsbane is WARNING: DISK WARNING - free space: / 5950 MB (19% inode=93%): [06:34:57] s4 replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 43436.000000 [06:35:26] MySQL slave on daphne is CRITICAL: (Return code of 139 is out of bounds) [06:39:36] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [07:10:06] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [07:10:48] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [07:11:06] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 979455.000000 [07:11:06] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:11:06] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:12:07] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [07:12:07] APT on yarrow is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [07:12:07] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:12:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:12:37] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 317444.000000 [07:12:37] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 58770 MB (9% inode=99%): [07:12:47] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [07:13:06] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:13:07] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:13:07] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:13:07] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [07:13:36] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:13:37] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [07:13:48] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:13:48] / on thyme is UNKNOWN: NRPE: Unable to read output [07:13:48] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:13:48] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:13:48] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [07:13:48] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 265507 MB (4% inode=62%): [07:13:48] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:13:49] APT on yucca is CRITICAL: APT CRITICAL: 6 packages available for upgrade (5 critical updates). [07:14:17] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:14:17] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:14:56] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:14:56] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:15:57] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [07:15:57] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:15:57] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:15:57] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:15:57] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:15:57] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:15:57] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:15:58] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:15:58] APT on sage is CRITICAL: APT CRITICAL: 8 packages available for upgrade (5 critical updates). [07:15:59] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:16:07] APT on nightshade is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [07:16:07] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:16:17] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1080714.000000 [07:16:17] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:16:47] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [07:16:56] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:16:57] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:18:16] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:18:17] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 33753.000000 [07:18:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:18:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:18:26] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:18:26] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:18:26] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:18:26] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:18:56] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [07:19:26] RAID on thyme is UNKNOWN: NRPE: Unable to read output [07:19:26] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:19:26] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:19:26] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 646545.000000 [07:20:16] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [07:20:16] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 12246.000000 [07:20:36] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [07:20:57] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [07:21:47] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [07:21:48] APT on z-dat-s1-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [07:23:56] /tmp on wolfsbane is WARNING: DISK WARNING - free space: / 5790 MB (19% inode=93%): [07:24:48] / on wolfsbane is WARNING: DISK WARNING - free space: / 5792 MB (19% inode=93%): [07:34:56] s4 replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 47035.000000 [07:35:26] MySQL slave on daphne is CRITICAL: (Return code of 139 is out of bounds) [07:39:42] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [07:59:17] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3565.000000 [08:07:36] Free Memory on turnera is WARNING: WARNING - 6.8% (569916 kB) free! [08:08:37] Free Memory on turnera is OK: OK - 7.2% (599496 kB) free. [08:10:06] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [08:10:47] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [08:11:07] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 981252.000000 [08:11:07] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:12:06] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [08:12:06] APT on yarrow is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [08:12:06] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:12:07] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:12:16] s4 replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1760.000000 [08:12:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:12:37] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 320345.000000 [08:12:37] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 58669 MB (9% inode=99%): [08:12:48] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [08:13:06] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:13:07] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:13:07] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:13:07] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [08:13:37] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:13:37] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [08:14:16] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:14:17] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:14:37] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:14:47] / on thyme is UNKNOWN: NRPE: Unable to read output [08:14:47] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:14:47] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:14:47] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [08:14:47] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 265252 MB (4% inode=62%): [08:14:48] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:14:48] APT on yucca is CRITICAL: APT CRITICAL: 6 packages available for upgrade (5 critical updates). [08:14:57] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:14:57] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:16:17] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1080598.000000 [08:16:17] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:16:56] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [08:16:56] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:16:56] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:16:56] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:16:56] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:16:57] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:16:57] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:16:57] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:16:58] APT on sage is CRITICAL: APT CRITICAL: 8 packages available for upgrade (5 critical updates). [08:16:58] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:16:59] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:16:59] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:17:00] APT on nightshade is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [08:17:06] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:17:47] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [08:18:26] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:18:26] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:18:26] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:18:26] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:19:06] MySQL slave on z-dat-s1-b is CRITICAL: (Service Check Timed Out) [08:19:16] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:19:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:19:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:19:26] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 34512.000000 [08:19:27] RAID on thyme is UNKNOWN: NRPE: Unable to read output [08:19:27] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:19:27] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:19:27] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 645617.000000 [08:20:17] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [08:20:37] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [08:20:57] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [08:21:48] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [08:21:48] APT on z-dat-s1-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [08:24:47] / on wolfsbane is WARNING: DISK WARNING - free space: / 5623 MB (18% inode=93%): [08:24:57] /tmp on wolfsbane is WARNING: DISK WARNING - free space: / 5623 MB (18% inode=93%): [08:34:57] s4 replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 41656.000000 [08:35:26] MySQL slave on daphne is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 41592 [08:39:37] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [09:02:26] wikidata replag on z-dat-s5-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1844.000000 [09:02:56] MySQL slave on z-dat-s5-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1853 [09:10:06] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [09:11:07] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 980349.000000 [09:11:07] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:11:47] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [09:12:06] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [09:12:06] APT on yarrow is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [09:12:07] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:12:07] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:12:38] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:12:38] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 323261.000000 [09:12:38] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 58411 MB (9% inode=99%): [09:13:06] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [09:13:06] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:13:06] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:13:06] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:13:37] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:13:37] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [09:13:47] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [09:14:37] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:14:47] / on thyme is UNKNOWN: NRPE: Unable to read output [09:14:47] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:14:47] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:14:48] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [09:14:48] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 264881 MB (4% inode=62%): [09:14:48] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:14:48] APT on yucca is CRITICAL: APT CRITICAL: 6 packages available for upgrade (5 critical updates). [09:15:16] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:15:16] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:15:56] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:15:57] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:16:17] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1080476.000000 [09:16:18] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:16:57] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [09:16:57] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:16:57] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:16:57] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:16:57] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:16:57] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:16:57] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:16:58] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:16:58] APT on sage is CRITICAL: APT CRITICAL: 8 packages available for upgrade (5 critical updates). [09:16:59] APT on nightshade is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [09:16:59] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:17:00] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:17:48] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [09:18:26] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:18:26] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:18:26] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:18:26] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:19:16] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:19:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:19:18] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:19:25] RAID on thyme is UNKNOWN: NRPE: Unable to read output [09:19:26] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:19:26] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:19:26] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 644481.000000 [09:19:48] MySQL slave on z-dat-s1-b is CRITICAL: (Service Check Timed Out) [09:20:16] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 33146.000000 [09:20:37] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [09:21:16] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [09:21:57] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [09:22:47] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [09:22:47] APT on z-dat-s1-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [09:24:47] / on wolfsbane is WARNING: DISK WARNING - free space: / 5424 MB (18% inode=93%): [09:24:56] /tmp on wolfsbane is WARNING: DISK WARNING - free space: / 5424 MB (18% inode=93%): [09:34:57] s4 replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 22244.000000 [09:35:26] MySQL slave on daphne is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 22121 [09:39:37] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [09:59:26] wikidata replag on z-dat-s5-b is OK: QUERY OK: SELECT ts_rc_age() returned 1752.000000 [09:59:57] MySQL slave on z-dat-s5-b is OK: Uptime: 96480 Threads: 6 Questions: 302141881 Slow queries: 709 Opens: 19599 Flush tables: 1 Open tables: 256 Queries per second avg: 3131.652 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1747 [10:10:06] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [10:11:06] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 977515.000000 [10:11:07] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:11:48] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [10:13:07] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [10:13:07] APT on yarrow is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [10:13:07] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:13:07] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:13:07] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:13:07] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:13:07] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:13:08] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [10:13:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:13:37] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:13:37] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 326150.000000 [10:13:37] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 58238 MB (9% inode=99%): [10:13:37] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [10:13:47] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [10:14:36] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:14:47] / on thyme is UNKNOWN: NRPE: Unable to read output [10:14:48] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [10:14:48] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 264420 MB (4% inode=62%): [10:14:48] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:14:48] APT on yucca is CRITICAL: APT CRITICAL: 6 packages available for upgrade (5 critical updates). [10:14:57] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:14:57] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:15:17] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:15:17] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:15:56] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:15:56] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:16:16] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1079676.000000 [10:16:17] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:16:57] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [10:16:57] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:16:57] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:16:57] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:16:57] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:16:57] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:16:58] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:16:58] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:16:58] APT on sage is CRITICAL: APT CRITICAL: 8 packages available for upgrade (5 critical updates). [10:16:59] APT on nightshade is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [10:16:59] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:17:00] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:17:00] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:17:17] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:17:47] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [10:18:25] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:18:26] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:18:26] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:18:26] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:19:17] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:19:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:19:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:19:26] RAID on thyme is UNKNOWN: NRPE: Unable to read output [10:19:26] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:19:26] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:19:26] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 643768.000000 [10:20:17] MySQL slave on z-dat-s1-b is CRITICAL: (Service Check Timed Out) [10:20:17] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 30898.000000 [10:20:37] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [10:21:16] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [10:22:48] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [10:22:48] APT on z-dat-s1-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [10:22:56] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [10:25:48] MySQL on z-dat-s1-b is CRITICAL: Too many connections [10:25:48] / on wolfsbane is WARNING: DISK WARNING - free space: / 5260 MB (17% inode=93%): [10:25:57] /tmp on wolfsbane is WARNING: DISK WARNING - free space: / 5259 MB (17% inode=93%): [10:27:57] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 17753 MB (1% inode=99%): [10:34:57] s4 replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 9328.000000 [10:35:25] MySQL slave on daphne is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 9173 [10:39:37] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [10:57:37] Free Memory on turnera is WARNING: WARNING - 5.2% (436288 kB) free! [10:58:26] MySQL slave on daphne is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3450 [10:58:36] Free Memory on turnera is CRITICAL: CRITICAL - 3.9% (325676 kB) free! [10:58:57] s4 replag on daphne is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3364.000000 [11:00:36] Free Memory on turnera is WARNING: WARNING - 5.2% (434244 kB) free! [11:04:26] MySQL slave on daphne is OK: Uptime: 314273 Threads: 26 Questions: 50004921 Slow queries: 66247 Opens: 14776 Flush tables: 1 Open tables: 765 Queries per second avg: 159.113 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1709 [11:04:57] s4 replag on daphne is OK: QUERY OK: SELECT ts_rc_age() returned 1621.000000 [11:10:07] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [11:11:06] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 975461.000000 [11:11:06] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:11:47] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [11:13:06] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [11:13:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:13:37] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 329023.000000 [11:13:37] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:13:37] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 58044 MB (9% inode=99%): [11:13:37] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [11:13:48] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [11:14:07] APT on yarrow is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [11:14:07] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:14:07] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:14:07] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:14:07] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:14:07] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:14:07] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [11:14:37] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:14:47] / on thyme is UNKNOWN: NRPE: Unable to read output [11:14:47] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [11:14:47] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 263998 MB (4% inode=62%): [11:14:48] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:14:48] APT on yucca is CRITICAL: APT CRITICAL: 6 packages available for upgrade (5 critical updates). [11:14:56] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:14:57] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:15:16] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:15:17] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:15:57] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:15:57] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:16:16] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1080102.000000 [11:16:17] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:16:56] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:16:57] APT on sage is CRITICAL: APT CRITICAL: 8 packages available for upgrade (5 critical updates). [11:17:17] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:17:57] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [11:17:57] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:17:57] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:17:57] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:17:57] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:17:57] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:17:57] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:17:58] APT on nightshade is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [11:17:58] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:17:59] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:17:59] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:18:26] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:18:26] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:18:26] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:18:26] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:18:47] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [11:19:17] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:19:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:19:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:19:26] RAID on thyme is UNKNOWN: NRPE: Unable to read output [11:19:26] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:19:26] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:19:26] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 644939.000000 [11:19:48] MySQL on z-dat-s1-b is OK: Uptime: 212735 Threads: 351 Questions: 147928258 Slow queries: 2327 Opens: 1216 Flush tables: 1 Open tables: 405 Queries per second avg: 695.363 [11:20:16] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 27574.000000 [11:20:48] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [11:21:17] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [11:21:37] MySQL slave on z-dat-s1-b is CRITICAL: (Service Check Timed Out) [11:21:37] Free Memory on turnera is WARNING: WARNING - 6.5% (545676 kB) free! [11:22:48] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [11:22:48] APT on z-dat-s1-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [11:22:57] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [11:25:48] / on wolfsbane is WARNING: DISK WARNING - free space: / 5069 MB (16% inode=93%): [11:25:56] /tmp on wolfsbane is WARNING: DISK WARNING - free space: / 5067 MB (16% inode=93%): [11:26:47] MySQL on z-dat-s1-b is CRITICAL: Too many connections [11:33:35] [[Special:Log/newusers]] create 10 * Apn * (New user account) [11:36:37] Free Memory on turnera is OK: OK - 7.1% (591484 kB) free. [11:39:36] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [11:54:27] MySQL on z-dat-s1-b is CRITICAL: Too many connections [12:09:26] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 666361.000000 [12:10:07] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [12:11:07] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 978484.000000 [12:11:17] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:11:57] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [12:13:07] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [12:13:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:13:37] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:13:57] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [12:14:07] APT on yarrow is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [12:14:07] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [12:14:16] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 57758 MB (9% inode=99%): [12:14:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [12:14:17] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:14:26] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 332288.000000 [12:14:57] / on thyme is UNKNOWN: NRPE: Unable to read output [12:14:57] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:14:57] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:15:07] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [12:15:07] APT on yucca is CRITICAL: APT CRITICAL: 6 packages available for upgrade (5 critical updates). [12:15:17] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:15:17] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 263672 MB (4% inode=62%): [12:15:27] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:15:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:15:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:16:06] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:16:07] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:16:47] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1082768.000000 [12:16:57] APT on sage is CRITICAL: APT CRITICAL: 8 packages available for upgrade (5 critical updates). [12:17:07] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:17:37] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:17:58] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [12:17:58] APT on nightshade is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [12:18:07] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:18:07] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:18:07] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:18:08] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:18:08] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:18:08] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:18:17] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:18:17] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:18:17] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:18:27] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:18:37] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:18:47] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:18:47] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:18:57] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [12:19:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:19:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:19:36] RAID on thyme is UNKNOWN: NRPE: Unable to read output [12:19:36] s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2192.000000 [12:19:47] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 644958.000000 [12:19:47] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:19:47] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:20:27] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 26541.000000 [12:20:57] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [12:21:47] MySQL slave on z-dat-s1-b is CRITICAL: Cant connect to MySQL server on z-dat-s1-b (146) [12:22:57] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [12:23:07] APT on z-dat-s1-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [12:23:08] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [12:25:57] /tmp on wolfsbane is WARNING: DISK WARNING - free space: / 4853 MB (16% inode=93%): [12:26:47] / on wolfsbane is WARNING: DISK WARNING - free space: / 4844 MB (16% inode=93%): [12:39:36] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [12:43:37] wikidata replag on z-dat-s5-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2139.000000 [12:43:37] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3632.000000 [12:43:57] MySQL slave on z-dat-s5-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2155 [12:46:17] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:46:18] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:46:18] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:46:18] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:46:18] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:47:17] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:47:17] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [12:54:27] MySQL on z-dat-s1-b is CRITICAL: Cant connect to MySQL server on z-dat-s1-b (146) [13:09:27] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 665040.000000 [13:10:07] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [13:11:07] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 981672.000000 [13:11:24] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:11:57] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [13:13:07] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [13:13:37] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:13:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:13:57] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [13:14:07] APT on yarrow is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [13:14:07] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [13:14:16] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 57536 MB (9% inode=99%): [13:14:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [13:14:17] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:14:27] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 335156.000000 [13:14:57] / on thyme is UNKNOWN: NRPE: Unable to read output [13:14:57] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:14:57] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:15:07] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [13:15:07] APT on yucca is CRITICAL: APT CRITICAL: 6 packages available for upgrade (5 critical updates). [13:15:17] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:15:17] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 263325 MB (4% inode=62%): [13:15:27] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:15:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:15:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:16:07] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:16:08] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:16:52] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1085204.000000 [13:16:57] APT on sage is CRITICAL: APT CRITICAL: 8 packages available for upgrade (5 critical updates). [13:17:07] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:17:37] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:17:57] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [13:17:58] APT on nightshade is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [13:18:07] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:18:08] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:18:08] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:18:08] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:18:08] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:18:08] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:18:17] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:18:17] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:18:18] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:18:27] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:18:37] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:18:47] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:18:47] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:18:57] APT on mayapple is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [13:19:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:19:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:19:37] RAID on thyme is UNKNOWN: NRPE: Unable to read output [13:19:46] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 642988.000000 [13:19:47] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:19:47] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:20:27] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 26822.000000 [13:20:57] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [13:21:46] MySQL slave on z-dat-s1-b is CRITICAL: Cant connect to MySQL server on z-dat-s1-b (146) [13:22:57] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [13:23:07] APT on z-dat-s1-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [13:23:07] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [13:25:57] /tmp on wolfsbane is WARNING: DISK WARNING - free space: / 4628 MB (15% inode=93%): [13:26:47] / on wolfsbane is WARNING: DISK WARNING - free space: / 4610 MB (15% inode=93%): [13:39:37] wikidata replag on z-dat-s5-b is OK: QUERY OK: SELECT ts_rc_age() returned 1766.000000 [13:39:57] MySQL slave on z-dat-s5-b is OK: Uptime: 109680 Threads: 4 Questions: 310167653 Slow queries: 968 Opens: 20738 Flush tables: 1 Open tables: 255 Queries per second avg: 2827.932 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1732 [13:43:37] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7232.000000 [13:46:16] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:46:16] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:46:17] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:46:17] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:46:18] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:47:17] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:47:17] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [13:52:04] Silke Meyer * [Toolserver-announce] Tool Labs: Summary of Feedback on the Draft for the Roadmap [13:54:27] MySQL on z-dat-s1-b is CRITICAL: Cant connect to MySQL server on z-dat-s1-b (146) [13:56:31] hello all [13:58:57] APT on mayapple is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:03:07] APT on yarrow is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:03:57] APT on nightshade is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:04:06] APT on yucca is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:06:00] APT on sage is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:09:26] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 663485.000000 [14:09:47] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [14:10:07] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [14:11:07] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 984044.000000 [14:11:16] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:11:57] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [14:13:07] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [14:13:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:13:37] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:13:57] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [14:14:07] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [14:14:18] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 57396 MB (9% inode=99%): [14:14:18] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [14:14:18] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:14:26] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 338038.000000 [14:14:56] / on thyme is UNKNOWN: NRPE: Unable to read output [14:14:57] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:14:57] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:15:06] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [14:15:17] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:15:17] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 262990 MB (4% inode=62%): [14:15:27] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:15:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:15:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:16:07] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:16:07] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:16:46] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1085220.000000 [14:17:07] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:17:37] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:17:57] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [14:18:07] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:18:07] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:18:08] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:18:08] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:18:08] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:18:08] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:18:17] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:18:18] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:18:18] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:18:26] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:18:36] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:18:47] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:18:47] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:19:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:19:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:19:37] RAID on thyme is UNKNOWN: NRPE: Unable to read output [14:19:46] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 640455.000000 [14:19:47] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:19:48] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:20:27] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 23684.000000 [14:20:57] APT on z-dat-s5-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:21:46] MySQL slave on z-dat-s1-b is CRITICAL: Cant connect to MySQL server on z-dat-s1-b (146) [14:22:57] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [14:23:07] APT on z-dat-s1-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:23:07] APT on z-dat-s2-b is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [14:24:07] APT on z-dat-s1-b is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:24:07] APT on z-dat-s2-b is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:24:17] impatient nagios… [14:24:57] APT on z-dat-s5-b is OK: APT OK: 0 packages available for upgrade (0 critical updates). [14:25:57] /tmp on wolfsbane is WARNING: DISK WARNING - free space: / 4440 MB (14% inode=93%): [14:26:47] / on wolfsbane is WARNING: DISK WARNING - free space: / 4414 MB (14% inode=93%): [14:30:57] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1861 [14:31:07] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1862.000000 [14:36:47] / on wolfsbane is OK: DISK OK - free space: / 14890 MB (49% inode=93%): [14:36:57] /tmp on wolfsbane is OK: DISK OK - free space: / 14897 MB (49% inode=93%): [14:43:37] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7248.000000 [14:46:16] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:46:17] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:46:17] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:46:18] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:46:18] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:47:17] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:47:17] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [14:54:27] MySQL on z-dat-s1-b is CRITICAL: Cant connect to MySQL server on z-dat-s1-b (146) [15:09:27] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 662111.000000 [15:10:07] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [15:11:08] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 986766.000000 [15:11:17] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:11:57] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [15:13:07] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [15:13:37] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:13:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:13:57] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [15:14:07] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [15:14:17] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 56935 MB (9% inode=99%): [15:14:18] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [15:14:18] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:14:26] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 340898.000000 [15:14:57] / on thyme is UNKNOWN: NRPE: Unable to read output [15:14:57] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:14:58] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:15:07] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [15:15:17] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:15:17] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 262685 MB (4% inode=62%): [15:15:27] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:15:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:15:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:16:06] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:16:07] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:16:47] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1085389.000000 [15:17:07] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:17:37] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:17:57] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [15:18:06] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:18:07] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:18:08] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:18:08] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:18:08] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:18:08] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:18:16] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:18:17] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:18:17] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:18:27] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:18:37] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:18:47] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:18:47] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:19:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:19:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:19:37] RAID on thyme is UNKNOWN: NRPE: Unable to read output [15:19:46] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 637800.000000 [15:19:47] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:19:47] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:20:27] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10903.000000 [15:21:46] MySQL slave on z-dat-s1-b is CRITICAL: Cant connect to MySQL server on z-dat-s1-b (146) [15:22:57] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [15:30:57] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2528 [15:31:07] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2529.000000 [15:39:27] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [15:43:37] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7284.000000 [15:46:17] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:46:17] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:46:17] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:46:17] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:46:17] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:47:17] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:47:17] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [15:54:27] MySQL on z-dat-s1-b is CRITICAL: Cant connect to MySQL server on z-dat-s1-b (146) [16:03:27] s4 replag on z-dat-s5-b is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3359.000000 [16:09:26] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 660342.000000 [16:09:57] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3607 [16:10:07] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3608.000000 [16:10:08] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [16:11:07] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 989708.000000 [16:11:17] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:11:58] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [16:13:07] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [16:13:37] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:13:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:13:57] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [16:14:07] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [16:14:18] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 56742 MB (9% inode=99%): [16:14:18] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [16:14:18] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:14:26] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 339704.000000 [16:14:57] / on thyme is UNKNOWN: NRPE: Unable to read output [16:14:57] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:14:57] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:15:07] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [16:15:17] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:15:17] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 262400 MB (4% inode=62%): [16:15:27] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:15:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:15:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:16:07] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:16:07] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:16:26] s4 replag on z-dat-s5-b is OK: QUERY OK: SELECT ts_rc_age() returned 1746.000000 [16:16:47] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1085663.000000 [16:17:07] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:17:37] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:17:56] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [16:18:06] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:18:07] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:18:08] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:18:08] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:18:08] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:18:08] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:18:17] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:18:17] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:18:17] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:18:27] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:18:37] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:18:47] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:18:47] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:19:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:19:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:19:36] RAID on thyme is UNKNOWN: NRPE: Unable to read output [16:19:46] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 636087.000000 [16:19:47] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:19:47] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:21:46] MySQL slave on z-dat-s1-b is CRITICAL: Cant connect to MySQL server on z-dat-s1-b (146) [16:22:57] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [16:39:40] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [16:43:37] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 5778.000000 [16:46:18] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:46:18] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:46:18] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:46:18] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:46:18] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:47:17] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:47:17] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [16:54:27] MySQL on z-dat-s1-b is CRITICAL: Cant connect to MySQL server on z-dat-s1-b (146) [16:56:10] Marc A. Pelletier * [Toolserver-l] Department of the reporting of statuses [16:58:04] Daniel Schwen * Re: [Toolserver-l] Wikidata tables [17:00:57] Free Memory on damiana is WARNING: WARNING - 6.3% (528388 kB) free! [17:03:58] this funtion was disabled on wikis https://commons.wikimedia.org/w/index.php?title=Special:ActiveUsers&limit=1&username= , you know a alternativ tool on the toolserver ( i like to update the templates at commons) Thank you^^ [17:05:42] Steinsplitter: no [17:05:52] ah, okay. Thank yoz [17:09:26] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 657653.000000 [17:09:57] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4576 [17:10:07] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 4579.000000 [17:10:08] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [17:11:07] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 992914.000000 [17:11:17] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:11:57] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [17:13:07] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [17:13:37] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:13:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:13:57] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [17:14:07] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [17:14:16] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 56577 MB (9% inode=99%): [17:14:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [17:14:17] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:14:26] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 338763.000000 [17:14:57] / on thyme is UNKNOWN: NRPE: Unable to read output [17:14:57] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:14:57] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:15:07] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [17:15:17] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:15:17] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 262185 MB (4% inode=62%): [17:15:27] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:15:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:15:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:16:07] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:16:07] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:16:47] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1084520.000000 [17:17:07] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:17:37] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:18:03] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [17:18:07] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:18:08] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:18:08] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:18:08] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:18:08] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:18:08] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:18:16] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:18:17] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:18:17] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:18:27] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:18:37] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:18:47] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:18:47] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:19:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:19:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:19:37] RAID on thyme is UNKNOWN: NRPE: Unable to read output [17:19:46] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 636191.000000 [17:19:47] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:19:47] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:21:47] MySQL slave on z-dat-s1-b is CRITICAL: Cant connect to MySQL server on z-dat-s1-b (146) [17:22:37] s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3534.000000 [17:22:57] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [17:27:57] Free Memory on damiana is CRITICAL: CRITICAL - 3.7% (307520 kB) free! [17:34:57] Free Memory on damiana is WARNING: WARNING - 5.3% (448204 kB) free! [17:39:57] Free Memory on damiana is CRITICAL: CRITICAL - 4.9% (407436 kB) free! [17:41:37] s4 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1672.000000 [17:46:17] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:46:18] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:46:18] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:46:18] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:46:18] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:47:17] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:47:17] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [17:52:27] MySQL on z-dat-s1-b is OK: Uptime: 20772 Threads: 2 Questions: 53 Slow queries: 0 Opens: 28 Flush tables: 1 Open tables: 21 Queries per second avg: 0.2 [18:03:07] /sql on z-dat-s1-b is WARNING: DISK WARNING - free space: /sql 106312 MB (10% inode=99%): [18:09:27] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 654134.000000 [18:09:37] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [18:09:57] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4327 [18:10:07] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 4327.000000 [18:10:07] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [18:11:07] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 996248.000000 [18:11:17] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:11:57] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [18:13:07] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [18:13:37] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:13:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:13:57] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [18:14:07] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [18:14:16] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 56283 MB (9% inode=99%): [18:14:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [18:14:17] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:14:27] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 338018.000000 [18:14:57] / on thyme is UNKNOWN: NRPE: Unable to read output [18:14:57] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:14:57] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:15:07] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [18:15:17] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:15:17] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 261943 MB (4% inode=62%): [18:15:27] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:15:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:15:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:16:08] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:16:08] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:16:47] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1082075.000000 [18:17:07] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:17:36] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:17:56] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [18:18:07] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:18:14] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:18:14] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:18:14] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:18:14] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:18:14] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:18:17] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:18:17] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:18:17] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:18:27] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:18:37] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:18:47] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:18:47] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:19:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:19:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:19:37] RAID on thyme is UNKNOWN: NRPE: Unable to read output [18:19:47] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 635502.000000 [18:19:47] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:19:47] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:21:47] MySQL slave on z-dat-s1-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 134841 [18:22:57] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [18:32:57] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3547 [18:33:07] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3541.000000 [18:39:57] Free Memory on damiana is CRITICAL: CRITICAL - 2.9% (241536 kB) free! [18:46:17] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:46:18] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:46:18] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:46:18] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:46:18] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:47:17] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [18:47:18] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:09:27] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 651419.000000 [19:09:37] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [19:10:07] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [19:11:07] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 999553.000000 [19:11:17] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:11:58] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [19:13:07] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [19:13:36] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:13:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:13:57] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [19:14:07] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [19:14:17] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 56045 MB (9% inode=99%): [19:14:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [19:14:17] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:14:27] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 336776.000000 [19:14:57] / on thyme is UNKNOWN: NRPE: Unable to read output [19:14:57] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:14:57] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:15:07] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [19:15:16] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:15:17] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 261726 MB (4% inode=62%): [19:15:26] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:15:26] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:15:58] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:16:07] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:16:47] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1078818.000000 [19:17:07] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:17:08] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:17:37] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:17:57] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [19:18:07] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:18:07] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:18:17] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:18:17] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:18:17] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:18:27] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:18:37] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:18:47] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:18:47] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:19:07] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:19:08] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:19:08] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:19:08] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:19:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:19:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:19:37] RAID on thyme is UNKNOWN: NRPE: Unable to read output [19:19:47] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 634613.000000 [19:19:47] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:19:47] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:21:47] MySQL slave on z-dat-s1-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 118357 [19:22:57] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [19:32:58] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3436 [19:33:07] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3437.000000 [19:38:57] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3616 [19:39:07] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3617.000000 [19:39:57] Free Memory on damiana is CRITICAL: CRITICAL - 2.7% (225464 kB) free! [19:46:17] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:46:17] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:46:17] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:47:16] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:47:18] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:47:18] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [19:47:18] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:09:27] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 649434.000000 [20:10:17] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [20:11:07] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1002783.000000 [20:11:17] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:11:57] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [20:13:07] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [20:13:37] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:13:37] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:13:57] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [20:14:17] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [20:14:17] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 55543 MB (9% inode=99%): [20:14:17] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [20:14:17] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:14:27] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 335678.000000 [20:14:57] / on thyme is UNKNOWN: NRPE: Unable to read output [20:14:57] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:14:58] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:15:08] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [20:15:17] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:15:17] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 261430 MB (4% inode=62%): [20:15:27] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:15:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:15:57] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:16:07] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:16:46] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1076586.000000 [20:17:08] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:17:08] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:17:37] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:17:57] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [20:18:08] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:18:08] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:18:16] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:18:16] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:18:17] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:18:26] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:18:36] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:18:47] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:18:47] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:19:07] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:19:08] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:19:08] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:19:08] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:19:17] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:19:17] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:19:37] RAID on thyme is UNKNOWN: NRPE: Unable to read output [20:19:47] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 632388.000000 [20:19:47] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:19:47] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:21:47] MySQL slave on z-dat-s1-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 106978 [20:22:57] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [20:38:57] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 5411 [20:39:27] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 5432.000000 [20:39:37] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [20:39:59] Free Memory on damiana is CRITICAL: CRITICAL - 1.7% (146212 kB) free! [20:46:16] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:46:17] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:46:18] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:47:17] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:47:17] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:47:17] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [20:47:17] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [21:01:33] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [21:01:45] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [21:01:46] FC 0/19 on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [21:01:47] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [21:01:47] FC 0/22 on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [21:01:47] FC 0/20 on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [21:01:55] FC 0/9 on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [21:01:55] eth0 on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [21:01:55] FC 0/23 on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [21:01:55] FC 0/14 on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [21:01:55] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [21:01:55] FC 0/21 on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Return code of 142 is out of bounds) [21:03:01] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is CRITICAL: (Service Check Timed Out) [21:20:36] [[Special:Log/newusers]] create 10 * Raczy * (New user account) [22:00:25] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 338351.000000 [22:00:25] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:00:25] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [22:00:25] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 55186 MB (9% inode=99%): [22:00:25] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:00:26] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:00:26] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:00:27] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:00:27] Sun Grid Engine execd on mayapple is CRITICAL: CRITICAL: execd not communicating [22:00:28] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [22:00:28] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3423 [22:00:34] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:00:35] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:00:35] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:00:35] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:00:35] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:00:35] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:00:35] toolserver.org HTTP on wolfsbane is OK: HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.011 second response time [22:00:44] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:00:54] Sun Grid Engine execd on yarrow is OK: Host and Queues Ok [22:01:04] toolserver.org HTTP on ortelius is OK: HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.104 second response time [22:01:14] s5 replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1480.000000 [22:01:14] MySQL slave on daphne is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3381 [22:01:24] Sun Grid Engine execd on mayapple is OK: Host and Queues Ok [22:01:44] Free Memory on damiana is OK: OK - 91.9% (7705016 kB) free. [22:02:10] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3604.000000 [22:02:10] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3601.000000 [22:02:13] Load avg. on yarrow is WARNING: WARNING - load average: 14.37, 21.51, 19.87 [22:03:15] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3613.000000 [22:03:55] SSH on z-dat-s4-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [22:04:20] The toolserver geohack tool is acting up again: "HTTP Error 500 Internal server error" [22:05:05] Complain to WMDE about it :-) [22:05:45] s4 replag on daphne is OK: QUERY OK: SELECT ts_rc_age() returned 1692.000000 [22:06:16] MySQL slave on daphne is OK: Uptime: 353977 Threads: 15 Questions: 56262147 Slow queries: 74603 Opens: 16563 Flush tables: 1 Open tables: 768 Queries per second avg: 158.942 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1447 [22:06:36] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [22:06:36] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [22:06:36] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 261172 MB (4% inode=62%): [22:06:36] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [22:06:36] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [22:06:55] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:06:55] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [22:07:06] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [22:07:15] Load avg. on yarrow is OK: OK - load average: 1.09, 8.51, 14.65 [22:09:54] @Dispenser: my German is REALLY bad ;-) [22:10:24] Great! You can be an angry American! [22:11:16] /sql on z-dat-s1-b is OK: DISK OK - free space: /sql 107108 MB (11% inode=99%): [22:11:55] I could be, except I'm living in Fiji. [22:13:00] I don't know if they monitor this channel but the people running toolderver and working on the "migration" to wikimedia-labs really need to get their act in gear. [22:13:44] End of June, if Labs doesn't get into action then WMDE is open to funding the Toolserver again [22:15:13] s/funding/expanding/ [22:15:26] sounds good [22:15:56] brb [22:18:40] ok [22:19:05] s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3311.000000 [22:19:15] RAID on thyme is UNKNOWN: NRPE: Unable to read output [22:19:25] / on thyme is UNKNOWN: NRPE: Unable to read output [22:23:45] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [22:25:05] s4 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1599.000000 [22:28:46] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 17361 MB (1% inode=99%): [22:37:05] s1 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1797.000000 [22:37:25] MySQL slave on rosemary is OK: Uptime: 359214 Threads: 15 Questions: 260282451 Slow queries: 142582 Opens: 20287 Flush tables: 1 Open tables: 1466 Queries per second avg: 724.588 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1762 [22:51:16] /sql on z-dat-s1-b is WARNING: DISK WARNING - free space: /sql 106794 MB (10% inode=99%): [22:52:21] nacht ts [22:59:36] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:59:36] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:59:36] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:59:45] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [22:59:45] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [22:59:46] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:59:46] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:59:46] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:59:55] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [22:59:56] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:59:56] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:59:56] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1012587.000000 [22:59:56] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:59:56] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [22:59:56] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:00:05] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:00:06] SMTP on z-dat-s5-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:00:06] FC 0/23 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:00:06] FC 0/14 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:00:06] eth0 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:00:06] SMTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:00:06] SMTP on z-dat-s2-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:00:07] SMTP on z-dat-s1-b is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:00:07] NTP on amaranth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:00:15] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 637723.000000 [23:00:16] FC 0/15 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:00:16] FC 0/3 [cassia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:00:16] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1085065.000000 [23:00:16] NTP on sage is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:00:16] FC 0/4 [hyacinth] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:00:16] FC 0/16 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:00:26] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [23:00:26] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 341952.000000 [23:00:26] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 55032 MB (9% inode=99%): [23:00:26] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:00:26] FC 0/5 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:00:26] FC 0/17 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:00:27] Load avg. on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:00:27] SMF on web.amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:00:27] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [23:00:37] FC 0/18 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:00:37] FC 0/6 [rosemary] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:00:37] FC 0/0 [SAN far1-n1-oe16-esams A1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:00:37] FC 0/1 [far1-n1-oe16-esams B1] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:00:37] FC 0/19 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:00:37] FC 0/7 [daphne] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:00:37] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:00:46] SMF on amaranth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:00:46] aliasd on mayapple is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:02:06] s4 replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7205.000000 [23:02:06] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7202.000000 [23:03:15] wikidata replag on z-dat-s5-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7213.000000 [23:05:16] /sql on z-dat-s1-b is OK: DISK OK - free space: /sql 107104 MB (11% inode=99%): [23:06:36] /mnt on thyme is UNKNOWN: NRPE: Unable to read output [23:06:36] s1 replag on thyme is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on thyme (146) [23:06:36] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 260941 MB (4% inode=62%): [23:06:37] /tmp on thyme is UNKNOWN: NRPE: Unable to read output [23:06:37] MySQL slave on z-dat-s5-b is CRITICAL: (Return code of 139 is out of bounds) [23:06:55] aliasd on nightshade is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:06:56] Load avg. on thyme is UNKNOWN: NRPE: Unable to read output [23:08:05] FMA on thyme is CRITICAL: ERROR - unexpected output from snmpwalk [23:19:16] RAID on thyme is UNKNOWN: NRPE: Unable to read output [23:19:26] / on thyme is UNKNOWN: NRPE: Unable to read output [23:23:45] Environment IPMI on thyme is UNKNOWN: NRPE: Unable to read output [23:59:36] FC 0/2 [hemlock] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:59:36] FC 0/8 [adenia] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:59:36] FC 0/10 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:59:45] MySQL slave on z-dat-s1-b is CRITICAL: (Return code of 139 is out of bounds) [23:59:45] MySQL slave on z-dat-s2-b is CRITICAL: (Return code of 139 is out of bounds) [23:59:46] FC 0/20 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:59:46] FC 0/11 [fsw2-n1-oe16-esams] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:59:46] FC 0/9 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:59:55] FMA on amaranth is CRITICAL: ERROR - unexpected output from snmpwalk [23:59:55] FC 0/12 [thyme] on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:59:55] FC 0/21 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:59:55] NTP on yucca is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:59:55] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 1016186.000000 [23:59:55] FC 0/13 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt. [23:59:55] FC 0/22 on fsw1-n1-oe16-esams.mgmt is UNKNOWN: ERROR: Description table : No response from remote host fsw1-n1-oe16-esams.mgmt.