[00:00:50] Load avg. on willow is CRITICAL: CRITICAL - load average: 37.83, 22.30, 17.91 [00:02:53] Load avg. on willow is WARNING: WARNING - load average: 28.09, 22.30, 18.19 [00:04:39] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [00:10:19] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [00:11:29] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [00:13:19] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [00:48:19] s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1950.000000 [00:48:19] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [00:50:00] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [00:51:39] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [00:51:49] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [00:51:49] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [00:51:49] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 567629 MB (10% inode=49%): [00:52:10] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [00:52:39] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [00:52:39] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [00:52:50] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [00:53:00] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:53:10] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [00:54:10] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [00:55:19] SMTP on yarrow is CRITICAL: Connection refused [00:56:40] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [01:02:49] Load avg. on willow is WARNING: WARNING - load average: 19.93, 19.16, 17.22 [01:04:39] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [01:10:20] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [01:11:29] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [01:13:19] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [01:31:20] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3627.000000 [01:48:19] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [01:50:00] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [01:51:39] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [01:51:59] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [01:51:59] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 567098 MB (10% inode=49%): [01:52:00] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [01:52:19] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [01:52:39] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [01:52:40] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [01:53:01] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:53:49] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [01:54:09] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [01:54:20] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [01:55:20] SMTP on yarrow is CRITICAL: Connection refused [01:56:40] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [02:02:59] Load avg. on willow is WARNING: WARNING - load average: 18.88, 18.90, 18.01 [02:04:40] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [02:10:20] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [02:11:29] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [02:13:19] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [02:31:20] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 5750.000000 [02:48:29] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [02:50:09] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [02:51:40] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [02:51:59] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [02:51:59] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 564324 MB (10% inode=49%): [02:52:00] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [02:52:40] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [02:52:40] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [02:53:09] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:53:19] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [02:54:01] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [02:54:09] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [02:54:20] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [02:56:20] SMTP on yarrow is CRITICAL: Connection refused [02:56:40] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [03:03:00] Load avg. on willow is WARNING: WARNING - load average: 17.36, 17.94, 17.22 [03:04:41] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [03:11:20] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [03:11:29] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [03:14:19] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [03:31:19] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 8140.000000 [03:33:21] 3(commented) [TS-1285] Retrieve data from expired user account <10https://jira.toolserver.org/browse/TS-1285> (Matthew Bowker) [03:48:29] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [03:50:19] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [03:52:09] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [03:52:09] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 564196 MB (10% inode=49%): [03:52:09] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [03:52:39] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [03:52:39] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [03:52:39] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [03:53:09] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:54:09] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [03:54:09] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [03:54:20] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [03:54:20] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [03:57:20] SMTP on yarrow is CRITICAL: Connection refused [03:57:40] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [04:01:09] Load avg. on willow is CRITICAL: CRITICAL - load average: 30.20, 21.89, 18.82 [04:02:09] Load avg. on willow is WARNING: WARNING - load average: 22.80, 21.33, 18.82 [04:03:19] Sun Grid Engine execd on ortelius is WARNING: short@ortelius exceedes load threshold: alarm hl:np_load_short=2.187500/1.00, alarm hl:np_load_long=1.049805/1.50, alarm hl:mem_free=21262.000000M/300M, alarm hl:available=1/0: all.q@ortelius exceedes load threshold: alarm hl:np_load_short=2.187500/1.10, alarm hl:np_load_long=1.049805/1.50, alarm hl:mem_free=21262.000000M/300M, alarm hl:available=1/0 [04:05:19] Sun Grid Engine execd on ortelius is OK: short@ortelius OK: all.q@ortelius OK [04:05:39] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [04:11:10] Load avg. on willow is CRITICAL: CRITICAL - load average: 31.05, 23.00, 20.15 [04:11:29] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [04:12:19] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [04:12:54] [[Special:Log/newusers]] create 10 * Jidanni * (New user account) [04:14:20] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [04:18:09] Load avg. on willow is WARNING: WARNING - load average: 15.53, 19.79, 19.96 [04:21:19] s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3584.000000 [04:22:21] 3(created) [GEOHACK-18] Add Baidu for points in China; GeoHack; Minor Improvement <10https://jira.toolserver.org/browse/GEOHACK-18> (Jidanni) [04:28:21] 3(updated) [GEOHACK-18] Add Baidu for points in China <10https://jira.toolserver.org/browse/GEOHACK-18> (Jidanni) [04:30:21] 3(updated) [GEOHACK-18] HTML that can't even stand a different font size <10https://jira.toolserver.org/browse/GEOHACK-18> (Jidanni) [04:30:22] 3(commented) [GEOHACK-18] HTML that can't even stand a different font size <10https://jira.toolserver.org/browse/GEOHACK-18> (Jidanni) [04:31:19] s4 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1797.000000 [04:46:21] 3(commented) [GEOHACK-18] HTML that can't even stand a different font size <10https://jira.toolserver.org/browse/GEOHACK-18> (Jidanni) [04:49:19] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [04:50:19] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [04:52:09] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [04:52:09] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 564388 MB (10% inode=49%): [04:52:09] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [04:52:39] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [04:52:39] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [04:52:39] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [04:53:20] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:54:09] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [04:54:19] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [04:54:19] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [04:54:29] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [04:56:09] Load avg. on willow is WARNING: WARNING - load average: 15.53, 16.98, 18.27 [04:57:19] SMTP on yarrow is CRITICAL: Connection refused [04:57:40] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [05:05:41] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [05:11:29] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [05:12:20] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [05:14:19] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [05:22:21] 3(created) [GEOHACK-19] URMAP links wrong; GeoHack; Bug <10https://jira.toolserver.org/browse/GEOHACK-19> (Jidanni) [05:29:40] /sql on daphne is CRITICAL: DISK CRITICAL - free space: /aux0 8355 MB (0% inode=99%): [05:41:09] Load avg. on willow is CRITICAL: CRITICAL - load average: 35.54, 25.32, 21.57 [05:49:30] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [05:50:19] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [05:52:09] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [05:52:09] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 562093 MB (10% inode=49%): [05:52:09] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [05:52:40] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [05:52:41] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [05:52:41] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [05:53:20] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:54:10] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [05:54:19] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [05:54:20] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [05:54:21] 3(created) [GEOHACK-20] Google Maps Link Ugly, Long, and even causes two points to display. Why not use the "loc:" parameter instead?; GeoHack; Bug <10https://jira.toolserver.org/browse/GEOHACK-20> (Jidanni) [05:54:31] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [05:55:10] Load avg. on willow is WARNING: WARNING - load average: 17.60, 18.97, 19.99 [05:56:11] Load avg. on willow is CRITICAL: CRITICAL - load average: 23.09, 20.54, 20.50 [05:58:19] SMTP on yarrow is CRITICAL: Connection refused [05:58:39] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [06:06:40] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [06:11:29] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [06:12:19] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [06:14:31] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [06:49:30] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [06:50:20] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [06:52:10] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [06:52:10] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 561811 MB (10% inode=49%): [06:52:10] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [06:53:38] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [06:53:39] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [06:53:40] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [06:54:10] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [06:54:19] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:54:30] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [06:55:18] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [06:55:20] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [06:57:09] Load avg. on willow is CRITICAL: CRITICAL - load average: 20.09, 23.07, 23.51 [06:58:19] SMTP on yarrow is CRITICAL: Connection refused [06:58:39] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [07:06:39] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [07:12:19] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [07:12:29] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [07:14:30] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [07:49:34] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [07:50:23] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [07:52:13] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [07:52:13] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 561039 MB (10% inode=49%): [07:52:13] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [07:53:43] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [07:53:43] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [07:53:43] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [07:54:13] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [07:54:34] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [07:55:24] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:55:24] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [07:56:13] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [07:57:13] Load avg. on willow is CRITICAL: CRITICAL - load average: 18.13, 20.15, 20.79 [07:58:24] SMTP on yarrow is CRITICAL: Connection refused [07:58:43] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [08:06:42] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [08:12:24] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [08:12:33] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [08:15:23] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [08:23:14] Load avg. on willow is WARNING: WARNING - load average: 15.36, 18.07, 19.80 [08:49:35] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [08:50:22] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [08:52:13] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [08:52:13] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 560511 MB (10% inode=49%): [08:52:13] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [08:53:43] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [08:53:43] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [08:53:43] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [08:54:14] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [08:54:34] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [08:55:34] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [08:56:22] 3(created) [TS-1314] mkuser reports errors; Toolserver: Accounts; Bug <10https://jira.toolserver.org/browse/TS-1314> (Daniel Kinzler) [08:56:24] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [08:56:24] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:58:23] SMTP on yarrow is CRITICAL: Connection refused [08:58:43] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [09:00:21] 3(created) [TS-1315] tool for dosplaying basic user info; Toolserver: Accounts; Minor Task <10https://jira.toolserver.org/browse/TS-1315> (Daniel Kinzler) [09:04:20] 3(commented) [TS-1315] tool for dosplaying basic user info <10https://jira.toolserver.org/browse/TS-1315> (MZMcBride) [09:06:42] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [09:12:23] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [09:12:34] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [09:15:34] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [09:23:14] Load avg. on willow is WARNING: WARNING - load average: 16.80, 18.26, 18.91 [09:23:34] Sun Grid Engine execd on ortelius is WARNING: short@ortelius exceedes load threshold: alarm hl:np_load_short=1.800781/1.00, alarm hl:np_load_long=0.858398/1.50, alarm hl:mem_free=21682.000000M/300M, alarm hl:available=1/0: all.q@ortelius exceedes load threshold: alarm hl:np_load_short=1.800781/1.10, alarm hl:np_load_long=0.858398/1.50, alarm hl:mem_free=21682.000000M/300M, alarm hl:available=1/0 [09:25:34] Sun Grid Engine execd on ortelius is OK: short@ortelius OK: all.q@ortelius OK [09:50:22] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [09:50:33] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [09:53:14] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [09:53:14] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 558144 MB (10% inode=49%): [09:53:14] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [09:53:42] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [09:53:42] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [09:53:43] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [09:54:16] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [09:54:35] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [09:55:34] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [09:56:23] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:57:22] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [09:58:23] SMTP on yarrow is CRITICAL: Connection refused [09:58:43] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [10:06:15] Load avg. on willow is CRITICAL: CRITICAL - load average: 22.80, 21.48, 20.27 [10:06:43] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [10:08:13] Load avg. on willow is WARNING: WARNING - load average: 17.31, 19.83, 19.80 [10:11:13] Load avg. on willow is CRITICAL: CRITICAL - load average: 22.59, 20.57, 20.05 [10:12:23] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [10:13:33] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [10:16:34] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [10:31:14] Load avg. on willow is WARNING: WARNING - load average: 20.02, 17.56, 18.10 [10:50:35] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [10:50:35] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [10:53:22] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [10:54:35] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [10:54:35] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [10:54:35] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [10:54:35] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [10:54:35] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 560407 MB (10% inode=49%): [10:54:35] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [10:55:14] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [10:55:35] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [10:56:23] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:57:23] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [10:58:23] SMTP on yarrow is CRITICAL: Connection refused [10:58:43] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [11:02:15] /aux0 on hemlock is OK: DISK OK - free space: /aux0 699613 MB (13% inode=55%): [11:06:44] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [11:12:23] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [11:13:35] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [11:16:35] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [11:31:13] Load avg. on willow is WARNING: WARNING - load average: 23.25, 20.74, 19.48 [11:50:24] Load avg. on willow is CRITICAL: CRITICAL - load average: 26.75, 21.84, 20.29 [11:50:35] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [11:50:43] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [11:53:24] Load avg. on willow is WARNING: WARNING - load average: 18.23, 20.12, 19.88 [11:53:24] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [11:53:44] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [11:53:44] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [11:53:44] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [11:54:26] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [11:54:26] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 40050 MB (4% inode=99%): [11:54:35] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [11:55:24] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [11:56:24] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:56:34] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [11:57:24] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [11:58:25] SMTP on yarrow is CRITICAL: Connection refused [11:58:44] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [12:00:25] Load avg. on willow is CRITICAL: CRITICAL - load average: 31.59, 21.42, 20.12 [12:06:44] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [12:12:34] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [12:14:35] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [12:17:34] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [12:30:24] Load avg. on willow is WARNING: WARNING - load average: 26.55, 20.59, 19.62 [12:43:34] Sun Grid Engine execd on ortelius is WARNING: short@ortelius exceedes load threshold: alarm hl:np_load_short=1.074219/1.00, alarm hl:np_load_long=0.528320/1.50, alarm hl:mem_free=21758.000000M/300M, alarm hl:available=1/0 [12:44:34] Sun Grid Engine execd on ortelius is OK: short@ortelius OK: all.q@ortelius OK [12:50:44] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [12:50:53] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [12:53:36] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [12:53:44] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [12:53:44] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [12:53:44] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [12:54:43] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [12:55:25] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [12:55:35] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [12:56:35] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:56:44] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [12:57:35] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [12:58:35] SMTP on yarrow is CRITICAL: Connection refused [12:58:43] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [13:06:45] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [13:12:35] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [13:14:36] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [13:17:36] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [13:30:26] Load avg. on willow is WARNING: WARNING - load average: 26.28, 20.36, 19.35 [13:33:37] Free Memory on damiana is WARNING: WARNING - 6.8% (283456 kB) free! [13:34:38] Free Memory on damiana is OK: OK - 7.7% (323052 kB) free. [13:50:45] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [13:51:25] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [13:53:37] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [13:53:46] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [13:53:46] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [13:53:46] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [13:54:55] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [13:55:37] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [13:55:37] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [13:56:38] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:56:46] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [13:57:38] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [13:58:38] SMTP on yarrow is CRITICAL: Connection refused [13:58:46] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [14:00:37] Load avg. on willow is CRITICAL: CRITICAL - load average: 31.49, 21.80, 19.84 [14:01:39] Load avg. on willow is WARNING: WARNING - load average: 24.34, 21.52, 19.87 [14:06:47] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [14:10:39] Load avg. on willow is CRITICAL: CRITICAL - load average: 24.92, 20.90, 20.01 [14:12:38] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [14:14:38] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [14:17:38] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [14:21:20] [[Special:Log/newusers]] create 10 * Nosy2 * (New user account) [14:29:38] Free Memory on damiana is WARNING: WARNING - 6.5% (273464 kB) free! [14:31:25] Hello all [14:32:17] hi DaB [14:34:47] Free Memory on damiana is OK: OK - 7.5% (313204 kB) free. [14:36:20] 3(created) [GEOHACK-21] Broken links to the various Chinese versions; GeoHack; Bug <10https://jira.toolserver.org/browse/GEOHACK-21> (Jidanni) [14:38:24] 3(commented) [TS-1309] daphne s2/s5 corrupted and offline <10https://jira.toolserver.org/browse/TS-1309> (DaB.) [14:40:46] [[Special:Log/newusers]] create 10 * Nosy3 * (New user account) [14:50:40] DaBPunkt: just curious for S2, I still see replag only liniair going up, replication still not started? [14:50:47] Load avg. on willow is WARNING: WARNING - load average: 14.12, 13.67, 15.98 [14:50:55] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [14:51:25] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [14:51:48] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1865.000000 [14:52:20] 3(created) [TS-1316] Create MMP 'stimmberechtigung' with ireas, guandalug and files from ~guandalug; Toolserver: Accounts; Task <10https://jira.toolserver.org/browse/TS-1316> (Robin Krahl) [14:53:47] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [14:53:47] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [14:53:47] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [14:53:47] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [14:54:55] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [14:55:47] Load avg. on willow is OK: OK - load average: 12.21, 12.69, 14.88 [14:55:47] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [14:56:36] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [14:56:48] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:56:48] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [14:57:49] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [14:57:49] @replag [14:57:49] DaBPunkt: s2-user: 4d 23h 9m 31s [+0.99 s/s]; s2-user-c: 35m 35s [-]; s3-rr-a: 25s [+0.00 s/s]; s3-user: 25s [+0.00 s/s]; s5-user-c: 35m 35s [+0.01 s/s] [14:58:06] @replag [14:58:06] DaBPunkt: s2-user: 4d 23h 9m 32s [+0.06 s/s]; s2-user-c: 35m 44s [+0.55 s/s]; s3-rr-a: 41s [+0.99 s/s]; s3-user: 41s [+0.99 s/s]; s5-user-c: 35m 44s [+0.56 s/s] [14:58:49] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [14:59:22] 3(commented) [TS-1309] daphne s2/s5 corrupted and offline <10https://jira.toolserver.org/browse/TS-1309> (DaB.) [14:59:36] SMTP on yarrow is CRITICAL: Connection refused [15:01:22] 3(assigned) [TS-1316] Create MMP 'stimmberechtigung' with ireas, guandalug and files from ~guandalug <10https://jira.toolserver.org/browse/TS-1316> (DaB.) [15:06:48] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [15:07:06] [[Toolserver:Homepage]] 10https://wiki.toolserver.org/w/index.php?diff=6699&oldid=6663&rcid=8852 * Nosy * (-7) () [15:12:47] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [15:14:47] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [15:17:47] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [15:23:21] 3(resolved) [TS-1316] Create MMP 'stimmberechtigung' with ireas, guandalug and files from ~guandalug <10https://jira.toolserver.org/browse/TS-1316> (DaB.) [15:25:21] 3(assigned) [TS-1313] More space - double quota for drtrigon <10https://jira.toolserver.org/browse/TS-1313> (DaB.) [15:27:47] Free Memory on damiana is CRITICAL: CRITICAL - 3.9% (161732 kB) free! [15:28:21] 3(resolved) [TS-1313] More space - double quota for drtrigon <10https://jira.toolserver.org/browse/TS-1313> (DaB.) [15:28:46] Free Memory on damiana is OK: OK - 23.7% (991692 kB) free. [15:30:23] 3(assigned) [TS-1312] Add user alexxw to project erfgoed <10https://jira.toolserver.org/browse/TS-1312> (DaB.) [15:34:20] 3(resolved) [TS-1312] Add user alexxw to project erfgoed <10https://jira.toolserver.org/browse/TS-1312> (DaB.) [15:36:28] 3(assigned) [TS-1311] Install Perl module LWP::Protocol::https <10https://jira.toolserver.org/browse/TS-1311> (DaB.) [15:36:28] 3(updated) [TS-1311] Install Perl module LWP::Protocol::https <10https://jira.toolserver.org/browse/TS-1311> (DaB.) [15:39:20] 3(resolved) [TS-1310] Add "seahorseruler" to CVN multi-maintainer group <10https://jira.toolserver.org/browse/TS-1310> (DaB.) [15:43:24] 3(assigned) [TS-1297] Adjust status.toolserver.org <10https://jira.toolserver.org/browse/TS-1297> (DaB.) [15:43:27] 3(commented) [TS-1299] Create user sgeadmin also local on each server <10https://jira.toolserver.org/browse/TS-1299> (DaB.) [15:43:29] 3(commented) [TS-1297] Adjust status.toolserver.org <10https://jira.toolserver.org/browse/TS-1297> (DaB.) [15:46:20] 3(commented) [TS-1289] Give user sgeadmin acces to sql-mapnik <10https://jira.toolserver.org/browse/TS-1289> (DaB.) [15:48:20] 3(assigned) [TS-1286] Please reactivate my account <10https://jira.toolserver.org/browse/TS-1286> (DaB.) [15:48:21] 3(resolved) [TS-1286] Please reactivate my account <10https://jira.toolserver.org/browse/TS-1286> (DaB.) [15:50:20] 3(commented) [TS-1316] Create MMP 'stimmberechtigung' with ireas, guandalug and files from ~guandalug <10https://jira.toolserver.org/browse/TS-1316> (Robin Krahl) [15:50:56] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [15:51:25] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [15:52:46] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3131.000000 [15:52:57] @replag [15:52:57] DaBPunkt: s1-rr-a-c: 7m 36s [+0.00 s/s]; s2-user: 4d 22h 3m 36s [-1.20 s/s]; s2-user-c: 52m 30s [+0.31 s/s]; s3-rr-a: 41s [-]; s3-user: 41s [-]; s5-user-c: 52m 30s [+0.31 s/s]; s6-rr-a: 14s [+0.00 s/s]; s6-user: 14s [+0.00 s/s] [15:53:29] @replag [15:53:29] DaBPunkt: s1-rr-a-c: 7m 53s [+0.53 s/s]; s2-user: 4d 22h 3m 17s [-0.59 s/s]; s2-user-c: 52m 46s [+0.50 s/s]; s3-rr-a: 14s [-0.85 s/s]; s3-user: 14s [-0.85 s/s]; s5-user-c: 52m 46s [+0.50 s/s]; s6-rr-a: 25s [+0.35 s/s]; s6-user: 25s [+0.35 s/s] [15:53:47] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [15:53:47] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [15:53:47] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [15:53:55] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [15:54:55] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [15:56:47] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [15:56:56] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [15:57:35] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [15:57:47] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:58:47] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [15:59:22] 3(updated) [ACCAPP-449] Account Approval for the toolserver withing the RENDER project <10https://jira.toolserver.org/browse/ACCAPP-449> (Gerrit Holz) [15:59:47] SMTP on yarrow is CRITICAL: Connection refused [15:59:47] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [16:05:47] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3627.000000 [16:06:56] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [16:07:22] 3(commented) [TS-1282] We need a shared account for me together with Anselm Metzger from Wikimedia Deutschland. <10https://jira.toolserver.org/browse/TS-1282> (DaB.) [16:09:21] 3(commented) [TS-1281] problem with fr.wiki data base <10https://jira.toolserver.org/browse/TS-1281> (DaB.) [16:10:20] @replag [16:10:20] DaBPunkt: s2-user: 4d 22h 32s [-0.16 s/s]; s2-user-c: 1h 3m 19s [+0.63 s/s]; s3-rr-a: 43s [+0.03 s/s]; s3-user: 43s [+0.03 s/s]; s5-user-c: 1h 3m 19s [+0.63 s/s] [16:12:47] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [16:14:46] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [16:17:47] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [16:41:26] [[Multi-maintainer projects]] 10https://wiki.toolserver.org/w/index.php?diff=6700&oldid=6546&rcid=8853 * Dab * (+345) (/* Creating a new project and adding users */ +Taking over) [16:51:05] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [16:51:25] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [16:53:54] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [16:53:55] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [16:53:56] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [16:54:55] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [16:54:55] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [16:56:55] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [16:56:55] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [16:57:36] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [16:57:47] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:58:47] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [16:59:47] SMTP on yarrow is CRITICAL: Connection refused [16:59:55] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [17:05:19] @replag [17:05:19] DaBPunkt: s1-rr-a-c: 17m 1s [+0.13 s/s]; s1-user: 19s [+0.00 s/s]; s2-user: 4d 21h 32m 39s [-0.51 s/s]; s2-user-c: 1h 26m 4s [+0.41 s/s]; s3-rr-a: 18s [-0.01 s/s]; s3-user: 18s [-0.01 s/s]; s5-user-c: 1h 26m 4s [+0.41 s/s] [17:06:46] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 5033.000000 [17:06:55] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [17:12:05] PING on amaranth is WARNING: PING WARNING - Packet loss = 40%, RTA = 187.00 ms [17:12:05] PING on web.amaranth is WARNING: PING WARNING - Packet loss = 20%, RTA = 184.00 ms [17:13:46] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [17:14:05] PING on web.amaranth is CRITICAL: PING CRITICAL - Packet loss = 60%, RTA = 177.00 ms [17:15:05] PING on web.amaranth is WARNING: PING WARNING - Packet loss = 40%, RTA = 179.00 ms [17:15:05] PING on amaranth is CRITICAL: PING CRITICAL - Packet loss = 60%, RTA = 191.00 ms [17:15:05] jira.toolserver.org on web.amaranth is WARNING: HTTP WARNING: HTTP/1.1 200 OK - 514 bytes in 5.459 second response time [17:15:15] journal.toolserver.org on web.amaranth is WARNING: HTTP WARNING: HTTP/1.1 200 OK - 213 bytes in 13.461 second response time [17:15:46] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [17:16:04] journal.toolserver.org on web.amaranth is OK: HTTP OK: HTTP/1.1 200 OK - 213 bytes in 0.901 second response time [17:16:14] jira.toolserver.org on web.amaranth is CRITICAL: CRITICAL - Socket timeout after 16 seconds [17:17:05] PING on amaranth is WARNING: PING WARNING - Packet loss = 20%, RTA = 177.00 ms [17:17:06] jira.toolserver.org on web.amaranth is OK: HTTP OK: HTTP/1.1 200 OK - 514 bytes in 4.878 second response time [17:17:15] fisheye.toolserver.org on web.amaranth is WARNING: HTTP WARNING: HTTP/1.1 200 OK - 273 bytes in 16.829 second response time [17:17:46] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [17:18:15] fisheye.toolserver.org on web.amaranth is OK: HTTP OK: HTTP/1.1 200 OK - 273 bytes in 13.079 second response time [17:29:45] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3507.000000 [17:30:24] /sql on daphne is CRITICAL: DISK CRITICAL - free space: /aux0 8354 MB (0% inode=99%): [17:38:56] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3620.000000 [17:51:05] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [17:51:25] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [17:52:40] @replag [17:52:40] Thehelpfulone: s2-user: 4d 21h 8m 30s [-0.51 s/s]; s2-user-c: 1h 5m 3s [-0.44 s/s]; s3-rr-a: 35s [+0.01 s/s]; s3-user: 35s [+0.01 s/s]; s5-user-c: 1h 5m 3s [-0.44 s/s] [17:53:56] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [17:53:56] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [17:53:56] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [17:54:56] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [17:54:57] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [17:56:56] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3561.000000 [17:56:56] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [17:56:56] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [17:57:56] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:58:35] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [17:58:56] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [17:59:57] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [18:00:45] SMTP on yarrow is CRITICAL: Connection refused [18:06:57] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [18:11:57] Load avg. on willow is WARNING: WARNING - load average: 24.15, 17.22, 13.04 [18:14:45] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [18:16:45] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [18:18:45] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [18:19:58] Load avg. on willow is OK: OK - load average: 9.38, 14.55, 13.99 [18:19:58] s4 replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1771.000000 [18:23:24] 3(created) [ACCAPP-461] Analysing the development of Bots; Account Approval; New Account <10https://jira.toolserver.org/browse/ACCAPP-461> (Franz Herbach) [18:43:10] [[Kribo]] N 10https://wiki.toolserver.org/w/index.php?oldid=6701&rcid=8854 * Krinkle * (+4001) (Created page with "'''Kribo''' is a small PHP-framework for creating simple IRC bots. By default it has little function but it's power is in the extensibility through plugins. == Documentation == ...") [18:51:15] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [18:51:25] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [18:53:57] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [18:53:57] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [18:53:57] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [18:54:21] 3(created) [ACCAPP-462] i need a account for fa.wikipedia; Account Approval; New Account <10https://jira.toolserver.org/browse/ACCAPP-462> (Darafsh Kaviyani) [18:55:05] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [18:55:57] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [18:56:57] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [18:56:57] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [18:57:58] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:58:35] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [18:58:57] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [18:59:57] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [19:00:45] SMTP on yarrow is CRITICAL: Connection refused [19:02:59] hoe woul i create a bot for #USEagency like on #mediawiki? [19:03:04] *how [19:03:17] *ie david is [19:03:40] I still have no idea what you mean, and you should probably ask it in #mediawiki instead of here [19:04:28] that were 97 keystrokes well-spent [19:05:57] :) [19:07:56] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [19:08:22] who would i create a bot like #mediawiki's mw-bot? [19:08:40] did you even read my last message? [19:09:11] if that is to me, no i lost connection to channel [19:09:23] 20:03 < valhallasw> I still have no idea what you mean, and you should probably ask it in #mediawiki instead of here [19:11:06] * valhallasw wonders how leaving a channel is the same as losing connection [19:11:37] what was the message about creating bot? keep loosing channel? [19:11:50] 20:03 < valhallasw> I still have no idea what you mean, and you should probably ask it in #mediawiki instead of here [19:12:37] valhallasw: And you can spend it over and over again! ;-) [19:12:58] [[WmfDbBot]] N 10https://wiki.toolserver.org/w/index.php?oldid=6702&rcid=8855 * Krinkle * (+1781) (Created page with "{{DISPLAYTITLE:wmfDbBot}} '''wmfDbBot''' is a PHP interface to retrieve information about the databases of Wikimedia Foundation wikis. == Commands == === Info === : info ...") [19:13:04] multichill: arrow-up is awesome [19:13:12] valhallasw: And you can spend it over and over again! ;-) [19:13:24] [[WmfDbBot]] 10https://wiki.toolserver.org/w/index.php?diff=6703&oldid=6702&rcid=8856 * Krinkle * (+2) () [19:13:37] in other news: /ignore tsnag is also pretty awesome :D [19:14:45] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [19:15:43] [[WmfDbBot]] 10https://wiki.toolserver.org/w/index.php?diff=6704&oldid=6703&rcid=8857 * Krinkle * (+448) () [19:15:58] valhallasw: http://nagios.toolserver.org/cgi-bin/status.cgi?host=all&servicestatustypes=28 looks pretty scary. If my Nagios would look like that I wouldn't be going home now ;-) [19:16:46] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [19:17:32] I'm hoping that's just an err in nagios not being up to date with the reality, right ? Stuff generally seems to work right now on TS. [19:18:45] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [19:18:49] Krinkle: a large batch is nightshade being down [19:18:58] * Krinkle always uses willow [19:19:10] aha, ok [19:19:21] as long as there's still redundancy with willow being up, it's not too bad. [19:19:28] !wmfdbbot [19:19:47] although I guess the cronsub stuff will be less spread now, putting more load in willow [19:20:44] 19:20pm up 10 days 6:43, 41 users, load average: 11.73, 11.18, 11.28 [19:20:47] I've seen worse [19:20:58] back when the TS was just a single server.... [19:21:43] also: the all.q is run on three servers (also ortelius and wolfsbane) [19:26:38] multichill: the only real problem on this page is the "SGE on willow"-message [19:27:23] 3(created) [TS-1317] over-quota warn-mail missing / does not work; Toolserver: Accounts; Bug <10https://jira.toolserver.org/browse/TS-1317> (drtrigon) [19:27:24] DaBPunkt: wouldn't it make more sense just to turn off messages that are wrong? [19:27:57] [[Kribo]] 10https://wiki.toolserver.org/w/index.php?diff=6705&oldid=6701&rcid=8858 * Krinkle * (-67) () [19:28:11] valhallasw: normaly you acknowledge such messages and fine – but our nagios needs some changings to allow that [19:28:12] tsnag is a bit like "the boy that cried wolf" now ;-) [19:28:18] ah, I see [19:28:23] [[Kribo]] 10https://wiki.toolserver.org/w/index.php?diff=6706&oldid=6705&rcid=8859 * Krinkle * (+31) (+[[Category:Tools by Krinkle]]) [19:28:31] [[WmfDbBot]] 10https://wiki.toolserver.org/w/index.php?diff=6707&oldid=6704&rcid=8860 * Krinkle * (+30) (+[[Category:Tools by Krinkle]]) [19:29:13] valhallasw: the bot is not that bad – I have a old cellphone and the nagios-sms-service filled my sms-space complete ;) [19:29:22] completly [19:29:25] :D [19:50:55] s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1861.000000 [19:51:20] @replag [19:51:20] DaBPunkt: s1-rr-a-c: 31m 21s [+0.09 s/s]; s2-user: 4d 20h 13m 13s [-0.47 s/s]; s3-rr-a: 24s [-0.00 s/s]; s3-user: 24s [-0.00 s/s] [19:51:25] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [19:51:25] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [19:54:05] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [19:54:05] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [19:54:05] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [19:55:05] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [19:56:05] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [19:57:05] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [19:57:05] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [19:58:05] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:59:05] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [19:59:35] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [20:00:41] DaBPunkt: whem will all databases be avaiable? Any prediction? [20:00:45] SMTP on yarrow is CRITICAL: Connection refused [20:00:55] Alchimista: whcih are missig? [20:00:56] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [20:01:01] https://toolserver.org/~pathoschild/accounteligibility/?user=Waithamai&wiki=&event=23 [20:01:12] "Exception: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'enwiktionary_p.user' doesn't exist" [20:02:21] DaBPunkt: ^ [20:02:27] oh, the table is missing [20:02:32] not good [20:07:55] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [20:09:21] 3(commented) [TS-1309] daphne s2/s5 corrupted and offline <10https://jira.toolserver.org/browse/TS-1309> (DaB.) [20:11:20] Alchimista: fixed. Was a minor problem at the end :) [20:11:21] 3(commented) [TS-1309] daphne s2/s5 corrupted and offline <10https://jira.toolserver.org/browse/TS-1309> (DaB.) [20:11:37] DaBPunkt: thanks! [20:13:20] 3(commented) [TS-1309] daphne s2/s5 corrupted and offline <10https://jira.toolserver.org/browse/TS-1309> (DaB.) [20:14:45] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [20:16:45] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [20:18:46] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [20:20:56] Hi does anyone have any updates on Nightshade being down? https://jira.toolserver.org/browse/MNT-1198 [20:22:15] hello Robbie [20:22:47] hello [20:23:50] Robbie: will be down for 1 more week minimum [20:24:05] but why do you need exactly this node? [20:24:20] 3(commented) [MNT-1198] nightshade down because of missing disc <10https://jira.toolserver.org/browse/MNT-1198> (DaB.) [20:24:42] DaBPunkt: if brought up, will cron be disabled for a while to prevent dual cron runs? [20:25:15] Akoopal: the cron-files on nightshade are gone, so there will be no dual-runs [20:25:17] I have written an application that relies on the word of the day RSS feed. The WOTD feed relies on nightshade it seems. I was hoping it would come back up so the service would be restored. Looks like I'm going to have to try and write my own RSS feed for WOTD. [20:25:34] DaBPunkt: ok, clear [20:25:41] thanks for the info [20:25:42] And if the cron jobs are all gone then the RSS feed will never be back [20:26:08] Robbie: that depens if the tool-author will come back or not [20:26:26] I have tried contacting them and no response, so I'm guessing they don't maintain the account [20:26:38] How difficult would it be for me to make a toolserver account and pull the WOTD out of the wiktionary database? [20:26:39] if not, his/her account will be diabled within 6 months automaticaly [20:27:20] I have no idea how difficult it is for you ;) [20:27:49] well generally, assuming I know PHP and mysql well enough to do this normally [20:28:00] on my own web server [20:28:12] does toolserver give you access to the WOTD in the database do you know? [20:28:30] or generally wiktionary pages [20:29:36] Robbie: the toolserver offers access to the meta-data of every public wmf-project. We have no article-text, but anything else [20:30:09] what exactly is the meta data? [20:31:01] anything beside the article-text. Like the history, categories, template-links, interwiki-links, user-data etc. pp [20:31:59] hmm ok it probably won't have what I want in the database then [20:33:08] I guess what I need is access to the archive data http://en.wiktionary.org/wiki/Wiktionary:Word_of_the_day/Archive [20:33:20] there have been many RSS feeds over time it seems but they're now all broken [20:33:26] the last remaining one relied on nightshade [20:43:56] s4 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1736.000000 [20:44:55] @replag [20:44:55] DaBPunkt: s1-rr-a-c: 30m 1s [-0.02 s/s]; s2-user: 4d 18h 26m 21s [-1.99 s/s]; s3-rr-a: 42s [+0.01 s/s]; s3-user: 42s [+0.01 s/s] [20:47:55] s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1843.000000 [20:51:26] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [20:51:26] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [20:54:05] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [20:54:05] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [20:54:15] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [20:55:15] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [20:56:15] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [20:57:15] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [20:57:15] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [20:58:05] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:59:15] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [20:59:35] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [21:00:46] SMTP on yarrow is CRITICAL: Connection refused [21:01:55] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [21:08:56] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [21:09:56] s4 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1669.000000 [21:14:46] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [21:16:22] [[Special:Log/newusers]] create 10 * Jpgroß * (New user account) [21:16:45] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [21:18:46] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [21:32:25] @replag [21:32:25] DaBPunkt: s1-rr-a-c: 12m 33s [-0.37 s/s]; s2-user: 4d 14h 36m 51s [-4.83 s/s]; s3-rr-a: 31s [-0.00 s/s]; s3-user: 31s [-0.00 s/s] [21:47:46] SMTP on yarrow is OK: SMTP OK - 0.018 sec. response time [21:51:35] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [21:51:46] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [21:54:06] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [21:54:06] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [21:54:25] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [21:55:16] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [21:56:15] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [21:57:15] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [21:57:25] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [21:58:05] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:59:15] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [21:59:46] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [22:01:55] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [22:06:56] SMTP on yarrow is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:09:55] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [22:14:48] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [22:16:47] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [22:18:46] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [22:29:02] @replag [22:29:02] DaBPunkt: s1-rr-a-c: 17m 34s [+0.09 s/s]; s2-user: 4d 12h 48m 9s [-1.92 s/s]; s3-rr-a: 39s [+0.00 s/s]; s3-user: 39s [+0.00 s/s]; s5-rr-a: 11m 46s [+0.00 s/s]; s5-user: 11m 46s [+0.00 s/s] [22:37:47] SMTP on yarrow is OK: SMTP OK - 0.019 sec. response time [22:49:55] s5 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1943.000000 [22:51:47] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [22:52:35] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [22:54:06] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [22:54:06] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [22:54:25] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [22:56:15] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [22:56:15] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [22:56:20] 3(commented) [TS-1315] tool for dosplaying basic user info <10https://jira.toolserver.org/browse/TS-1315> (Daniel Kinzler) [22:57:15] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [22:57:25] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [22:58:06] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:59:26] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [22:59:47] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [23:01:55] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [23:06:01] @replag [23:06:01] DaBPunkt: s1-rr-a-c: 42s [-0.46 s/s]; s2-user: 4d 11h 39m 38s [-1.85 s/s]; s3-rr-a: 15s [-0.01 s/s]; s3-user: 15s [-0.01 s/s]; s5-rr-a: 48m 31s [+0.99 s/s]; s5-user: 48m 31s [+0.99 s/s] [23:10:07] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [23:14:50] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [23:16:49] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [23:18:08] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3628.000000 [23:18:48] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [23:51:56] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [23:52:48] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [23:54:07] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [23:54:07] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [23:54:47] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [23:55:06] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 39308 MB (4% inode=99%): [23:56:50] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [23:56:50] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [23:57:49] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [23:57:49] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [23:58:08] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:59:48] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [23:59:48] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146)