[00:00:10] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [00:02:00] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [00:04:11] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [00:04:12] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [00:44:19] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [00:45:39] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [00:46:00] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [00:46:40] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [00:47:59] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:48:09] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [00:48:09] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [00:48:29] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [00:49:00] SMTP on yarrow is CRITICAL: Connection refused [00:49:00] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [00:49:10] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [00:49:10] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [00:49:19] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [00:50:00] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [01:00:10] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [01:03:01] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [01:05:09] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [01:05:09] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [01:25:49] s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1908.000000 [01:44:29] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [01:46:00] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [01:46:40] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [01:46:49] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [01:47:59] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:48:10] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [01:48:10] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [01:48:30] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [01:49:00] SMTP on yarrow is CRITICAL: Connection refused [01:49:00] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [01:49:10] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [01:49:10] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [01:49:30] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [01:50:00] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [01:55:49] s4 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1705.000000 [02:00:09] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [02:02:19] Load avg. on willow is WARNING: WARNING - load average: 17.56, 15.21, 12.69 [02:03:19] Load avg. on willow is OK: OK - load average: 14.34, 14.72, 12.67 [02:04:02] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [02:05:09] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [02:05:09] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [02:12:19] Load avg. on willow is WARNING: WARNING - load average: 17.21, 15.80, 13.96 [02:12:49] s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1840.000000 [02:22:48] s4 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1573.000000 [02:44:30] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [02:45:59] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [02:46:40] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [02:46:49] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [02:48:00] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:48:59] SMTP on yarrow is CRITICAL: Connection refused [02:48:59] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [02:49:09] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [02:49:09] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [02:49:30] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [02:50:00] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [02:50:09] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [02:50:10] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [02:50:29] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [03:01:08] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [03:04:59] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [03:06:09] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [03:06:09] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [03:32:20] Load avg. on willow is WARNING: WARNING - load average: 15.82, 16.11, 14.59 [03:45:30] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [03:46:00] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [03:46:49] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [03:46:49] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [03:47:20] Load avg. on willow is OK: OK - load average: 14.34, 14.93, 14.89 [03:48:00] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:48:59] SMTP on yarrow is CRITICAL: Connection refused [03:49:01] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [03:49:09] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [03:49:10] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [03:49:29] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [03:50:00] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [03:50:09] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [03:51:09] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [03:51:29] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [04:01:09] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [04:02:19] Load avg. on willow is WARNING: WARNING - load average: 19.52, 17.99, 15.90 [04:05:01] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [04:06:09] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [04:06:10] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [04:23:20] Load avg. on willow is OK: OK - load average: 12.78, 14.00, 14.86 [04:45:30] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [04:46:00] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [04:46:49] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [04:47:01] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [04:48:00] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:49:01] SMTP on yarrow is CRITICAL: Connection refused [04:49:01] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [04:49:10] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [04:49:10] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [04:49:29] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [04:49:59] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [04:50:09] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [04:51:09] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [04:51:29] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [05:01:10] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [05:02:19] Load avg. on willow is WARNING: WARNING - load average: 16.91, 15.03, 13.48 [05:03:20] Load avg. on willow is OK: OK - load average: 13.80, 14.51, 13.40 [05:05:01] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [05:06:09] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [05:06:09] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [05:12:19] Load avg. on willow is WARNING: WARNING - load average: 16.77, 16.29, 14.49 [05:29:10] /sql on daphne is CRITICAL: DISK CRITICAL - free space: /aux0 8356 MB (0% inode=99%): [05:45:30] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [05:46:09] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [05:46:50] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [05:47:01] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [05:48:01] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:49:01] SMTP on yarrow is CRITICAL: Connection refused [05:49:01] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [05:49:09] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [05:49:29] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [05:50:00] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [05:50:10] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [05:50:10] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [05:51:09] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [05:51:29] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [06:02:10] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [06:02:19] Load avg. on willow is WARNING: WARNING - load average: 20.55, 17.82, 15.18 [06:06:00] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [06:06:10] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [06:06:10] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [06:08:19] Load avg. on willow is OK: OK - load average: 12.34, 15.00, 14.81 [06:12:19] Load avg. on willow is WARNING: WARNING - load average: 16.86, 15.61, 15.07 [06:35:30] Hey, is there any news on the status of nightshade being down? [06:46:10] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [06:46:29] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [06:46:49] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [06:48:00] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [06:48:09] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:49:09] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [06:49:09] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [06:49:29] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [06:50:00] SMTP on yarrow is CRITICAL: Connection refused [06:50:09] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [06:50:09] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [06:50:09] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [06:51:20] 3(commented) [TS-1309] daphne s2/s5 corrupted and offline <10https://jira.toolserver.org/browse/TS-1309> (Marlen Caemmerer) [06:51:29] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [06:52:09] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [07:02:09] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [07:05:59] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [07:07:09] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [07:07:09] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [07:46:09] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [07:46:28] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [07:46:59] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [07:47:59] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [07:48:09] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:49:09] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [07:49:09] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [07:49:29] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [07:49:59] SMTP on yarrow is CRITICAL: Connection refused [07:50:10] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [07:50:10] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [07:50:10] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [07:51:20] Load avg. on willow is CRITICAL: CRITICAL - load average: 17.59, 18.62, 20.41 [07:51:28] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [07:53:09] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [07:53:19] Load avg. on willow is WARNING: WARNING - load average: 15.48, 17.46, 19.75 [08:00:20] Load avg. on willow is CRITICAL: CRITICAL - load average: 32.14, 19.29, 19.25 [08:02:09] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [08:06:59] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [08:07:09] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [08:07:09] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [08:07:20] Load avg. on willow is WARNING: WARNING - load average: 17.40, 19.73, 19.89 [08:46:10] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [08:46:29] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [08:47:00] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [08:48:00] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [08:48:09] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:49:10] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [08:49:20] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [08:49:29] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [08:50:00] SMTP on yarrow is CRITICAL: Connection refused [08:50:09] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [08:50:09] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [08:50:09] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [08:51:29] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [08:53:10] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [09:03:10] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [09:07:00] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [09:07:21] Load avg. on willow is WARNING: WARNING - load average: 14.62, 16.03, 16.41 [09:08:09] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [09:08:09] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [09:19:26] 3(created) [TS-1312] Add user alexxw to project erfgoed; Toolserver: Accounts; Task <10https://jira.toolserver.org/browse/TS-1312> (Multichill) [09:32:21] hi [09:32:57] anybody know if nightshade still exists? [09:33:45] i can't connect to it with ssh [09:45:27] I see no errors from nightshade [09:45:40] in my log of this channel [09:46:09] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [09:46:29] Load avg. on willow is OK: OK - load average: 14.15, 14.27, 14.97 [09:46:39] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [09:47:10] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [09:48:00] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [09:48:20] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:49:20] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [09:49:20] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [09:49:29] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [09:49:59] SMTP on yarrow is CRITICAL: Connection refused [09:50:20] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [09:50:20] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [09:51:09] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [09:51:29] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [09:53:20] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [10:02:20] Load avg. on willow is WARNING: WARNING - load average: 16.45, 15.07, 14.41 [10:03:21] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [10:04:30] Load avg. on willow is OK: OK - load average: 14.39, 14.73, 14.37 [10:07:00] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [10:09:09] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [10:09:09] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [10:13:47] ToAruShiroiNeko: how do you use nightshade? [10:14:18] ssh: connect to host nightshade.toolserver.org port 22: Network is unreachable [10:14:25] here what i have ^ [10:16:46] btw is there irc logs? [10:17:54] somewhere over the cloud [10:19:13] bayo_O: reading the topic might help you [10:19:25] Wikimedia Toolserver | Status: broken: login server nightshade, s2 | Wiki: https://wiki.toolserver.org | New accounts: http://bit.ly/4E9nR0 | Channel logs: http://bit.ly/dDoAjS | SQL query request service: http://bit.ly/9fqWG1 | Replag: say "@replag" or see graphs: http://bit.ly/a7F74v [10:19:32] * valhallasw points at "Channel logs" ;-) [10:26:57] thanks [10:27:14] my stupid irc client do not display it :-/ [10:30:29] Load avg. on willow is WARNING: WARNING - load average: 17.42, 14.41, 15.73 [10:34:30] Load avg. on willow is OK: OK - load average: 11.19, 13.07, 14.91 [10:46:39] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [10:47:10] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [10:47:10] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [10:48:28] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:49:00] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [10:49:29] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [10:49:40] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [10:49:40] Free Memory on damiana is WARNING: WARNING - 6.1% (254720 kB) free! [10:49:59] SMTP on yarrow is CRITICAL: Connection refused [10:50:19] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [10:51:10] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [10:51:19] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [10:51:19] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [10:51:39] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [10:53:49] valhallasw as in I do not see it fixed [10:53:52] * ToAruShiroiNeko stabs valhallasw [10:54:15] valhallasw if you have time, could I ask you to run a query for me? [10:54:19] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [10:54:44] I want to know how many article-space content is deleted in en.wikipedia and commons [10:54:58] also how many files are deleted in commons [11:02:29] Load avg. on willow is WARNING: WARNING - load average: 16.97, 16.91, 15.11 [11:02:40] Free Memory on damiana is CRITICAL: CRITICAL - 4.7% (196088 kB) free! [11:04:19] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [11:04:40] Free Memory on damiana is WARNING: WARNING - 5.6% (232432 kB) free! [11:07:00] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [11:08:55] ToAruShiroiNeko: you killed me, I'm dead [11:10:10] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [11:10:10] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [11:10:32] ToAruShiroiNeko: also: if you have the query - yes. If not - no. [11:27:39] Free Memory on damiana is CRITICAL: CRITICAL - 3.4% (144344 kB) free! [11:28:39] Free Memory on damiana is OK: OK - 8.6% (360060 kB) free. [11:46:39] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [11:48:09] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [11:48:09] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [11:48:29] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:49:00] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [11:49:24] 3(commented) [TS-1309] daphne s2/s5 corrupted and offline <10https://jira.toolserver.org/browse/TS-1309> (Marlen Caemmerer) [11:50:18] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [11:50:40] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [11:50:40] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [11:50:59] SMTP on yarrow is CRITICAL: Connection refused [11:51:19] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [11:51:19] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [11:51:39] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [11:52:09] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [11:53:29] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 39973 MB (4% inode=99%): [11:54:19] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [12:00:29] Load avg. on willow is CRITICAL: CRITICAL - load average: 33.81, 20.01, 17.18 [12:01:30] Load avg. on willow is WARNING: WARNING - load average: 22.03, 19.12, 17.05 [12:04:19] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [12:07:00] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [12:10:10] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [12:10:19] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [12:28:29] Load avg. on willow is OK: OK - load average: 11.69, 13.52, 14.81 [12:46:40] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [12:48:09] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [12:48:29] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [12:48:29] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:49:09] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [12:50:19] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [12:50:39] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [12:51:00] SMTP on yarrow is CRITICAL: Connection refused [12:51:19] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [12:51:20] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [12:51:29] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [12:51:40] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [12:52:19] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [12:54:19] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [12:59:59] I think this is the right channel, sorry if it's not. But is there a possibility to search for wikimedia images using a timestamp? [13:00:23] Being an EXIF creation date timestamp [13:00:40] (Or any other exif data really) [13:02:29] Load avg. on willow is WARNING: WARNING - load average: 16.82, 17.03, 15.16 [13:04:19] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [13:05:09] Rinsmaster: You can probably pull that from the image table, but it will be slow [13:05:38] See https://www.mediawiki.org/wiki/Manual:Image_table [13:05:57] multichill: Ah yea, I was just looking into the API documentation [13:07:00] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [13:08:40] Load avg. on willow is OK: OK - load average: 13.17, 14.67, 14.69 [13:10:10] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [13:10:19] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [13:12:40] Load avg. on willow is WARNING: WARNING - load average: 16.00, 15.46, 15.01 [13:13:50] multichill: From what I understand, my only option would be to request a list like "/w/api.php?action=query&list=allimages&aiprop=metadata", and then check them all one by one? [13:15:12] Rinsmaster: for questions on how to use the api, you might find more knowlegeable people in #mediawiki-api [13:15:43] Ah okay, thanks, I will try there :) [13:16:52] valhallasw: Hm, would that be on Freenode? [13:17:08] er [13:17:29] try #mediawiki then :-) [13:17:38] sorry, I thought there was an api-specific channel [13:18:06] Hah, no prob, I'll look over there :) [13:46:40] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [13:48:18] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [13:48:40] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:48:40] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [13:49:10] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [13:50:18] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [13:50:40] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [13:51:09] SMTP on yarrow is CRITICAL: Connection refused [13:51:19] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [13:51:29] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [13:51:40] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [13:51:40] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [13:52:29] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [13:54:19] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [14:02:40] Load avg. on willow is WARNING: WARNING - load average: 15.17, 15.62, 15.12 [14:04:30] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [14:04:40] Load avg. on willow is OK: OK - load average: 12.87, 14.54, 14.77 [14:07:09] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [14:10:09] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [14:10:29] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [14:12:41] Load avg. on willow is WARNING: WARNING - load average: 16.72, 15.92, 15.24 [14:46:50] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [14:48:20] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [14:49:18] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [14:49:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:49:40] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [14:50:19] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [14:50:39] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [14:51:09] SMTP on yarrow is CRITICAL: Connection refused [14:51:18] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [14:51:29] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [14:51:40] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [14:51:50] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [14:52:29] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [14:54:19] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [14:57:15] CRITICAL: ERROR! CRITICAL: ERROR! CRITICAL: ERROR! CRITICAL: ERROR! CRITICAL: ERROR! [14:57:22] * ToAruShiroiNeko panics [14:57:51] SMF = Super-Mother-Faker? [15:02:40] Load avg. on willow is WARNING: WARNING - load average: 16.38, 15.44, 14.14 [15:04:30] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [15:06:41] Load avg. on willow is OK: OK - load average: 13.96, 14.88, 14.21 [15:08:09] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [15:10:09] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [15:10:28] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [15:12:40] Load avg. on willow is WARNING: WARNING - load average: 17.34, 16.02, 14.81 [15:42:51] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 38929 MB (9% inode=99%): [15:43:35] Hello, anybody out there who could help using pywikipediabot on toolserver as a crontab? It seems like in #pywikipediabot there is nobody active arround. :( [15:45:49] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 84955 MB (21% inode=99%): [15:46:50] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [15:48:29] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [15:49:19] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [15:49:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:49:39] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [15:50:19] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [15:50:40] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [15:51:09] SMTP on yarrow is CRITICAL: Connection refused [15:51:19] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [15:51:28] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [15:51:50] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [15:52:29] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [15:52:40] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [15:54:19] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [16:00:49] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 35853 MB (8% inode=99%): [16:02:40] Load avg. on willow is WARNING: WARNING - load average: 16.12, 16.12, 14.80 [16:04:29] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [16:04:49] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 22107 MB (5% inode=99%): [16:08:09] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [16:08:39] Load avg. on willow is OK: OK - load average: 13.48, 14.91, 14.70 [16:10:19] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [16:10:29] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [16:12:40] Load avg. on willow is WARNING: WARNING - load average: 17.27, 15.87, 15.11 [16:13:50] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 146988 MB (36% inode=99%): [16:46:50] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [16:48:29] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [16:50:19] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [16:50:19] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [16:50:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:50:39] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [16:50:39] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [16:51:09] SMTP on yarrow is CRITICAL: Connection refused [16:51:19] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [16:51:29] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [16:51:50] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [16:52:29] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [16:53:40] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [16:55:19] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [17:02:40] Load avg. on willow is WARNING: WARNING - load average: 15.55, 15.93, 14.73 [17:04:30] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [17:07:40] Load avg. on willow is OK: OK - load average: 13.38, 14.94, 14.65 [17:08:09] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [17:10:19] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [17:10:29] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [17:12:39] Load avg. on willow is WARNING: WARNING - load average: 16.27, 15.33, 14.83 [17:29:29] /sql on daphne is CRITICAL: DISK CRITICAL - free space: /aux0 8356 MB (0% inode=99%): [17:29:29] Hello all [17:30:22] * matthewrbowker waves to DaBPunkt [17:46:37] hello DaBPunkt [17:46:39] busy? [17:46:51] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [17:47:07] ToAruShiroiNeko: not realy [17:47:48] oh ko then [17:48:39] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [17:50:29] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [17:50:29] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [17:50:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:50:40] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [17:50:51] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [17:51:09] SMTP on yarrow is CRITICAL: Connection refused [17:51:12] ToAruShiroiNeko: I am not realy busy. So what do you want? :) [17:51:30] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [17:51:30] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [17:51:31] oh [17:51:50] well I was wondering if you could run two cuerries to establish the total number of deleted pages in en.wikipedia and on commons [17:51:51] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [17:52:10] as a bonus you can perhaps also run it on de.wikipedia and other wikis in the top 10 [17:52:25] Deleted pages is kind of an ambiguous count. [17:52:29] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [17:52:30] What if a page was deleted 20 times? [17:52:34] Does that count as 1 or 20? [17:53:39] Joan: as "why the hell did the admins not block the article?" ;) [17:53:40] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [17:53:50] Haha [17:54:03] Well, it used to be much harder to do create-protections. ;-) [17:54:48] ToAruShiroiNeko: arictles or all pages? [17:55:29] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [17:56:32] @replag [17:56:32] matthewrbowker: s1-rr-a-c: 36s [-0.00 s/s]; s2-user: 4d 2h 16m 59s [-]; s3-rr-a: 17s [+0.00 s/s]; s3-user: 17s [+0.00 s/s] [18:00:27] 3(created) [ACCAPP-460] I want run a robot for fawiki, and im want mostly to faster doing job.; Account Approval; New Account <10https://jira.toolserver.org/browse/ACCAPP-460> (Mahdi haji) [18:01:59] / on willow is WARNING: DISK WARNING - free space: / 23321 MB (20% inode=99%): [18:04:29] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [18:08:09] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [18:08:21] 3(commented) [TS-1309] daphne s2/s5 corrupted and offline <10https://jira.toolserver.org/browse/TS-1309> (DaB.) [18:10:19] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [18:11:29] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [18:11:40] Load avg. on willow is WARNING: WARNING - load average: 20.89, 18.52, 16.95 [18:14:51] how do i stope cronie from sending me cron emails? :/ [18:17:46] adam_: pipe output to /dev/null [18:17:54] e.g. command > /dev/null [18:18:21] That'll only redirect stdout, though. [18:18:22] Not stderr. [18:18:25] Or something. [18:18:36] That's usually a good thing. [18:19:02] hmm [18:19:12] already got the > /dev/null [18:19:24] using cronsub as well, anything else special? [18:19:25] Are you using print or echo in your scripts? [18:19:37] What is cronie actually sending you? [18:19:51] /opt/local/bin/cronsub[51]: php: cannot open [18:19:55] although the scripts still run [18:20:06] Right. Probably makes sense to fix that error rather than trying to ignore it. ;-) [18:21:11] You're doing something like... "php script.php"? [18:21:23] It sounds like cronsub can't figure out where php is. You can usually just specify it. [18:21:44] Like... /usr/bin/php script.php. [18:46:59] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [18:47:36] @replag [18:47:36] Akoopal: s1-rr-a-c: 2m 10s [+0.03 s/s]; s2-user: 4d 3h 8m 3s [+1.00 s/s] [18:48:39] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [18:50:29] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [18:50:29] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [18:50:50] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [18:50:50] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:51:30] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [18:51:30] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [18:51:50] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [18:51:59] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [18:52:09] SMTP on yarrow is CRITICAL: Connection refused [18:52:29] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [18:53:50] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [18:55:29] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [18:56:29] Is toolserver always this... errory? [18:56:59] You're talking about the bot? [18:57:38] yes [18:57:40] Life significantly improves if you ignore it. It's a steady stream of noise. [19:00:11] I really should configure irssi to ignore it [19:00:30] it's a big fat white 3 continuously demanding attention [19:01:20] I generally hate ignoring users on IRC. But that bot is just too fucking much. [19:02:01] / on willow is WARNING: DISK WARNING - free space: / 23261 MB (20% inode=99%): [19:02:11] I find it quite interesting to follow. [19:02:31] Perhaps it should tweet. [19:02:32] Oh, darn. Willow has 23 gigs of free space left... :P [19:02:43] I don't think that's possible with Nagios [19:02:49] To tweet? [19:02:58] You could easily write a script to do it, one way or another. [19:03:07] True... [19:03:43] Twitter requires OAuth nowadays. [19:03:45] [[Special:Log/newusers]] create 10 * Babatolian * (New user account) [19:03:48] Which makes things a bit more annoying. [19:03:53] But it's simple enough. [19:04:30] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [19:06:09] / on willow is OK: DISK OK - free space: / 24184 MB (21% inode=99%): [19:08:09] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [19:10:18] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [19:11:28] 3(commented) [TS-1309] daphne s2/s5 corrupted and offline <10https://jira.toolserver.org/browse/TS-1309> (Marlen Caemmerer) [19:11:29] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [19:11:49] Load avg. on willow is WARNING: WARNING - load average: 24.29, 21.35, 18.52 [19:16:21] 3(created) [TS-1313] More space - double quota for drtrigon; Toolserver: Accounts; Task <10https://jira.toolserver.org/browse/TS-1313> (drtrigon) [19:47:11] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [19:48:39] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [19:50:29] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [19:50:38] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [19:50:51] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [19:50:51] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:51:29] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [19:51:29] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [19:51:49] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [19:51:49] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 581444 MB (10% inode=50%): [19:51:59] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [19:52:10] SMTP on yarrow is CRITICAL: Connection refused [19:52:39] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [19:53:50] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [19:55:28] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [20:00:50] Load avg. on willow is CRITICAL: CRITICAL - load average: 30.13, 20.01, 17.53 [20:01:50] Load avg. on willow is WARNING: WARNING - load average: 22.43, 19.63, 17.55 [20:03:55] nighty~ [20:04:29] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [20:08:09] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [20:10:19] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [20:11:29] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [20:29:51] Load avg. on willow is OK: OK - load average: 11.91, 13.34, 14.86 [20:32:50] Load avg. on willow is WARNING: WARNING - load average: 13.54, 14.52, 15.15 [20:33:50] Load avg. on willow is OK: OK - load average: 12.44, 14.02, 14.93 [20:42:00] Sun Grid Engine execd on ortelius is WARNING: short@ortelius exceedes load threshold: alarm hl:np_load_short=1.130859/1.00, alarm hl:np_load_long=0.634765/1.50, alarm hl:mem_free=21869.000000M/300M, alarm hl:available=1/0: all.q@ortelius exceedes load threshold: alarm hl:np_load_short=1.130859/1.10, alarm hl:np_load_long=0.634765/1.50, alarm hl:mem_free=21869.000000M/300M, alarm hl:available=1/0 [20:42:59] Sun Grid Engine execd on ortelius is OK: short@ortelius OK: all.q@ortelius OK [20:47:10] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [20:48:50] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [20:50:39] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [20:50:59] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:51:28] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [20:51:50] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [20:51:50] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 577002 MB (10% inode=50%): [20:51:50] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [20:52:00] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [20:52:29] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [20:52:29] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [20:52:39] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [20:53:09] SMTP on yarrow is CRITICAL: Connection refused [20:53:50] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [20:55:30] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [21:02:51] Load avg. on willow is WARNING: WARNING - load average: 16.27, 15.99, 14.88 [21:04:30] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [21:08:50] Load avg. on willow is OK: OK - load average: 13.21, 14.81, 14.74 [21:09:10] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [21:10:19] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [21:11:29] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [21:12:51] Load avg. on willow is WARNING: WARNING - load average: 15.81, 15.41, 14.99 [21:36:50] Load avg. on willow is WARNING: WARNING - load average: 12.98, 14.63, 15.82 [21:42:50] Load avg. on willow is OK: OK - load average: 12.82, 13.50, 14.87 [21:47:10] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [21:48:50] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [21:50:39] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [21:51:29] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [21:51:50] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [21:51:50] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 576910 MB (10% inode=50%): [21:51:50] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:51:51] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [21:52:09] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [21:52:29] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [21:52:29] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [21:52:39] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [21:53:51] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [21:54:10] SMTP on yarrow is CRITICAL: Connection refused [21:56:29] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [22:02:50] Load avg. on willow is WARNING: WARNING - load average: 15.00, 15.21, 14.44 [22:03:50] Load avg. on willow is OK: OK - load average: 13.82, 14.82, 14.35 [22:04:29] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [22:09:10] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [22:11:20] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [22:11:29] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [22:47:19] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [22:48:49] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [22:50:49] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [22:51:29] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [22:51:50] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [22:51:50] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 566872 MB (10% inode=49%): [22:51:59] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [22:51:59] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:52:09] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [22:52:29] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [22:52:29] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [22:52:39] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [22:54:00] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [22:54:19] SMTP on yarrow is CRITICAL: Connection refused [22:56:28] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [23:02:49] Load avg. on willow is WARNING: WARNING - load average: 15.74, 16.51, 15.21 [23:04:39] Cluster on yarrow is UNKNOWN: NRPE: Unable to read output [23:08:50] Load avg. on willow is OK: OK - load average: 13.37, 14.72, 14.83 [23:09:20] SMF on yarrow is UNKNOWN: NRPE: Unable to read output [23:11:28] Environment on yarrow is UNKNOWN: NRPE: Unable to read output [23:12:20] Sun Grid Engine execd on yarrow is UNKNOWN: NRPE: Unable to read output [23:12:50] Load avg. on willow is WARNING: WARNING - load average: 17.09, 16.21, 15.43 [23:46:49] Load avg. on willow is WARNING: WARNING - load average: 16.52, 15.65, 15.48 [23:47:20] FC 0/6 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/6:UP: 1 int NOK : CRITICAL [23:49:01] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [23:50:49] FC 0/7 on fsw1-n1-oe16-esams.mgmt is CRITICAL: FC port 0/7:UP: 1 int NOK : CRITICAL [23:51:29] s2 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [23:51:50] MySQL slave on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [23:51:50] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 566660 MB (10% inode=49%): [23:52:09] SMF on daphne is CRITICAL: ERROR - maintenance: svc:/network/trainwreck:default [23:52:22] nacht ts [23:52:28] s4 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146) [23:52:28] MySQL on daphne is CRITICAL: Cant connect to MySQL server on daphne (146) [23:52:51] SMF on damiana is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [23:53:00] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:53:00] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [23:53:49] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 40280 MB (4% inode=99%): [23:54:00] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [23:54:19] SMTP on yarrow is CRITICAL: Connection refused [23:56:29] s5 replag on daphne is CRITICAL: QUERY CRITICAL: Cant connect to MySQL server on daphne (146)