[00:03:39] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [00:21:00] Load avg. on willow is CRITICAL: CRITICAL - load average: 27.96, 23.06, 20.63 [00:23:59] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 68836 [00:23:59] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 562653 [00:25:00] Load avg. on willow is WARNING: WARNING - load average: 15.08, 19.36, 19.70 [00:25:39] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:28:00] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 477482 MB (9% inode=43%): [00:28:00] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:29:38] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:32:00] Load avg. on willow is CRITICAL: CRITICAL - load average: 25.27, 21.43, 20.36 [00:37:37] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 68989.000000 [00:40:13] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 562996.000000 [00:42:25] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [00:50:25] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 40865 MB (10% inode=99%): [01:04:19] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [01:16:35] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24386 MB (5% inode=99%): [01:24:05] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 563911 [01:24:05] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 69011 [01:25:34] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 54406 MB (13% inode=99%): [01:26:34] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:28:35] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 479332 MB (9% inode=43%): [01:28:35] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:30:34] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:32:35] Load avg. on willow is CRITICAL: CRITICAL - load average: 20.59, 21.32, 21.95 [01:38:05] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 68918.000000 [01:41:05] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 564256.000000 [01:42:35] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:04:35] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [02:19:09] /sql on thyme is WARNING: DISK WARNING - free space: /sql 130856 MB (13% inode=99%): [02:24:09] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 68891 [02:24:09] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 565190 [02:27:27] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:28:38] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 478272 MB (9% inode=43%): [02:28:38] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:30:55] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:32:55] Load avg. on willow is CRITICAL: CRITICAL - load average: 18.59, 21.97, 22.21 [02:34:56] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 37331 MB (9% inode=99%): [02:39:05] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 68806.000000 [02:41:05] Kai Krueger * [Toolserver-l] OpenStreetMap on the toolserver (Ptolemy) [02:41:08] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 565557.000000 [02:42:59] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:43:59] Load avg. on willow is WARNING: WARNING - load average: 16.16, 17.92, 19.84 [02:45:58] Load avg. on willow is CRITICAL: CRITICAL - load average: 26.97, 21.19, 20.80 [02:48:58] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24322 MB (5% inode=99%): [02:52:59] Load avg. on willow is WARNING: WARNING - load average: 16.05, 18.90, 19.93 [03:00:58] Load avg. on willow is CRITICAL: CRITICAL - load average: 35.45, 24.17, 21.46 [03:05:07] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [03:15:08] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 50158 MB (12% inode=99%): [03:24:41] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 68923 [03:24:42] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 566500 [03:27:41] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:28:41] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 476607 MB (9% inode=43%): [03:28:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:31:09] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:39:41] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 68859.000000 [03:41:42] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 566924.000000 [03:43:49] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [04:01:24] Load avg. on willow is CRITICAL: CRITICAL - load average: 37.84, 30.55, 28.00 [04:05:24] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [04:24:44] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 69288 [04:25:04] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 568058 [04:28:05] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:29:34] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 478543 MB (9% inode=43%): [04:29:34] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:31:28] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:39:44] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 69329.000000 [04:41:44] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 568502.000000 [04:44:07] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:01:46] Load avg. on willow is CRITICAL: CRITICAL - load average: 24.21, 22.80, 22.68 [05:05:56] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [05:19:56] Load avg. on willow is WARNING: WARNING - load average: 14.96, 17.48, 19.72 [05:20:53] Load avg. on willow is CRITICAL: CRITICAL - load average: 21.84, 19.29, 20.23 [05:21:55] Load avg. on willow is WARNING: WARNING - load average: 17.71, 18.62, 19.95 [05:42:34] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 570145.000000 [05:45:04] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:57:56] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 102080 MB (10% inode=98%): [05:57:56] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 102080 MB (10% inode=98%): [06:06:45] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [06:15:27] Load avg. on willow is CRITICAL: CRITICAL - load average: 36.05, 23.21, 19.32 [06:16:28] Load avg. on willow is WARNING: WARNING - load average: 28.02, 23.50, 19.69 [06:22:51] [[Special:Log/delete]] delete 10 * Matthewrbowker * (deleted "[[02User talk:Alper90410]]": Spam) [06:23:22] [[Special:Log/block]] block 10 * Matthewrbowker * (blocked [[02User:Alper90410]] with an expiry time of infinite (account creation disabled): Spam) [06:25:47] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 69884 [06:26:28] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 571259 [06:29:26] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:30:45] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 40311 MB (9% inode=99%): [06:30:45] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 477783 MB (9% inode=43%): [06:30:45] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:31:45] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:40:53] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 69670.000000 [06:43:02] s1 replag on thyme is CRITICAL: (Service Check Timed Out) [06:45:45] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:54:44] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24218 MB (5% inode=99%): [07:03:45] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 53785 MB (13% inode=99%): [07:06:53] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [07:16:47] Load avg. on willow is WARNING: WARNING - load average: 14.02, 16.95, 18.25 [07:25:47] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 69311 [07:26:47] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 572761 [07:29:47] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:30:47] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 478211 MB (9% inode=43%): [07:30:55] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:31:55] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:41:14] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 69058.000000 [07:43:44] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 573266.000000 [07:46:14] Load avg. on willow is CRITICAL: CRITICAL - load average: 25.31, 21.69, 20.12 [07:46:14] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [07:48:13] Load avg. on willow is WARNING: WARNING - load average: 18.36, 20.48, 19.87 [08:01:09] Load avg. on willow is CRITICAL: CRITICAL - load average: 30.27, 22.12, 20.16 [08:07:10] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [08:23:04] Load avg. on willow is CRITICAL: CRITICAL - load average: 19.29, 19.86, 20.41 [08:25:04] Load avg. on willow is WARNING: WARNING - load average: 17.04, 18.79, 19.93 [08:26:15] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 68731 [08:26:55] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 574103 [08:30:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:30:56] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:31:34] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 478068 MB (9% inode=43%): [08:31:56] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:41:57] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 68601.000000 [08:43:50] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 574466.000000 [08:46:50] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:03:32] [[Special:Log/newusers]] create 10 * Suicide101 * (New user account) [09:07:58] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [09:24:22] Load avg. on willow is OK: OK - load average: 13.30, 13.61, 14.95 [09:26:52] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 68103 [09:27:01] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 575596 [09:30:23] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:31:05] Dr. Trigon * Re: [Toolserver-l] Update of SGE on Thursday 5. July [09:31:52] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 477944 MB (9% inode=43%): [09:31:52] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:32:52] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:33:22] Load avg. on willow is WARNING: WARNING - load average: 13.70, 15.16, 15.25 [09:42:53] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 67949.000000 [09:43:52] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 575943.000000 [09:46:52] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:52:05] Dr. Trigon * Re: [Toolserver-l] Wpipe [10:01:06] Josh Grant * Re: [Toolserver-l] Update of SGE on Thursday 5. July [10:08:05] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [10:27:06] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 67400 [10:27:06] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 576736 [10:31:06] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:32:06] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 477733 MB (9% inode=43%): [10:32:06] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:33:06] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:34:06] Load avg. on willow is WARNING: WARNING - load average: 19.31, 20.55, 19.02 [10:43:10] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 67312.000000 [10:44:32] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 577105.000000 [10:47:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [11:08:33] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [11:13:13] Load avg. on willow is CRITICAL: CRITICAL - load average: 25.10, 22.73, 20.25 [11:20:11] Load avg. on willow is WARNING: WARNING - load average: 14.87, 19.35, 19.89 [11:27:11] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 66842 [11:27:29] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 578310 [11:31:26] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:32:25] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 477500 MB (9% inode=43%): [11:32:46] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:33:46] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:43:40] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 66710.000000 [11:44:41] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 578504.000000 [11:48:23] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:08:41] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [12:20:22] Load avg. on willow is WARNING: WARNING - load average: 22.28, 18.08, 17.39 [12:27:22] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 66134 [12:27:40] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 579013 [12:32:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:33:21] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 462555 MB (8% inode=42%): [12:33:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:34:40] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:43:40] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 66044.000000 [12:44:59] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:45:40] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 579232.000000 [12:48:23] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:02:06] Magnus Manske * Re: [Toolserver-l] Wpipe [13:03:21] Load avg. on ptolemy is WARNING: WARNING - load average: 19.01, 16.11, 11.55 [13:04:59] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [13:09:40] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [13:18:23] Load avg. on ptolemy is OK: OK - load average: 7.36, 14.95, 14.73 [13:21:23] Load avg. on willow is WARNING: WARNING - load average: 19.13, 18.93, 18.64 [13:27:22] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 65682 [13:27:41] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 579679 [13:32:23] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:33:22] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 474737 MB (8% inode=43%): [13:34:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:34:42] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:43:41] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 65570.000000 [13:46:00] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 580007.000000 [13:49:23] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:58:55] [[Category:Geo tools]] !M 10https://wiki.toolserver.org/w/index.php?diff=7459&oldid=5270&rcid=10151 * Osiris * (+19) () [14:09:41] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [14:19:22] /sql on thyme is WARNING: DISK WARNING - free space: /sql 128395 MB (13% inode=99%): [14:21:23] Load avg. on willow is WARNING: WARNING - load average: 20.02, 19.52, 19.13 [14:27:21] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 65210 [14:27:41] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 580889 [14:33:22] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 474691 MB (8% inode=43%): [14:33:23] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:34:40] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:34:41] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:43:50] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 65243.000000 [14:46:00] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 581370.000000 [14:49:23] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:10:41] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [15:22:22] Load avg. on willow is WARNING: WARNING - load average: 13.41, 14.07, 15.01 [15:23:05] DaB. * Re: [Toolserver-l] Update of SGE on Thursday 5. July [15:23:23] Load avg. on willow is OK: OK - load average: 14.50, 14.18, 14.98 [15:27:23] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 65261 [15:28:40] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 582495 [15:30:30] Load avg. on willow is WARNING: WARNING - load average: 18.56, 16.72, 15.92 [15:33:22] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 474604 MB (8% inode=43%): [15:33:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:34:40] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:35:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:43:50] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 65248.000000 [15:45:59] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 583057.000000 [15:50:23] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:11:41] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [16:27:23] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 65455 [16:28:40] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 584218 [16:31:22] Load avg. on willow is WARNING: WARNING - load average: 20.32, 19.01, 18.61 [16:33:22] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 473692 MB (8% inode=43%): [16:33:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:35:41] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:36:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:39:04] [[Special:Log/newusers]] create 10 * BeatrisZ9 * (New user account) [16:43:50] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 65512.000000 [16:46:00] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 584728.000000 [16:51:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:11:41] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [17:28:22] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 65755 [17:28:40] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 585651 [17:29:22] /sql on cassia is CRITICAL: DISK CRITICAL - free space: /sql 46696 MB (3% inode=98%): [17:31:23] Load avg. on willow is WARNING: WARNING - load average: 21.05, 17.86, 17.88 [17:33:22] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 473627 MB (8% inode=43%): [17:34:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:35:40] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:36:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:43:50] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 65883.000000 [17:46:00] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 586056.000000 [17:49:23] Load avg. on willow is OK: OK - load average: 11.11, 13.45, 14.95 [17:52:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:58:23] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 100829 MB (10% inode=98%): [17:58:23] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 100829 MB (10% inode=98%): [18:12:41] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [18:26:49] [[File talk:Wikimedia Community Logo-Toolserver.png]] !N 10https://wiki.toolserver.org/w/index.php?oldid=7460&rcid=10153 * Spectralyst * (+24) (HolaPedia ~~~~ xD) [18:28:22] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 66152 [18:28:41] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 587031 [18:33:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 473249 MB (8% inode=43%): [18:34:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:35:41] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:37:40] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:44:50] s1 replag on rosemary is CRITICAL: (Service Check Timed Out) [18:46:10] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 587418.000000 [18:46:22] Load avg. on willow is WARNING: WARNING - load average: 16.73, 16.59, 14.97 [18:53:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:55:22] Load avg. on willow is OK: OK - load average: 14.01, 14.81, 14.86 [19:07:21] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 40454 MB (9% inode=99%): [19:12:41] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [19:28:22] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 66749 [19:29:41] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 588506 [19:33:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 473490 MB (8% inode=43%): [19:34:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:35:22] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24084 MB (5% inode=99%): [19:36:41] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:38:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:45:22] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 66943.000000 [19:46:10] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 588904.000000 [19:46:22] Load avg. on willow is CRITICAL: CRITICAL - load average: 32.70, 21.89, 16.16 [19:47:22] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 54520 MB (13% inode=99%): [19:47:22] Load avg. on willow is WARNING: WARNING - load average: 25.45, 21.77, 16.49 [19:53:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [20:13:41] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [20:28:22] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 67313 [20:29:41] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 589957 [20:34:23] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 472861 MB (8% inode=43%): [20:34:23] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:36:41] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:37:22] Load avg. on willow is OK: OK - load average: 13.70, 14.40, 14.91 [20:39:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:45:22] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 67523.000000 [20:46:23] Load avg. on willow is WARNING: WARNING - load average: 19.01, 17.57, 16.18 [20:50:09] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 590465.000000 [20:50:22] Load avg. on willow is CRITICAL: CRITICAL - load average: 30.02, 20.16, 17.40 [20:53:23] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:10:22] Load avg. on willow is WARNING: WARNING - load average: 20.78, 15.87, 15.70 [21:13:40] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [21:28:22] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 67737 [21:30:41] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 591560 [21:35:22] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 472727 MB (8% inode=43%): [21:35:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:36:40] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:39:27] @replag [21:39:27] Platonides: s1-rr-a: 18h 48m 47s [+0.02 s/s]; s1-user: 6d 20h 22m 51s [+0.32 s/s]; s3-rr-a: 23s [+0.00 s/s]; s3-user: 23s [+0.00 s/s] [21:39:40] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:45:22] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 67792.000000 [21:46:02] !ops Yes, hello. festeringvag here. [21:50:10] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 591974.000000 [21:54:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:59:22] Load avg. on willow is OK: OK - load average: 11.10, 12.23, 14.83 [22:14:41] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [22:29:22] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 67693 [22:30:41] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 593040 [22:35:22] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 472654 MB (8% inode=43%): [22:35:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:36:41] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:40:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:45:23] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 67961.000000 [22:50:22] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 593533.000000 [22:54:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [23:06:22] Load avg. on willow is WARNING: WARNING - load average: 15.54, 15.34, 13.88 [23:08:23] Load avg. on willow is OK: OK - load average: 13.21, 14.74, 13.84 [23:14:40] ethernet 0/1/14 [willow.mgmt] on asw-oe10-esams.mgmt is CRITICAL: GigabitEthernet0/1/14:UP: 1 int NOK : CRITICAL [23:16:22] Load avg. on willow is WARNING: WARNING - load average: 18.29, 17.09, 15.36 [23:29:22] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 68200 [23:31:41] MySQL slave on thyme is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 594472 [23:35:22] /aux0 on hemlock is WARNING: DISK WARNING - free space: /aux0 471403 MB (8% inode=43%): [23:35:23] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:36:41] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:40:41] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:45:22] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 68200.000000 [23:50:22] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 594927.000000 [23:50:22] Load avg. on willow is WARNING: WARNING - load average: 18.01, 15.81, 15.57 [23:52:22] Load avg. on willow is OK: OK - load average: 9.89, 13.57, 14.76 [23:55:23] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default