[00:00:32] wikidata replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3390.000000 [00:01:12] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3616.000000 [00:01:13] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3617 [00:01:42] wikidata replag on daphne is OK: QUERY OK: SELECT ts_rc_age() returned 1545.000000 [00:04:32] wikidata replag on cassia is OK: QUERY OK: SELECT ts_rc_age() returned 1349.000000 [00:07:52] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7206.000000 [00:08:12] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7227.000000 [00:08:13] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7227.000000 [00:08:32] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7249.000000 [00:08:42] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7257.000000 [00:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 270935 MB (5% inode=67%): [00:20:42] wikidata replag on thyme is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3418.000000 [00:23:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [00:24:42] wikidata replag on thyme is OK: QUERY OK: SELECT ts_rc_age() returned 1509.000000 [00:26:13] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:27:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [00:28:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [00:29:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:30:22] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [00:30:32] s1 replag on thyme is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2807.000000 [00:30:42] MySQL slave on thyme is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2808 [00:31:52] wikidata replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3576.000000 [00:32:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:33:13] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:36:52] wikidata replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1687.000000 [00:43:12] wikidata replag on z-dat-s7-a is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3141.000000 [00:46:12] wikidata replag on z-dat-s7-a is OK: QUERY OK: SELECT ts_rc_age() returned 1306.000000 [00:53:12] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [01:01:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4971 [01:01:13] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 4970.000000 [01:03:01] @replag [01:03:02] DaBPunkt: s1-rr-a: 44m 43s [+0.16 s/s]; s1-rr-a-wd: 1m 14s [+0.00 s/s]; s1-user: 1h 22m 54s [+0.32 s/s]; s1-user-wd: 1m 15s [+0.00 s/s]; s2-rr: 7m 20s [-0.10 s/s]; s2-user: 7m 20s [-0.10 s/s]; s2-user-c: error; s2-user-wd: 2h 37m 59s [+0.60 s/s] [01:03:03] DaBPunkt: s3-user: 42s [+0.00 s/s]; s3-user-wd: error; s4-user-wd: 1m 15s [+0.00 s/s]; s5-user-c: 45s [-0.00 s/s]; s6-user-wd: 2h 55m 28s [+0.74 s/s]; s7-user-wd: 1m 16s [+0.00 s/s] [01:03:27] @replag [01:03:27] DaBPunkt: s1-rr-a: 44m 43s [-]; s1-rr-a-wd: 1m 28s [+0.52 s/s]; s1-user: 1h 23m 10s [+0.59 s/s]; s1-user-wd: 1m 29s [+0.52 s/s]; s2-rr: 7m 40s [+0.74 s/s]; s2-user: 7m 40s [+0.74 s/s]; s2-user-c: error; s2-user-wd: 2h 38m 6s [+0.26 s/s] [01:03:28] DaBPunkt: s3-user: 16s [-0.96 s/s]; s3-user-wd: error; s4-user-wd: 1m 28s [+0.48 s/s]; s5-user-c: 60s [+0.58 s/s]; s6-user-wd: 2h 55m 53s [+1.01 s/s]; s7-user-wd: 1m 29s [+0.52 s/s] [01:08:13] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10826.000000 [01:08:32] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 9690.000000 [01:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 272806 MB (5% inode=67%): [01:23:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [01:26:13] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:27:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [01:28:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [01:29:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:30:22] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [01:30:32] s1 replag on thyme is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2380.000000 [01:30:42] MySQL slave on thyme is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2373 [01:32:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:33:12] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:38:52] DaBPunkt: is wikidata still being imported? [01:39:10] duh: on s3 and s6: yes [01:39:22] everywhere else it should be avaible [01:39:45] do you know how long it will take for those two to finish? [01:40:32] s6 should be very soon, s3 will take more time I gues (and I'm on the way to bed) [01:41:22] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 69840 MB (7% inode=99%): [01:41:58] ok thanks [01:42:04] ill just have my script avoid those for now [01:50:25] nacht ts [01:51:32] s1 replag on thyme is OK: QUERY OK: SELECT ts_rc_age() returned 1781.000000 [01:51:42] MySQL slave on thyme is OK: Uptime: 6063726 Threads: 12 Questions: 3368382050 Slow queries: 418029 Opens: 134417 Flush tables: 1 Open tables: 573 Queries per second avg: 555.497 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1773 [01:53:13] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [02:01:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 5090 [02:01:12] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 5090.000000 [02:08:12] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14427.000000 [02:08:32] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7715.000000 [02:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 272106 MB (5% inode=67%): [02:23:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [02:26:12] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:27:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [02:28:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:29:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:30:24] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [02:32:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:33:13] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:46:12] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2097 [02:53:12] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [03:01:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4806 [03:01:12] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 4807.000000 [03:08:12] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 5680.000000 [03:08:32] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 8957.000000 [03:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 271249 MB (5% inode=67%): [03:22:12] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3633 [03:23:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [03:26:13] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:27:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [03:28:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:29:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:30:13] wikidata replag on z-dat-s6-a is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3533.000000 [03:30:22] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [03:32:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:33:13] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:53:13] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [03:54:12] wikidata replag on z-dat-s6-a is OK: QUERY OK: SELECT ts_rc_age() returned 1573.000000 [04:01:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4921 [04:01:12] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 4921.000000 [04:08:42] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10914.000000 [04:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 270220 MB (5% inode=67%): [04:22:12] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 6334 [04:23:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [04:26:21] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:27:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [04:28:22] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [04:29:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:30:22] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [04:32:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:33:12] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:53:12] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [05:01:12] MySQL slave on rosemary is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 5514 [05:01:12] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 5514.000000 [05:08:42] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 12755.000000 [05:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 271291 MB (5% inode=67%): [05:22:12] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 7999 [05:23:01] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [05:26:22] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:27:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [05:28:21] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:29:13] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:30:22] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [05:32:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:33:12] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:40:12] s1 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 3540.000000 [05:40:13] MySQL slave on rosemary is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3539 [05:53:12] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [05:59:24] @replag [05:59:24] liangent: s1-rr-a-wd: 1m 43s [+0.00 s/s]; s1-user: 33m 18s [-0.17 s/s]; s1-user-wd: 1m 41s [+0.00 s/s]; s2-rr: 2h 34m 27s [+0.50 s/s]; s2-user: 2h 34m 27s [+0.50 s/s]; s2-user-c: error; s2-user-wd: 3h 51m 16s [+0.25 s/s]; s3-user: 4m 36s [+0.01 s/s] [05:59:25] liangent: s3-user-wd: 1m 43s [-]; s4-user-wd: 1m 43s [+0.00 s/s]; s5-user-c: 46s [-0.00 s/s]; s6-user-wd: 1m 43s [-0.59 s/s]; s7-user-wd: 1m 41s [+0.00 s/s] [06:01:12] s1 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1735.000000 [06:01:13] MySQL slave on rosemary is OK: Uptime: 409633 Threads: 14 Questions: 326595131 Slow queries: 82372 Opens: 30128 Flush tables: 1 Open tables: 3208 Queries per second avg: 797.287 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1735 [06:08:42] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 13981.000000 [06:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 271141 MB (5% inode=67%): [06:22:12] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 9227 [06:23:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [06:26:22] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:27:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [06:28:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:29:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:30:22] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [06:32:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:33:13] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:53:12] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [07:08:42] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 14712.000000 [07:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 270973 MB (5% inode=67%): [07:22:12] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 9378 [07:23:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [07:26:22] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:27:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [07:28:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [07:29:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:30:22] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [07:32:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:33:13] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:53:12] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [08:08:42] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 13468.000000 [08:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 270726 MB (5% inode=67%): [08:22:13] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 8397 [08:23:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [08:26:22] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:27:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [08:28:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:29:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:30:22] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [08:32:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:33:12] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:53:12] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [09:08:42] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10952.000000 [09:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 268767 MB (5% inode=67%): [09:22:12] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 5297 [09:23:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [09:26:21] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:27:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [09:28:33] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:29:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:30:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [09:32:12] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:33:12] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:53:13] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [10:08:42] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 6107.000000 [10:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 269908 MB (5% inode=67%): [10:22:13] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4335 [10:23:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [10:26:22] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:27:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [10:28:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [10:29:13] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:30:31] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [10:32:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:33:12] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:53:12] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [10:56:05] @replag [10:56:05] Merlissimo: s1-rr-a-wd: 32s [-0.01 s/s]; s1-user-wd: 39s [-0.00 s/s]; s2-rr: 1h 5m 34s [-0.21 s/s]; s2-user: 1h 5m 34s [-0.21 s/s]; s2-user-c: error; s2-user-wd: 2h 9m 13s [-0.05 s/s]; s3-user: error; s3-user-wd: error [10:56:06] Merlissimo: s4-user-wd: 36s [-0.00 s/s]; s5-user-c: 1m 41s [+0.01 s/s]; s6-user: error; s6-user-wd: error; s7-user: error; s7-user-wd: error [11:03:12] MySQL slave on z-dat-s2-b is WARNING: SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3527 [11:08:43] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 8504.000000 [11:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 271916 MB (5% inode=67%): [11:23:01] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [11:23:17] @replag [11:23:17] Merlissimo: s1-rr-a-wd: 37s [+0.00 s/s]; s1-user-wd: 38s [-0.00 s/s]; s2-rr: 54m 49s [-0.40 s/s]; s2-user: 54m 49s [-0.40 s/s]; s2-user-c: error; s2-user-wd: 2h 36m 24s [+1.00 s/s]; s3-user: 1m 51s [+0.02 s/s]; s3-user-wd: 2h 36m 25s [+1.00 s/s] [11:23:18] Merlissimo: s4-user-wd: 39s [+0.00 s/s]; s5-user-c: 1m 38s [-0.00 s/s]; s6-user-wd: 40s [-0.00 s/s]; s7-user-wd: 37s [-0.00 s/s] [11:23:28] @replag all [11:23:29] Merlissimo: s1-rr-a: 4s [-0.07 s/s]; s1-rr-a-wd: 42s [+0.44 s/s]; s1-user: 4s [-0.10 s/s]; s1-user-c: 2s [-]; s1-user-wd: 37s [-0.09 s/s]; s2-rr: 54m 59s [+0.87 s/s]; s2-user: 54m 59s [+0.87 s/s]; s2-user-c: error [11:23:30] Merlissimo: s2-user-wd: 2h 36m 36s [+1.05 s/s]; s3-user: 1s [-9.62 s/s]; s3-user-wd: 2h 36m 36s [+0.96 s/s]; s4-rr-a: 2s [-]; s4-user: 2s [-]; s4-user-wd: 36s [-0.26 s/s]; s5-rr-a: 1s [-]; s5-user: 1s [-] [11:23:31] Merlissimo: s5-user-c: 1m 42s [+0.35 s/s]; s6-user: 1s [-]; s6-user-wd: 43s [+0.26 s/s]; s7-user: 2s [-]; s7-user-wd: 37s [-] [11:26:22] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:27:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [11:28:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [11:29:12] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:30:33] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [11:32:13] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:33:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:35:13] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3614 [11:53:52] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [12:08:42] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 12104.000000 [12:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 271483 MB (5% inode=67%): [12:23:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [12:26:22] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:27:21] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [12:28:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:29:13] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:30:31] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [12:32:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:33:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:35:12] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 5788 [12:40:22] /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 58078 MB (5% inode=99%): [12:46:12] /sql on thyme is WARNING: DISK WARNING - free space: /sql 91250 MB (9% inode=99%): [12:53:51] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [13:08:42] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 15704.000000 [13:18:21] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 270776 MB (5% inode=67%): [13:23:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [13:24:12] Free Memory on damiana is CRITICAL: CRITICAL - 5.0% (414864 kB) free! [13:26:22] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:27:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [13:28:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:29:22] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:30:12] Free Memory on damiana is WARNING: WARNING - 5.3% (444748 kB) free! [13:30:31] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [13:32:21] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:33:27] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:33:27] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 73156 MB (7% inode=99%): [13:35:12] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 5126 [13:43:12] Free Memory on damiana is CRITICAL: CRITICAL - 4.8% (400740 kB) free! [13:53:51] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [13:58:12] Free Memory on damiana is OK: OK - 11.2% (938068 kB) free. [14:01:13] /sql on thyme is OK: DISK OK - free space: /sql 129267 MB (13% inode=99%): [14:08:42] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 19304.000000 [14:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 268750 MB (5% inode=67%): [14:22:12] Free Memory on damiana is WARNING: WARNING - 6.2% (522604 kB) free! [14:23:01] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [14:26:21] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:27:22] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [14:28:13] Free Memory on damiana is CRITICAL: CRITICAL - 4.7% (396384 kB) free! [14:28:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [14:29:12] Free Memory on damiana is WARNING: WARNING - 5.1% (427452 kB) free! [14:29:22] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:30:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [14:32:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:33:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:35:12] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 7003 [14:50:32] APT on mayapple is WARNING: APT WARNING: 34 packages available for upgrade (0 critical updates). [14:53:52] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [15:08:42] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 22904.000000 [15:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 266142 MB (4% inode=67%): [15:23:01] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [15:26:22] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:27:32] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [15:28:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:29:22] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:30:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [15:32:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:33:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:33:31] Hi. Trying to get NewLangEx*.py for pgkbot at http://toolserver.org/~tangotango/pgkbot/ but its a 403 user account expired error [15:35:12] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 8210 [15:38:12] Free Memory on damiana is CRITICAL: CRITICAL - 3.1% (258336 kB) free! [15:50:32] APT on mayapple is WARNING: APT WARNING: 34 packages available for upgrade (0 critical updates). [15:53:52] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [16:02:26] t|c: that user hasnt been around in a loong time [16:03:10] Betacommand: I am trying to build pgkbot and possibly swmt although not sure how c# could work on freebsd [16:03:22] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 127215 MB (20% inode=99%): [16:04:04] t|c: pgkbot died like 5 years ago [16:04:18] is there anything I can use to relay and monitor wiki from irc? [16:04:42] t|c: what wiki? [16:05:07] a small wiki I installed for my website with somewhat large plans to grow [16:05:28] which has an ircd that is tied into the site [16:05:37] t|c: there should be code to auto publish the RC feed via irc [16:06:08] mediawiki shouldnt need third party software [16:07:02] t|c: http://www.mediawiki.org/wiki/Manual:IRC_RC_Bot [16:07:08] I'm intrigued by #cvn-sw relay and monitoring of small wiki's. It would be great to have our opers have that kind of eye [16:07:41] t|c: all that those bots do is copy the feed from irc.wikimedia.org [16:08:04] Ah. It makes sense now after reading their old docs [16:08:42] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 26504.000000 [16:09:04] wikimedia feeds the Recent changes to their irc and then bots like swmt and pgkbot parse it [16:09:29] and forward it to freenode chanels [16:09:46] thus their code wouldnt work for you [16:12:59] thank you Betacommand [16:13:11] t|c: any time [16:18:21] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 264816 MB (4% inode=67%): [16:23:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [16:26:22] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:27:32] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [16:28:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:29:22] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:30:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [16:32:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:33:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:35:12] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 6502 [16:38:13] Free Memory on damiana is CRITICAL: CRITICAL - 2.5% (211292 kB) free! [16:50:32] APT on mayapple is WARNING: APT WARNING: 34 packages available for upgrade (0 critical updates). [16:53:51] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [16:57:32] APT on yarrow is WARNING: APT WARNING: 33 packages available for upgrade (0 critical updates). [17:03:22] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 126930 MB (20% inode=99%): [17:08:42] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 30104.000000 [17:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 264003 MB (4% inode=67%): [17:23:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [17:26:22] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:27:34] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [17:28:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:29:22] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:30:42] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [17:32:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:33:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:35:36] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 8071 [17:38:13] Free Memory on damiana is CRITICAL: CRITICAL - 2.7% (228556 kB) free! [17:50:42] APT on mayapple is WARNING: APT WARNING: 34 packages available for upgrade (0 critical updates). [17:53:52] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [17:57:41] APT on yarrow is WARNING: APT WARNING: 33 packages available for upgrade (0 critical updates). [18:03:22] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 126586 MB (20% inode=99%): [18:08:45] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 33704.000000 [18:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 262434 MB (4% inode=67%): [18:23:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [18:26:23] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:27:32] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A: [18:28:42] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:29:22] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:30:42] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [18:31:41] @replag [18:31:44] Merlissimo: s1-rr-a: 5m 30s [+0.02 s/s]; s1-rr-a-wd: 21m 14s [+0.07 s/s]; s1-user: 5m 31s [+0.01 s/s]; s1-user-c: 5m 31s [+0.02 s/s]; s1-user-wd: 21m 8s [+0.07 s/s]; s2-rr: 2h 4m 28s [+0.13 s/s]; s2-user: 2h 4m 28s [+0.13 s/s]; s2-user-c: error [18:31:45] Merlissimo: s2-user-wd: 9h 44m 51s [+1.00 s/s]; s3-user: 5m 33s [+0.02 s/s]; s3-user-wd: 9h 44m 51s [+1.00 s/s]; s4-rr-a: 5m 31s [+0.02 s/s]; s4-user: 5m 31s [+0.02 s/s]; s4-user-wd: 26m 11s [+0.08 s/s]; s5-rr-a: 5m 33s [+0.02 s/s]; s5-user: 5m 33s [+0.02 s/s] [18:31:46] Merlissimo: s5-user-c: 25m 27s [+0.08 s/s]; s6-user: 5m 31s [+0.02 s/s]; s6-user-wd: 21m 8s [+0.07 s/s]; s7-user: 5m 34s [+0.02 s/s]; s7-user-wd: 21m 14s [+0.07 s/s] [18:32:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:33:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:35:13] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 7424 [18:38:12] Free Memory on damiana is CRITICAL: CRITICAL - 2.2% (184320 kB) free! [18:41:42] wikidata replag on daphne is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2167.000000 [18:42:22] s4 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2164.000000 [18:46:42] wikidata replag on thyme is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2171.000000 [18:46:52] wikidata replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2167.000000 [18:46:52] wikidata replag on z-dat-s7-a is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2173.000000 [18:47:22] wikidata replag on z-dat-s6-a is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2197.000000 [18:47:42] wikidata replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2220.000000 [18:50:42] APT on mayapple is WARNING: APT WARNING: 34 packages available for upgrade (0 critical updates). [18:53:52] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [18:57:42] APT on yarrow is WARNING: APT WARNING: 33 packages available for upgrade (0 critical updates). [19:03:22] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 126267 MB (20% inode=99%): [19:05:42] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3608.000000 [19:06:22] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3604.000000 [19:08:42] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 37304.000000 [19:10:42] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3601.000000 [19:10:43] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3610.000000 [19:10:52] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3613.000000 [19:10:52] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3607.000000 [19:11:22] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3637.000000 [19:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 259557 MB (4% inode=67%): [19:23:01] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [19:26:22] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:27:32] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (2 errors, 1 warning): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.A:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.Communicatio [19:28:45] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:29:22] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:30:42] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [19:32:26] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:33:22] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:35:12] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 9381 [19:38:12] Free Memory on damiana is CRITICAL: CRITICAL - 2.4% (197876 kB) free! [19:50:42] APT on mayapple is WARNING: APT WARNING: 34 packages available for upgrade (0 critical updates). [19:53:52] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [19:57:42] APT on yarrow is WARNING: APT WARNING: 33 packages available for upgrade (0 critical updates). [20:03:23] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 125955 MB (20% inode=99%): [20:04:55] hello all [20:05:09] hi DaBPunkt [20:05:42] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7208.000000 [20:06:31] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7206.000000 [20:07:24] switching cassia to read-only now [20:08:42] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 40904.000000 [20:10:42] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7201.000000 [20:10:42] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7211.000000 [20:10:52] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7213.000000 [20:10:52] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7207.000000 [20:11:22] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 7237.000000 [20:14:12] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [20:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 262000 MB (4% inode=67%): [20:23:02] Virtual disks on far1-n1-oe16-esams.mgmt is CRITICAL: OK 3, WARN 0, CRIT 1: far1-n1-fast3 FTOL, far1-n1-bulk CRIT, far1-n1-fast2 FTOL, far1-n1-fast FTOL [20:26:21] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:27:32] CAM on hemlock is CRITICAL: CRITICAL - Storage ts-array5 (3 errors): null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.description:S17:Tray.85.Battery.B:, null :OSGi.com.sun.storage.cam.agent(com.sun.netstorage.fm.storade.agent.Messages):monitor.CommunicationLost.desc:S3:oob:S4:2530:S9:ts-array5:, null :OSGi.com.sun.storage.cam.agent(device.2530):event.ProblemEvent.REC_EXPIRED_BATTERY.descri [20:28:42] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [20:29:22] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:30:42] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [20:32:22] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:33:21] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:35:13] MySQL slave on z-dat-s2-b is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 11718 [20:38:12] Free Memory on damiana is CRITICAL: CRITICAL - 2.3% (194852 kB) free! [20:44:22] s5 replag on cassia is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 2195.000000 [20:50:42] APT on mayapple is WARNING: APT WARNING: 34 packages available for upgrade (0 critical updates). [20:53:52] MySQL slave on z-dat-s4-a is CRITICAL: (Return code of 139 is out of bounds) [20:57:42] APT on yarrow is WARNING: APT WARNING: 33 packages available for upgrade (0 critical updates). [21:03:23] /sql on ptolemy is WARNING: DISK WARNING - free space: /sql 125611 MB (20% inode=99%): [21:05:42] wikidata replag on daphne is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10808.000000 [21:06:31] s4 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10805.000000 [21:08:22] s5 replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 3635.000000 [21:08:41] wikidata replag on z-dat-s2-b is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 44504.000000 [21:10:48] wikidata replag on cassia is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10800.000000 [21:10:48] wikidata replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10810.000000 [21:10:52] wikidata replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10807.000000 [21:10:52] wikidata replag on z-dat-s7-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10814.000000 [21:11:22] wikidata replag on z-dat-s6-a is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 10836.000000 [21:14:12] MySQL slave on cassia is CRITICAL: (Return code of 139 is out of bounds) [21:18:22] /mnt user-store on rosemary is CRITICAL: DISK CRITICAL - free space: /mnt 262240 MB (4% inode=67%): [21:21:50] /sql on rosemary is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:22:15] Sun Grid Engine execd on willow is CRITICAL: (Service Check Timed Out) [21:22:15] APT on yarrow is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:23:38] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 69578 MB (7% inode=99%): [21:23:54] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:27:26] DaBPunkt: www is giving 404's and I can't login. Do we have homedir problems or something like that atm? [21:27:49] let me look [21:27:53] DaBPunkt: I'm getting 500's, too. [21:28:00] Just what I came in here for. [21:28:23] mm, login fails here too. I will check the logs [21:29:32] And tsnag died, too... [21:29:37] DaBPunkt: homedir missing [21:31:56] http://nagios.toolserver.org/ doesn't seem to be very responsive [21:32:03] looks like the HA-cluster failed [21:32:57] The good old HA story: The availability of a non-HA-cluster tends to be higher that an HA-cluster :-( [21:33:47] damiana is complaining about low memory, but do not give the services away AFAIS [21:34:05] I will try to reboot it [21:41:36] SMTP on rosemary is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:41:36] s4 replag on rosemary is CRITICAL: QUERY CRITICAL: Lost connection to MySQL server at reading authorization packet, system error: 0 [21:41:36] s4 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 837.000000 [21:41:36] FMA on rosemary is CRITICAL: (Service Check Timed Out) [21:41:36] FMA on rosemary is OK: OK [21:43:53] ok, damiana is rebooting [21:44:22] turnera is taking over AFAIS [21:44:56] DaBPunkt: yes for sge [21:46:27] The Edit counter is showing a mainspace article as userspace [21:47:39] Maybe because I moevd the userspace article to the mainspace [21:52:29] ok, I think we are back [21:53:40] the user-databases are imported too BTW. I will import the user-rights now and make and views that tahn we should be done [21:55:20] * Matthew_ hugs DaBPunkt [21:55:22] Thanks man :) [22:12:42] doing the switch now [22:13:03] Any idea what caused the downtime DaBPunkt? [22:25:45] Hummm... some database errors with s2 and s7. Related to the switch? [22:26:24] s7 should be ok, i think. [22:27:34] Hmmmm... I'm getting an error trying to use an SUL tool. [22:28:08] s2 is rebooting as announced [22:29:08] Ah, yes, that's right. Forgot about that... [22:44:47] anyone know the filesystem blocksize on /mnt/user-store ? (not the nfs block size, the real one) [22:47:02] sorry, I have to reboot s5 too [23:14:01] @replag [23:14:33] DaBPunkt: s1-rr-a-wd: 55s [-1.52 s/s]; s1-user-wd: 4h 16m 46s [+1.00 s/s]; s2-rr: 3h 20m 57s [-0.30 s/s]; s2-user: 3h 20m 57s [-0.30 s/s]; s2-user-c: error; s2-user-wd: 14h 27m 9s [+1.00 s/s]; s3-user: 13s [-0.00 s/s]; s3-user-wd: 14h 27m 9s [+1.00 s/s] [23:14:34] DaBPunkt: s4-user-wd: 58s [-1.15 s/s]; s5-rr-a: 3h 6m 18s [+1.00 s/s]; s5-user: 1h 35m 14s [+0.81 s/s]; s5-user-c: error; s6-user-wd: 52s [-0.70 s/s]; s7-user-wd: 4h 2m 21s [+1.00 s/s] [23:34:23] @replag [23:34:23] DaBPunkt: s1-rr-a-wd: 58s [+0.00 s/s]; s1-user-wd: 3h 7m 47s [-3.39 s/s]; s2-rr: 2h 32m 2s [-2.40 s/s]; s2-user: 2h 32m 2s [-2.40 s/s]; s2-user-c: error; s2-user-wd: 14h 47m 30s [+1.00 s/s]; s3-user: 10s [-0.00 s/s]; s3-user-wd: 14h 47m 30s [+1.00 s/s] [23:34:24] DaBPunkt: s4-user-wd: 54s [-0.00 s/s]; s5-rr-a: 2h 18m 48s [-2.33 s/s]; s5-user: 1h 53m 10s [+0.88 s/s]; s5-user-c: error; s6-user-wd: 50s [-0.00 s/s]; s7-user-wd: 2h 40m 27s [-4.13 s/s] [23:38:42] @replag [23:38:42] DaBPunkt: s1-rr-a-wd: 53s [-0.15 s/s]; s1-user-wd: 2h 52m 21s [-3.65 s/s]; s2-rr: 2h 30m 20s [-1.46 s/s]; s2-user: 2h 30m 20s [-1.46 s/s]; s2-user-c: error; s2-user-wd: 14h 51m 7s [+0.60 s/s]; s3-user: 12s [-0.16 s/s]; s3-user-wd: 14h 50m 43s [+0.38 s/s] [23:38:43] DaBPunkt: s4-user-wd: 53s [-0.13 s/s]; s5-rr-a: 1h 2m 59s [-18.87 s/s]; s5-user: 1h 56m 51s [+0.89 s/s]; s5-user-c: error; s6-user-wd: 53s [-0.15 s/s]; s7-user: 10s [+0.08 s/s]; s7-user-wd: 2h 21m 55s [-4.13 s/s] [23:39:28] @replag [23:39:29] DaBPunkt: s1-rr-a-wd: 1m 1s [+0.17 s/s]; s1-user-wd: 2h 50m 32s [-2.33 s/s]; s2-rr: 2h 29m 12s [-1.45 s/s]; s2-user: 2h 29m 12s [-1.45 s/s]; s2-user-c: error; s2-user-wd: 14h 50m 19s [-1.02 s/s]; s3-user: 24s [+0.26 s/s]; s3-user-wd: 14h 48m 49s [-2.43 s/s] [23:39:30] DaBPunkt: s4-user-wd: 58s [+0.11 s/s]; s5-rr-a: 44m 16s [-23.97 s/s]; s5-user: 1h 57m 30s [+0.83 s/s]; s5-user-c: error; s6-user-wd: 59s [+0.13 s/s]; s7-user-wd: 2h 18m 15s [-4.69 s/s] [23:46:46] @replag [23:46:46] DaBPunkt: s1-rr-a-wd: 54s [-0.06 s/s]; s1-user-wd: 2h 24m 53s [-3.88 s/s]; s2-rr: 2h 16m 1s [-1.54 s/s]; s2-user: 2h 16m 1s [-1.54 s/s]; s2-user-c: error; s2-user-wd: 14h 44m 33s [-0.59 s/s]; s3-user: 10s [-]; s3-user-wd: 14h 26m 28s [-3.62 s/s] [23:46:48] DaBPunkt: s4-user-wd: 54s [-0.08 s/s]; s5-user: 2h 4m 26s [+1.01 s/s]; s5-user-c: error; s6-user-wd: 54s [-0.11 s/s]; s7-user-wd: 1h 46m 17s [-4.53 s/s] [23:54:06] DaB. * Re: [Toolserver-announce] [Toolserver-l] 2. Try: sql-s5 (dewiki) will be read-only tomorrow [23:54:17] good night [23:55:21] [[User:DGideas]] ! 10https://wiki.toolserver.org/w/index.php?diff=7833&oldid=7807&rcid=21613 * DGideas * (-11) ()