[00:01:54] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [00:06:03] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [00:06:03] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [00:15:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [00:23:13] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [00:26:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [00:31:53] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [00:31:54] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 56595 MB (9% inode=99%): [00:48:53] /sql on adenia is CRITICAL: DISK CRITICAL - free space: /sql 543 MB (0% inode=89%): [01:01:53] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [01:06:03] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [01:07:03] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [01:15:04] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [01:23:12] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:26:13] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [01:31:54] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [01:31:54] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 56425 MB (9% inode=99%): [01:57:54] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 87608 MB (9% inode=98%): [01:57:54] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 87608 MB (9% inode=98%): [02:02:03] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [02:06:03] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [02:07:04] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [02:15:03] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [02:23:13] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [02:26:12] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [02:31:54] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [02:31:54] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 56282 MB (9% inode=99%): [03:02:03] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [03:06:04] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [03:08:03] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [03:15:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [03:23:28] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [03:26:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [03:32:02] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [03:32:02] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 56156 MB (9% inode=99%): [03:41:01] /sql on z-dat-s7-a is WARNING: DISK WARNING - free space: /sql 43476 MB (10% inode=99%): [03:44:01] /sql on z-dat-s7-a is OK: DISK OK - free space: /sql 45883 MB (11% inode=99%): [03:53:31] MySQL on adenia is CRITICAL: Cant connect to MySQL server on adenia (146) [04:02:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [04:04:32] MySQL on adenia is OK: Uptime: 689 Threads: 29 Questions: 455 Slow queries: 6 Opens: 48 Flush tables: 1 Open tables: 35 Queries per second avg: 0.660 [04:06:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [04:08:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [04:13:32] MySQL on adenia is CRITICAL: Cant connect to MySQL server on adenia (146) [04:15:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [04:23:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [04:26:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [04:32:01] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [04:32:02] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 56017 MB (9% inode=99%): [04:44:43] Warning: mysql_connect(): Unknown MySQL server host 'sql-toolserver' (11) in /home/liangent/code/gerrit/mediawiki/core/extensions/Toolserver/classes/Toolserver.php on line 52 [05:02:31] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [05:07:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [05:07:51] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24264 MB (5% inode=99%): [05:09:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [05:14:31] MySQL on adenia is CRITICAL: Cant connect to MySQL server on adenia (146) [05:15:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [05:23:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [05:27:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [05:32:02] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [05:32:02] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 55843 MB (9% inode=99%): [05:51:01] /sql on cassia is CRITICAL: DISK CRITICAL - free space: /sql 11518 MB (0% inode=94%): [05:56:51] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 30135 MB (7% inode=99%): [06:02:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [06:07:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [06:10:31] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [06:15:31] MySQL on adenia is CRITICAL: Cant connect to MySQL server on adenia (146) [06:15:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [06:22:02] /tmp on willow is WARNING: DISK WARNING - free space: / 21973 MB (20% inode=99%): [06:22:02] / on willow is WARNING: DISK WARNING - free space: / 21973 MB (20% inode=99%): [06:23:32] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [06:27:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [06:33:01] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [06:33:01] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 55505 MB (9% inode=99%): [07:02:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [07:07:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [07:10:31] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [07:15:32] MySQL on adenia is CRITICAL: Cant connect to MySQL server on adenia (146) [07:15:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [07:23:01] /tmp on willow is WARNING: DISK WARNING - free space: / 21836 MB (20% inode=99%): [07:23:01] / on willow is WARNING: DISK WARNING - free space: / 21836 MB (20% inode=99%): [07:23:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:27:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [07:33:02] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [07:33:02] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 55299 MB (9% inode=99%): [08:02:31] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [08:07:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [08:10:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [08:11:52] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 23895 MB (5% inode=99%): [08:15:32] MySQL on adenia is CRITICAL: Cant connect to MySQL server on adenia (146) [08:15:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [08:23:01] /tmp on willow is WARNING: DISK WARNING - free space: / 21705 MB (20% inode=99%): [08:23:01] / on willow is WARNING: DISK WARNING - free space: / 21705 MB (20% inode=99%): [08:23:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [08:27:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [08:33:02] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [08:33:02] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 55121 MB (9% inode=99%): [08:52:52] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 26138 MB (6% inode=99%): [09:02:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [09:08:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [09:10:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [09:15:32] MySQL on adenia is CRITICAL: Cant connect to MySQL server on adenia (146) [09:15:33] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [09:23:02] /tmp on willow is WARNING: DISK WARNING - free space: / 21573 MB (20% inode=99%): [09:23:02] / on willow is WARNING: DISK WARNING - free space: / 21573 MB (20% inode=99%): [09:23:32] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [09:27:33] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [09:33:02] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [09:33:02] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 54885 MB (8% inode=99%): [09:40:04] Maarten Dammers * [Toolserver-l] Fwd: [Wiki Loves Monuments] toolsever crashed? [09:50:32] @replag [09:50:32] liangent: s1-rr-a: 35s [+0.00 s/s]; s1-user: 35s [+0.00 s/s]; s2-user: 5m 56s [+0.01 s/s]; s3-rr-a: 14s [-0.00 s/s]; s3-user: 14s [-0.00 s/s] [09:50:41] @replag [09:50:41] liangent: s1-rr-a: 32s [-0.32 s/s]; s1-user: 32s [-0.32 s/s]; s2-user: 5m 56s [-] [09:50:46] @replag [09:50:46] liangent: s1-rr-a: 29s [-0.60 s/s]; s1-user: 29s [-0.60 s/s]; s2-user: 6m 1s [+0.99 s/s]; s3-rr-a: 12s [-0.14 s/s]; s3-user: 12s [-0.14 s/s] [09:50:50] @replag [09:50:51] liangent: s1-rr-a: 26s [-0.69 s/s]; s1-user: 26s [-0.69 s/s]; s2-user: 6m 5s [+0.92 s/s]; s3-rr-a: 16s [+0.92 s/s]; s3-user: 16s [+0.92 s/s] [09:50:58] @replag [09:50:59] liangent: s1-rr-a: 27s [+0.12 s/s]; s1-user: 27s [+0.12 s/s]; s2-user: 5m 57s [-1.00 s/s]; s3-rr-a: 24s [+1.00 s/s]; s3-user: 24s [+1.00 s/s] [09:52:02] liangent: what's the point of asking for the replag multiple times per minute? [09:52:27] valhallasw: to see trend [09:52:34] MySQL on adenia is OK: Uptime: 629 Threads: 59 Questions: 1222 Slow queries: 4 Opens: 90 Flush tables: 1 Open tables: 80 Queries per second avg: 1.942 [09:52:41] s2 is jerking [09:53:00] no, you are just sampling too fast [09:53:17] @replag [09:53:28] valhallasw: s2-user: 6m 53s [+0.40 s/s]; s3-rr-a: 2m 2s [+0.71 s/s]; s3-user: 2m 13s [+0.73 s/s]; s4-user: 11s [+0.00 s/s]; s6-rr-a: 28s [+0.00 s/s]; s6-user: 28s [+0.00 s/s]; s7-rr-a: 24s [+0.00 s/s]; s7-user: 24s [+0.00 s/s] [09:53:40] now you can say something about s2 [09:54:39] valhallasw: so it's calculating slope from the previous @replag request? [09:54:43] I think so, yes [09:55:09] but you can also just look at the numbers [09:55:30] in your requests, s2 replag was 5m56, 5m56, 6m05, 5m57 [09:55:49] @replag [09:55:49] valhallasw: s2-user: 6m 21s [-0.21 s/s]; s3-rr-a: 37s [-0.56 s/s]; s3-user: 37s [-0.68 s/s]; s5-rr-a: 45s [+0.00 s/s]; s5-user: 45s [+0.00 s/s] [09:56:23] apparently even sampling every few minutes is problematic ;-) [09:57:02] /tmp on willow is OK: DISK OK - free space: / 22908 MB (21% inode=99%): [09:57:02] / on willow is OK: DISK OK - free space: / 22908 MB (21% inode=99%): [09:57:13] valhallasw: it look like a bug to me [09:57:30] it should do slope from say, the most recent hour [09:57:50] {{sofixit}} [09:58:36] calculating the slope for the last hour is difficult (sample continuously), calculating it from the last request is easy (just store the last value) [10:02:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [10:08:33] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [10:10:16] hello [10:10:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [10:10:37] db-host sql is running again but if you can free some space please do [10:15:12] nosy: I can move the erfgoed databases to another server if that helps [10:15:33] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [10:16:38] multichill: unfortunatelly i dont think so [10:16:46] they are not so very big [10:17:00] i have one user who is currently using a third of all space [10:17:04] Marlen Caemmerer * Re: [Toolserver-l] Fwd: [Wiki Loves Monuments] toolsever crashed? [10:17:14] i wrote him he is usually quite often online [10:17:15] * multichill is just using a lot of connections [10:17:15] _mysql_exceptions.OperationalError: (1226, "User 'erfgoed' has exceeded the 'max_user_connections' resource (current value: 45)") [10:17:27] so i hope he will free some space [10:17:49] nosy: Can you see if user "erfgoed" has some stuck connections? [10:18:52] multichill: it always seems to be the same point where it waits [10:18:55] | 1944 | erfgoed | wolfsbane.toolserver.org:58089 | p_erfgoed_p | Query | 563 | Waiting for table | SELECT `country`, `lang`, `id`, `country`, `lang`, `id`, `name`, `address`, `municipality`, `lat`, | [10:19:22] Can you just kill all the queries started from the webservers? [10:20:08] yes [10:21:28] Thank you nosy [10:22:47] multichill: done [10:23:33] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:28:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [10:33:02] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [10:33:02] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 54656 MB (8% inode=99%): [11:02:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [11:08:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [11:10:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [11:16:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [11:24:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [11:26:01] /sql on adenia is WARNING: DISK WARNING - free space: /sql 50953 MB (7% inode=99%): [11:27:01] /sql on adenia is CRITICAL: DISK CRITICAL - free space: /sql 35035 MB (5% inode=99%): [11:28:33] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [11:31:02] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 118128 MB (12% inode=99%): [11:33:01] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [11:33:02] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 54458 MB (8% inode=99%): [12:02:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [12:09:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [12:10:31] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [12:16:33] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [12:24:32] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:28:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [12:33:01] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [12:33:02] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 54308 MB (8% inode=99%): [13:02:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [13:09:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [13:10:34] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [13:16:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [13:24:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [13:29:31] /home on hemlock is WARNING: DISK WARNING - free space: /home 10103 MB (19% inode=78%): [13:29:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [13:33:01] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [13:33:01] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 53993 MB (8% inode=99%): [13:39:02] /sql on adenia is WARNING: DISK WARNING - free space: /sql 39240 MB (6% inode=99%): [13:40:02] /sql on adenia is CRITICAL: DISK CRITICAL - free space: /sql 32563 MB (5% inode=99%): [13:41:46] hello all [13:51:01] /sql on adenia is WARNING: DISK WARNING - free space: /sql 39594 MB (6% inode=99%): [13:58:02] /sql on z-dat-s6-a is WARNING: DISK WARNING - free space: /sql 89228 MB (9% inode=98%): [13:58:02] /sql on z-dat-s3-a is WARNING: DISK WARNING - free space: /sql 89228 MB (9% inode=98%): [14:02:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [14:07:20] fisheye.toolserver.org on web.amaranth is WARNING: HTTP WARNING: HTTP/1.1 200 OK - 272 bytes in 17.304 second response time [14:08:21] fisheye.toolserver.org on web.amaranth is CRITICAL: CRITICAL - Socket timeout after 21 seconds [14:08:32] /home on hemlock is OK: DISK OK - free space: /home 25452 MB (51% inode=90%): [14:09:11] fisheye.toolserver.org on web.amaranth is OK: HTTP OK: HTTP/1.1 200 OK - 273 bytes in 8.990 second response time [14:09:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [14:11:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [14:16:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [14:25:32] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [14:28:01] /sql on adenia is CRITICAL: DISK CRITICAL - free space: /sql 33857 MB (5% inode=99%): [14:30:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [14:31:02] /sql on adenia is WARNING: DISK WARNING - free space: /sql 40262 MB (6% inode=99%): [14:32:01] /sql on adenia is CRITICAL: DISK CRITICAL - free space: /sql 34394 MB (5% inode=99%): [14:33:02] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [14:33:02] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 53692 MB (8% inode=99%): [14:58:02] /sql on adenia is WARNING: DISK WARNING - free space: /sql 39199 MB (6% inode=99%): [14:59:02] /sql on adenia is CRITICAL: DISK CRITICAL - free space: /sql 35943 MB (5% inode=99%): [15:02:28] valhallasw: hallo. Do you have a moment of time? [15:02:31] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [15:02:53] DaBPunkt: yes [15:03:55] valhallasw: I try to run /home/valhallasw/unicodetest.py to detect if I build python correct. However it fails with "TypeError: normalize() argument 2 must be unicode, not str" [15:04:13] if the script working? [15:04:29] if ≥ is [15:05:26] DaBPunkt: should work now [15:05:39] DaBPunkt: this was the python3 version, not the python 2 version [15:05:47] it should now work for both [15:06:05] ok. It fails now for 2.5, but this is not important [15:09:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [15:11:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [15:14:35] mm, it still fails [15:16:32] APT on yarrow is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. run with -v for information. [15:16:33] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [15:25:33] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [15:30:33] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [15:33:02] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 53455 MB (8% inode=99%): [15:33:02] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [16:02:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [16:09:31] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [16:11:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [16:17:31] APT on yarrow is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. run with -v for information. [16:17:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [16:17:51] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24282 MB (5% inode=99%): [16:20:51] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 26534 MB (6% inode=99%): [16:25:33] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:31:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [16:34:03] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [16:34:03] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 53220 MB (8% inode=99%): [17:02:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [17:09:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [17:17:31] APT on yarrow is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. run with -v for information. [17:18:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [17:26:32] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [17:31:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [17:34:02] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [17:35:01] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 52963 MB (8% inode=99%): [17:50:01] /sql on adenia is WARNING: DISK WARNING - free space: /sql 40234 MB (6% inode=99%): [17:51:01] /sql on cassia is CRITICAL: DISK CRITICAL - free space: /sql 12388 MB (1% inode=95%): [17:52:02] /sql on adenia is CRITICAL: DISK CRITICAL - free space: /sql 38556 MB (5% inode=99%): [17:53:02] /sql on adenia is WARNING: DISK WARNING - free space: /sql 40312 MB (6% inode=99%): [18:02:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [18:09:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [18:11:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [18:17:32] APT on yarrow is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. run with -v for information. [18:18:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [18:27:33] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [18:32:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [18:34:01] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [18:36:01] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 52696 MB (8% inode=99%): [19:02:33] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [19:09:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [19:11:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [19:18:31] APT on yarrow is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. run with -v for information. [19:18:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [19:28:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:32:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [19:34:02] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [19:36:02] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 52497 MB (8% inode=99%): [20:02:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [20:09:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [20:11:31] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [20:18:32] APT on yarrow is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. run with -v for information. [20:18:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [20:21:05] Simon Walker * Re: [Toolserver-l] sql.toolserver.org will become deprecated [20:29:04] DaB. * Re: [Toolserver-l] sql.toolserver.org will become deprecated [20:29:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [20:32:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [20:35:01] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [20:36:02] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 52259 MB (8% inode=99%): [21:02:32] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [21:09:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [21:11:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [21:18:32] APT on yarrow is WARNING: APT WARNING: 4 packages available for upgrade (0 critical updates). [21:18:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [21:26:02] /sql on z-dat-s7-a is WARNING: DISK WARNING - free space: /sql 44577 MB (10% inode=99%): [21:30:32] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [21:32:31] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [21:35:02] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [21:36:01] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 51972 MB (8% inode=99%): [22:02:31] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [22:10:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [22:11:31] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [22:14:07] nacht ts [22:19:31] APT on yarrow is WARNING: APT WARNING: 4 packages available for upgrade (0 critical updates). [22:19:31] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [22:31:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [22:32:40] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [22:36:02] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [22:36:02] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 51763 MB (8% inode=99%): [23:02:33] Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output [23:10:32] Sun Grid Engine execd on ortelius is WARNING: NRPE: Unable to read output [23:11:32] FMA on amaranth is CRITICAL: Failed components: hc://:product-id=SUN-FIRE-X4150:server-id=amaranth:chassis-id=0819QAR1D1:serial=518545072303039020:part=72T256520HFD3SB:revision=--/motherboard=0/memory-controller=1/dram-channel=2/dimm=3/rank=7 [23:19:32] APT on yarrow is WARNING: APT WARNING: 4 packages available for upgrade (0 critical updates). [23:19:32] Sun Grid Engine execd on wolfsbane is WARNING: NRPE: Unable to read output [23:31:02] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 118425 MB (12% inode=99%): [23:31:31] Environment IPMI on thyme is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [23:33:32] SMF on web.amaranth is CRITICAL: ERROR - maintenance: svc:/application/jira:default [23:36:02] SRaid on nightshade is CRITICAL: NRPE: Unable to read output [23:36:02] /sql on ptolemy is CRITICAL: DISK CRITICAL - free space: /sql 51463 MB (8% inode=99%): [23:53:01] /sql on z-dat-s7-a is OK: DISK OK - free space: /sql 44878 MB (11% inode=99%): [23:58:50] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 24157 MB (5% inode=99%): [23:59:51] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 27496 MB (6% inode=99%):