[00:25:02] [[Special:Log/newusers]] create 10 * Agent 61 * (New user account) [02:28:04] K. Peachey * Re: [Toolserver-l] Toolserver wiki [03:25:14] 2013/10/01 03:18 CRIT yarrow /tmp DISK CRITICAL - free space: /tmp 0 MB (0% inode=99%): [03:26:14] 2013/10/01 03:25 OK yarrow /tmp DISK OK - free space: /tmp 4086 MB (96% inode=99%): [04:02:14] 2013/10/01 04:01 OK wolfsbane / DISK OK - free space: / 7232 MB (24% inode=93%): [04:02:14] 2013/10/01 04:01 OK wolfsbane /tmp DISK OK - free space: / 7271 MB (24% inode=93%): [05:49:20] 2013/10/01 05:48 CRIT z-dat-s4-a /sql CHECK_NRPE: Socket timeout after 30 seconds. [05:50:20] 2013/10/01 05:49 WARN z-dat-s4-a /sql DISK WARNING - free space: /sql 59264 MB (9% inode=99%): [05:52:20] 2013/10/01 05:48 CRIT z-dat-s4-a MySQL slave (Service Check Timed Out) [05:54:21] 2013/10/01 05:50 CRIT z-dat-s6-a MySQL Can't connect to MySQL server on 'z-dat-s6-a' (110) [05:55:21] 2013/10/01 05:48 CRIT hyacinth Load avg. Timeout while attempting connection [05:55:21] 2013/10/01 05:48 CRIT hyacinth RAID Timeout while attempting connection [05:55:21] 2013/10/01 05:51 CRIT z-dat-s7-a MySQL slave Can't connect to MySQL server on 'z-dat-s7-a' (110) [05:56:21] 2013/10/01 05:49 CRIT hyacinth / Timeout while attempting connection [05:56:21] 2013/10/01 05:49 CRIT hyacinth /tmp Timeout while attempting connection [05:56:21] 2013/10/01 05:49 CRIT hyacinth Environment IPMI Timeout while attempting connection [05:56:21] 2013/10/01 05:49 CRIT hyacinth SMTP CRITICAL - Socket timeout after 10 seconds [05:56:21] 2013/10/01 05:49 CRIT hyacinth SSH CRITICAL - Socket timeout after 10 seconds [05:56:22] 2013/10/01 05:48 CRIT z-dat-s3-a MySQL Can't connect to MySQL server on 'z-dat-s3-a' (110) [05:56:22] 2013/10/01 05:51 CRIT z-dat-s7-a MySQL Can't connect to MySQL server on 'z-dat-s7-a' (110) [05:57:21] 2013/10/01 05:56 OK hyacinth / DISK OK - free space: / 9139 MB (30% inode=86%): [05:57:21] 2013/10/01 05:57 OK hyacinth /tmp DISK OK - free space: /tmp 1988 MB (99% inode=99%): [05:57:21] 2013/10/01 05:57 OK hyacinth Environment IPMI ok: temperature ok fan ok voltage ok chassis ok [05:57:21] 2013/10/01 05:56 OK hyacinth Load avg. OK - load average: 0.06, 0.43, 1.09 [05:57:21] 2013/10/01 05:49 CRIT z-dat-s3-a MySQL slave Can't connect to MySQL server on 'z-dat-s3-a' (110) [05:57:22] 2013/10/01 05:48 CRIT z-dat-s4-a MySQL (Service Check Timed Out) [05:57:22] 2013/10/01 05:50 CRIT z-dat-s6-a MySQL slave Can't connect to MySQL server on 'z-dat-s6-a' (110) [05:58:21] 2013/10/01 05:58 OK hyacinth SMTP SMTP OK - 0.629 sec. response time [05:58:21] 2013/10/01 05:58 OK hyacinth SSH SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [05:58:21] 2013/10/01 05:57 OK z-dat-s3-a MySQL slave Uptime: 161216 Threads: 14 Questions: 183084156 Slow queries: 2789 Opens: 1764956 Flush tables: 1 Open tables: 16384 Queries per second avg: 1135.645 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 591 [05:58:21] 2013/10/01 05:57 OK z-dat-s4-a MySQL slave Uptime: 2317955 Threads: 7 Questions: 170739069 Slow queries: 101 Opens: 319 Flush tables: 1 Open tables: 250 Queries per second avg: 73.659 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 552 [05:59:21] 2013/10/01 05:58 OK hyacinth RAID OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [05:59:21] 2013/10/01 05:58 OK z-dat-s3-a MySQL Uptime: 161234 Threads: 17 Questions: 183138476 Slow queries: 2789 Opens: 1767245 Flush tables: 1 Open tables: 16384 Queries per second avg: 1135.855 [05:59:21] 2013/10/01 05:57 OK z-dat-s4-a MySQL Uptime: 2317964 Threads: 1 Questions: 170746908 Slow queries: 101 Opens: 319 Flush tables: 1 Open tables: 250 Queries per second avg: 73.662 [05:59:21] 2013/10/01 05:58 OK z-dat-s6-a MySQL Uptime: 2317985 Threads: 7 Questions: 697608616 Slow queries: 65416 Opens: 710610 Flush tables: 1 Open tables: 3262 Queries per second avg: 300.954 [05:59:21] 2013/10/01 05:58 OK z-dat-s6-a MySQL slave Uptime: 2317985 Threads: 8 Questions: 697608614 Slow queries: 65416 Opens: 710610 Flush tables: 1 Open tables: 3262 Queries per second avg: 300.954 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 212 [05:59:22] 2013/10/01 05:58 OK z-dat-s7-a MySQL Uptime: 1183385 Threads: 8 Questions: 224620753 Slow queries: 12947 Opens: 1128710 Flush tables: 1 Open tables: 3024 Queries per second avg: 189.812 [05:59:22] 2013/10/01 05:58 OK z-dat-s7-a MySQL slave Uptime: 1183385 Threads: 8 Questions: 224620754 Slow queries: 12947 Opens: 1128710 Flush tables: 1 Open tables: 3024 Queries per second avg: 189.812 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 307 [06:05:22] 2013/10/01 05:58 CRIT nightshade aliasd Connection refused [07:35:27] 2013/10/01 07:31 WARN ortelius toolserver.org HTTP HTTP WARNING: HTTP/1.1 200 OK - 239 bytes in 0.501 second response time [07:36:27] 2013/10/01 07:35 CRIT ortelius toolserver.org HTTP HTTP CRITICAL: HTTP/1.1 200 OK - 239 bytes in 8.372 second response time [07:37:27] 2013/10/01 07:36 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.138 second response time [10:49:37] 2013/10/01 10:42 CRIT ortelius toolserver.org HTTP HTTP CRITICAL: HTTP/1.1 200 OK - 239 bytes in 1.090 second response time [10:50:37] 2013/10/01 10:49 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.010 second response time [10:50:37] 2013/10/01 10:50 CRIT willow / DISK CRITICAL - free space: / 11108 MB (10% inode=97%): [10:51:37] 2013/10/01 10:50 CRIT willow /tmp DISK CRITICAL - free space: / 10449 MB (9% inode=97%): [11:07:38] 2013/10/01 11:00 CRIT ortelius toolserver.org HTTP HTTP CRITICAL: HTTP/1.1 200 OK - 239 bytes in 1.158 second response time [11:13:40] 2013/10/01 11:12 WARN ortelius toolserver.org HTTP HTTP WARNING: HTTP/1.1 200 OK - 239 bytes in 0.725 second response time [11:14:40] 2013/10/01 11:13 CRIT ortelius toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [11:19:40] 2013/10/01 11:18 WARN ortelius toolserver.org HTTP HTTP WARNING: HTTP/1.1 200 OK - 239 bytes in 0.661 second response time [11:20:40] 2013/10/01 11:19 CRIT ortelius toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [11:31:42] 2013/10/01 11:30 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.499 second response time [11:31:42] 2013/10/01 11:25 CRIT wolfsbane toolserver.org HTTP HTTP CRITICAL: HTTP/1.1 200 OK - 239 bytes in 1.298 second response time [11:34:41] 2013/10/01 11:34 OK wolfsbane toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.009 second response time [11:38:41] 2013/10/01 11:31 CRIT ortelius toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [11:44:42] 2013/10/01 11:39 CRIT wolfsbane toolserver.org HTTP HTTP CRITICAL: HTTP/1.1 200 OK - 239 bytes in 2.157 second response time [11:51:43] 2013/10/01 11:51 OK wolfsbane toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.003 second response time [12:00:43] 2013/10/01 11:54 CRIT wolfsbane toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [12:15:05] Silke Meyer * Re: [Toolserver-l] Toolserver wiki [12:38:45] 2013/10/01 12:38 OK wolfsbane toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.238 second response time [12:43:09] what's wrong with the toolserver? [12:43:44] http://pastebin.com/0X5dmsAm [12:45:45] 2013/10/01 12:39 CRIT wolfsbane toolserver.org HTTP CRITICAL - Socket timeout after 10 seconds [12:48:43] @admin does this work? [12:48:50] !admin does this work? [12:49:03] @replag [12:49:03] malafaya: s1-rr-a-wd: error; s1-user-wd: 1d 2h 39m 55s [+0.07 s/s]; s2-rr: 14s [-0.00 s/s]; s2-user: 14s [-0.00 s/s]; s2-user-c: error; s2-user-wd: 1d 2h 39m 55s [+0.16 s/s]; s3-user: 16s [-0.00 s/s]; s3-user-wd: 11w 3d 18h 28m 16s [+1.00 s/s] [12:49:04] malafaya: s4-user-wd: 7w 2d 23h 44m 46s [+1.00 s/s]; s5-rr-a-wd: 1d 2h 39m 55s [+0.07 s/s]; s6-user-wd: 7w 2d 23h 44m 46s [+1.00 s/s]; s7-user-wd: 7w 2d 23h 44m 46s [+1.00 s/s] [12:49:07] Yes, but pretty useless here in this chanel [12:49:14] malafaya: Don't play around [12:49:27] i'm not playing [12:49:32] what's wrong with the TS? [12:49:36] what was the URL that gave you that error? [12:49:40] any [12:49:44] https://toolserver.org/~tparis/pcount/ [12:49:48] https://toolserver.org/ [12:57:13] malafaya: looks like the webservers are down [13:01:20] and no one in this channel is able to do anything i suppose [13:01:27] #wikimedia-tech? [13:09:46] 2013/10/01 13:09 OK wolfsbane toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.007 second response time [13:16:46] 2013/10/01 13:10 CRIT wolfsbane toolserver.org HTTP HTTP CRITICAL: HTTP/1.1 200 OK - 239 bytes in 6.564 second response time [13:20:59] hi [13:21:26] does anyone know if toolserver.org being down is in any way related to that request for more disk space..or not [13:26:07] Danny_B: poke ^ [13:26:39] Might be a single server [13:44:47] 2013/10/01 13:44 CRIT wolfsbane Sun Grid Engine execd Connection refused by host [13:45:47] 2013/10/01 13:44 CRIT wolfsbane / Timeout while attempting connection [13:45:47] 2013/10/01 13:44 CRIT wolfsbane /tmp Timeout while attempting connection [13:45:47] 2013/10/01 13:44 CRIT wolfsbane Environment IPMI Connection refused or timed out [13:45:47] 2013/10/01 13:44 CRIT wolfsbane Load avg. Connection refused or timed out [13:45:47] 2013/10/01 13:44 CRIT wolfsbane NTP CRITICAL - Socket timeout after 10 seconds [13:45:48] 2013/10/01 13:44 CRIT wolfsbane PING PING CRITICAL - Packet loss = 100% [13:45:48] 2013/10/01 13:45 CRIT wolfsbane SMTP CRITICAL - Socket timeout after 10 seconds [13:45:49] 2013/10/01 13:45 CRIT wolfsbane SSH CRITICAL - Socket timeout after 10 seconds [13:47:47] 2013/10/01 13:47 OK wolfsbane / DISK OK - free space: / 8912 MB (29% inode=93%): [13:47:47] 2013/10/01 13:47 OK wolfsbane /tmp DISK OK - free space: / 8912 MB (29% inode=93%): [13:47:47] 2013/10/01 13:46 OK wolfsbane Load avg. OK - load average: 2.29, 0.73, 0.26 [13:47:47] 2013/10/01 13:46 OK wolfsbane PING PING OK - Packet loss = 0%, RTA = 0.34 ms [13:47:47] 2013/10/01 13:47 OK wolfsbane SMTP SMTP OK - 0.186 sec. response time [13:47:48] 2013/10/01 13:47 OK wolfsbane SSH SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [13:47:48] 2013/10/01 13:47 WARN wolfsbane Sun Grid Engine execd NRPE: Unable to read output [13:48:47] 2013/10/01 13:47 OK wolfsbane Environment IPMI ok: temperature ok fan ok voltage ok chassis ok [13:57:47] 2013/10/01 13:57 CRIT ortelius Sun Grid Engine execd Timeout while attempting connection [13:58:47] 2013/10/01 13:57 CRIT ortelius / Timeout while attempting connection [13:58:48] 2013/10/01 13:57 CRIT ortelius /tmp Timeout while attempting connection [13:58:48] 2013/10/01 13:58 CRIT ortelius DiskSuite Timeout while attempting connection [13:58:48] 2013/10/01 13:57 CRIT ortelius Environment IPMI Timeout while attempting connection [13:58:48] 2013/10/01 13:57 CRIT ortelius Load avg. Timeout while attempting connection [13:58:48] 2013/10/01 13:58 CRIT ortelius NTP CRITICAL - Socket timeout after 10 seconds [13:58:48] 2013/10/01 13:58 CRIT ortelius PING PING CRITICAL - Packet loss = 100% [13:58:49] 2013/10/01 13:57 CRIT ortelius SMTP CRITICAL - Socket timeout after 10 seconds [13:58:49] 2013/10/01 13:58 CRIT ortelius SSH CRITICAL - Socket timeout after 10 seconds [13:59:47] 2013/10/01 13:59 OK ortelius Environment IPMI ok: temperature ok fan ok voltage ok chassis ok [13:59:48] 2013/10/01 13:59 OK ortelius PING PING OK - Packet loss = 0%, RTA = 0.48 ms [14:00:48] 2013/10/01 13:59 OK ortelius / DISK OK - free space: / 13857 MB (46% inode=91%): [14:00:48] 2013/10/01 13:59 OK ortelius /tmp DISK OK - free space: /tmp 27336 MB (99% inode=99%): [14:00:48] 2013/10/01 14:00 OK ortelius Load avg. OK - load average: 1.44, 0.76, 0.30 [14:00:48] 2013/10/01 14:00 OK ortelius NTP NTP OK: Offset -0.055643 secs [14:00:48] 2013/10/01 14:00 OK ortelius SMTP SMTP OK - 0.278 sec. response time [14:00:48] 2013/10/01 14:00 OK ortelius SSH SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [14:00:48] 2013/10/01 14:00 WARN ortelius Sun Grid Engine execd NRPE: Unable to read output [14:07:47] 2013/10/01 14:07 OK wolfsbane toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.011 second response time [14:08:47] 2013/10/01 14:07 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.003 second response time [14:17:48] 2013/10/01 14:16 CRIT ortelius toolserver.org HTTP HTTP CRITICAL: HTTP/1.1 200 OK - 239 bytes in 1.262 second response time [14:21:48] 2013/10/01 14:20 OK ortelius toolserver.org HTTP HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.029 second response time [15:13:44] ah, it's back [16:22:34] [[Special:Log/newusers]] create 10 * Iraqi1Bot * (New user account) [16:37:22] [[Special:Log/newusers]] create 10 * عراقي1 * (New user account) [16:52:54] 2013/10/01 16:46 WARN wolfsbane / DISK WARNING - free space: / 5099 MB (17% inode=93%): [16:53:54] 2013/10/01 16:47 WARN wolfsbane /tmp DISK WARNING - free space: / 5112 MB (17% inode=93%): [18:07:58] 2013/10/01 18:07 OK wolfsbane / DISK OK - free space: / 7997 MB (26% inode=93%): [18:07:58] 2013/10/01 18:07 OK wolfsbane /tmp DISK OK - free space: / 7994 MB (26% inode=93%): [21:38:04] 2013/10/01 21:35 WARN wolfsbane Load avg. WARNING - load average: 4.81, 16.57, 10.67 [21:39:04] 2013/10/01 21:38 OK wolfsbane Load avg. OK - load average: 3.07, 13.98, 10.12 [23:29:09] 2013/10/01 23:28 CRIT amaranth PING PING CRITICAL - Packet loss = 100% [23:29:10] 2013/10/01 23:28 CRIT amaranth SMTP CRITICAL - Socket timeout after 10 seconds [23:29:10] 2013/10/01 23:28 CRIT amaranth SSH CRITICAL - Socket timeout after 10 seconds [23:29:10] 2013/10/01 23:28 CRIT web.amaranth PING PING CRITICAL - Packet loss = 100% [23:29:10] 2013/10/01 23:28 WARN web.amaranth jira.toolserver.org HTTP WARNING: HTTP/1.1 200 OK - 524 bytes in 9.536 second response time [23:29:10] 2013/10/01 23:28 CRIT web.amaranth wiki.toolserver.org CRITICAL - Socket timeout after 16 seconds [23:30:10] 2013/10/01 23:29 OK amaranth PING PING OK - Packet loss = 0%, RTA = 122.83 ms [23:30:10] 2013/10/01 23:29 OK amaranth SMTP SMTP OK - 0.505 sec. response time [23:30:10] 2013/10/01 23:29 OK amaranth SSH SSH OK - OpenSSH_5.6p1-hpn13v10 (protocol 2.0) [23:30:10] 2013/10/01 23:29 OK web.amaranth PING PING OK - Packet loss = 0%, RTA = 121.43 ms [23:30:10] 2013/10/01 23:29 OK web.amaranth jira.toolserver.org HTTP OK: HTTP/1.1 200 OK - 524 bytes in 0.522 second response time [23:30:10] 2013/10/01 23:29 OK web.amaranth wiki.toolserver.org HTTP OK: HTTP/1.1 200 OK - 349 bytes in 0.734 second response time