[03:08:17] 2013/11/11 03:07 OK wolfsbane / DISK OK - free space: / 7319 MB (24% inode=93%): [03:08:17] 2013/11/11 03:07 OK wolfsbane /tmp DISK OK - free space: / 7320 MB (24% inode=93%): [06:24:28] 2013/11/11 06:23 WARN z-dat-s6-a MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2980 [06:26:28] 2013/11/11 06:25 OK z-dat-s6-a MySQL slave Uptime: 241805 Threads: 4 Questions: 34807078 Slow queries: 18782 Opens: 14311 Flush tables: 1 Open tables: 914 Queries per second avg: 143.946 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1097 [06:30:28] 2013/11/11 06:30 WARN cassia s5 replag QUERY WARNING: 'SELECT ts_rc_age()' returned 3404.000000 [06:31:28] 2013/11/11 06:30 WARN cassia MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1935 [06:31:28] 2013/11/11 06:31 OK cassia s5 replag QUERY OK: 'SELECT ts_rc_age()' returned 18.000000 [06:32:28] 2013/11/11 06:31 OK cassia MySQL slave Uptime: 12867939 Threads: 9 Questions: 7907405730 Slow queries: 277640 Opens: 30563 Flush tables: 1 Open tables: 14347 Queries per second avg: 614.504 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [06:35:29] 2013/11/11 06:34 WARN z-dat-s7-a MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3291 [06:38:28] 2013/11/11 06:37 OK z-dat-s7-a MySQL slave Uptime: 2668121 Threads: 7 Questions: 418260801 Slow queries: 634598 Opens: 1018179 Flush tables: 1 Open tables: 3542 Queries per second avg: 156.762 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1389 [08:39:34] 2013/11/11 08:38 WARN z-dat-s3-a MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3330 [08:46:35] 2013/11/11 08:45 OK z-dat-s3-a MySQL slave Uptime: 44472 Threads: 25 Questions: 23573701 Slow queries: 2482 Opens: 141605 Flush tables: 1 Open tables: 16384 Queries per second avg: 530.79 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1486 [08:59:35] 2013/11/11 08:55 CRIT z-dat-s6-a MySQL Can't connect to MySQL server on 'z-dat-s6-a' (110) [09:00:35] 2013/11/11 08:59 OK z-dat-s6-a MySQL Uptime: 251065 Threads: 9 Questions: 36651123 Slow queries: 19394 Opens: 14612 Flush tables: 1 Open tables: 933 Queries per second avg: 145.982 [09:04:36] 2013/11/11 09:04 WARN z-dat-s2-b MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3240 [09:09:35] 2013/11/11 09:09 OK z-dat-s2-b MySQL slave Uptime: 4920024 Threads: 7 Questions: 2007961208 Slow queries: 35924 Opens: 16889914 Flush tables: 1 Open tables: 256 Queries per second avg: 408.120 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1773 [11:28:41] 2013/11/11 11:27 OK yarrow aliasd TCP OK - 0.001 second response time on port 984 [500 Not found.] [12:06:04] Marlen Caemmerer * [Toolserver-l] Maintenance: dumping commons/s4 database [12:06:41] 2013/11/11 12:00 CRIT cassia MySQL slave Slave IO: No Slave SQL: Yes Seconds Behind Master: (null) [12:07:42] 2013/11/11 12:07 OK cassia MySQL slave Uptime: 12888099 Threads: 9 Questions: 7930949121 Slow queries: 279779 Opens: 31259 Flush tables: 1 Open tables: 14347 Queries per second avg: 615.369 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1187 [12:09:41] 2013/11/11 12:02 CRIT z-dat-s3-a MySQL slave Slave IO: No Slave SQL: Yes Seconds Behind Master: (null) [12:16:43] 2013/11/11 12:10 CRIT cassia MySQL slave Slave IO: No Slave SQL: Yes Seconds Behind Master: (null) [12:28:44] 2013/11/11 12:22 WARN cassia s5 replag QUERY WARNING: 'SELECT ts_rc_age()' returned 2213.000000 [12:36:45] 2013/11/11 12:35 CRIT hyacinth / Connection refused or timed out [12:36:45] 2013/11/11 12:36 CRIT hyacinth /tmp Connection refused or timed out [12:36:45] 2013/11/11 12:36 CRIT hyacinth Environment IPMI Connection refused or timed out [12:36:45] 2013/11/11 12:35 CRIT hyacinth Load avg. Connection refused or timed out [12:36:45] 2013/11/11 12:36 CRIT hyacinth NTP CRITICAL - Socket timeout after 10 seconds [12:38:51] 2013/11/11 12:38 OK hyacinth PING PING OK - Packet loss = 0%, RTA = 0.13 ms [12:41:51] 2013/11/11 12:41 CRIT z-dat-s3-a CRITICAL - Host Unreachable (z-dat-s3-a) [12:41:52] 2013/11/11 12:41 CRIT z-dat-s6-a CRITICAL - Host Unreachable (z-dat-s6-a) [12:41:52] 2013/11/11 12:41 CRIT z-dat-s7-a CRITICAL - Host Unreachable (z-dat-s7-a) [12:43:52] 2013/11/11 12:36 CRIT hyacinth SSH CRITICAL - Socket timeout after 10 seconds [12:44:52] 2013/11/11 12:36 CRIT hyacinth /tmp Connection refused by host [12:44:52] 2013/11/11 12:36 CRIT hyacinth Environment IPMI Connection refused by host [12:44:52] 2013/11/11 12:36 CRIT hyacinth RAID Connection refused by host [12:44:52] 2013/11/11 12:36 CRIT hyacinth SMTP CRITICAL - Socket timeout after 10 seconds [12:45:52] 2013/11/11 12:35 CRIT hyacinth / Connection refused by host [12:45:52] 2013/11/11 12:35 CRIT hyacinth Load avg. Connection refused by host [12:52:52] 2013/11/11 12:52 CRIT cassia s5 replag QUERY CRITICAL: 'SELECT ts_rc_age()' returned 3653.000000 [12:57:52] 2013/11/11 12:57 CRIT hyacinth NTP CRITICAL - Socket timeout after 10 seconds [12:57:53] 2013/11/11 12:57 CRIT hyacinth PING CRITICAL - Host Unreachable (hyacinth) [12:59:53] 2013/11/11 12:59 OK hyacinth PING PING OK - Packet loss = 0%, RTA = 0.14 ms [13:01:53] 2013/11/11 13:01 OK cassia MySQL slave Uptime: 12891340 Threads: 8 Questions: 7936659126 Slow queries: 280313 Opens: 31259 Flush tables: 1 Open tables: 14347 Queries per second avg: 615.658 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [13:20:54] 2013/11/11 13:14 CRIT cassia MySQL slave Slave IO: No Slave SQL: Yes Seconds Behind Master: (null) [13:27:54] 2013/11/11 13:27 CRIT hyacinth PING CRITICAL - Host Unreachable (hyacinth) [13:28:54] 2013/11/11 13:28 CRIT hyacinth NTP CRITICAL - Socket timeout after 10 seconds [13:29:55] 2013/11/11 13:29 OK hyacinth PING PING OK - Packet loss = 0%, RTA = 0.21 ms [13:30:55] 2013/11/11 13:30 WARN cassia MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2904 [13:30:55] 2013/11/11 13:30 WARN cassia s5 replag QUERY WARNING: 'SELECT ts_rc_age()' returned 3168.000000 [13:31:56] 2013/11/11 13:31 OK cassia MySQL slave Uptime: 12893139 Threads: 9 Questions: 7940462447 Slow queries: 280615 Opens: 31259 Flush tables: 1 Open tables: 14347 Queries per second avg: 615.867 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1786 [13:32:56] 2013/11/11 13:32 OK cassia s5 replag QUERY OK: 'SELECT ts_rc_age()' returned 969.000000 [13:53:57] 2013/11/11 13:53 OK z-dat-s6-a PING OK - Packet loss = 0%, RTA = 0.28 ms [13:53:57] 2013/11/11 13:52 OK hyacinth / DISK OK - free space: / 8970 MB (29% inode=86%): [13:53:57] 2013/11/11 13:53 OK hyacinth /tmp DISK OK - free space: /tmp 62340 MB (100% inode=99%): [13:53:57] 2013/11/11 13:53 OK hyacinth Environment IPMI ok: temperature ok fan ok voltage ok chassis ok [13:53:57] 2013/11/11 13:52 OK hyacinth Load avg. OK - load average: 0.24, 0.07, 0.09 [13:53:57] 2013/11/11 13:53 OK hyacinth RAID OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [13:53:57] 2013/11/11 13:53 OK hyacinth SMTP SMTP OK - 0.431 sec. response time [13:53:58] 2013/11/11 13:53 OK hyacinth SSH SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [13:53:58] 2013/11/11 13:53 OK z-dat-s4-a / DISK OK - free space: / 9007 MB (30% inode=86%): [13:53:59] 2013/11/11 13:53 OK z-dat-s6-a Load avg. OK - load average: 0.67, 0.18, 0.12 [13:53:59] 2013/11/11 13:53 OK z-dat-s7-a PING PING OK - Packet loss = 0%, RTA = 0.25 ms [13:54:00] 2013/11/11 13:53 OK z-dat-s7-a SSH SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [13:54:57] 2013/11/11 13:54 OK z-dat-s3-a PING OK - Packet loss = 0%, RTA = 0.14 ms [13:54:57] 2013/11/11 13:54 OK z-dat-s4-a PING OK - Packet loss = 0%, RTA = 0.22 ms [13:54:57] 2013/11/11 13:53 OK z-dat-s7-a PING OK - Packet loss = 0%, RTA = 0.15 ms [13:54:57] 2013/11/11 13:54 OK z-dat-s3-a / DISK OK - free space: / 8983 MB (29% inode=86%): [13:54:57] 2013/11/11 13:54 OK z-dat-s3-a /sql DISK OK - free space: /sql 155952 MB (16% inode=99%): [13:57:03] 2013/11/11 12:35 CRIT z-dat-s7-a MySQL slave SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4905 [14:01:04] 2013/11/11 12:35 CRIT z-dat-s3-a MySQL Can't connect to MySQL server on 'z-dat-s3-a' (111) [14:01:04] 2013/11/09 22:22 CRIT z-dat-s4-a MySQL (Service Check Timed Out) [14:01:04] 2013/11/11 13:59 WARN z-dat-s6-a MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3231 [14:02:04] 2013/11/11 14:01 WARN z-dat-s7-a MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3286 [14:05:04] 2013/11/11 14:04 OK z-dat-s6-a MySQL slave Uptime: 653 Threads: 3 Questions: 244017 Slow queries: 25 Opens: 221 Flush tables: 1 Open tables: 202 Queries per second avg: 373.686 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1632 [14:07:04] 2013/11/11 14:06 OK z-dat-s7-a MySQL slave Uptime: 772 Threads: 6 Questions: 215931 Slow queries: 34 Opens: 608 Flush tables: 1 Open tables: 587 Queries per second avg: 279.703 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1421 [14:08:04] 2013/11/11 14:07 OK z-dat-s3-a MySQL Uptime: 813 Threads: 11 Questions: 4988 Slow queries: 1 Opens: 557 Flush tables: 1 Open tables: 550 Queries per second avg: 6.135 [14:44:07] 2013/11/11 14:43 WARN z-dat-s3-a MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3492 [14:53:08] 2013/11/11 14:52 OK z-dat-s3-a MySQL slave Uptime: 3513 Threads: 21 Questions: 1301329 Slow queries: 46 Opens: 14430 Flush tables: 1 Open tables: 14421 Queries per second avg: 370.432 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1750 [15:58:12] 2013/11/11 15:51 WARN wolfsbane / DISK WARNING - free space: / 6257 MB (20% inode=93%): [15:59:12] 2013/11/11 15:52 WARN wolfsbane /tmp DISK WARNING - free space: / 6271 MB (20% inode=93%): [16:37:04] Silke Meyer * [Toolserver-l] FYI: Toolserver account expiry in early January [16:42:24] Silke_WMDE misses a word in the last sentence [16:42:58] > Third, I would like to encourage all those who already moved over to Tool Labs to download and then their stuff themselves. [16:43:11] and then what their? [16:43:14] oh jeremyb is right. "delete" [16:43:39] "burnbridges" is a delete-all script mette wrote [16:43:44] ok, that was my second guess. first guess doesn't really make sense. (i was thinking migrate. but you're talking about already migrated people) [16:43:48] hah [16:57:04] Federico Leva (Nemo) * Re: [Toolserver-l] FYI: Toolserver account expiry in early January [17:17:42] amette_: apparently one of your scripts is giving some errors [17:17:49] i haven't seen first hand [17:23:48] the name burnbridges makes me sick :( [17:31:21] 2013/11/11 17:28 CRIT z-dat-s7-a SSH CRITICAL - Socket timeout after 10 seconds [17:32:21] 2013/11/11 17:28 CRIT z-dat-s4-a SMTP CRITICAL - Socket timeout after 10 seconds [17:35:21] 2013/11/11 17:28 CRIT z-dat-s3-a SMTP CRITICAL - Socket timeout after 10 seconds [17:35:21] 2013/11/11 17:28 CRIT z-dat-s3-a SSH CRITICAL - Socket timeout after 10 seconds [17:35:21] 2013/11/11 17:28 CRIT z-dat-s4-a SSH CRITICAL - Socket timeout after 10 seconds [17:35:21] 2013/11/11 17:28 CRIT z-dat-s7-a SMTP CRITICAL - Socket timeout after 10 seconds [17:36:21] 2013/11/11 17:29 CRIT z-dat-s6-a SMTP CRITICAL - Socket timeout after 10 seconds [17:36:21] 2013/11/11 17:29 CRIT z-dat-s6-a SSH CRITICAL - Socket timeout after 10 seconds [17:50:22] 2013/11/11 17:43 CRIT z-dat-s3-a MySQL Can't connect to MySQL server on 'z-dat-s3-a' (111) [17:50:22] 2013/11/11 17:43 CRIT z-dat-s3-a MySQL slave Can't connect to MySQL server on 'z-dat-s3-a' (111) [18:01:22] 2013/11/11 18:00 OK z-dat-s3-a MySQL Uptime: 1055 Threads: 13 Questions: 13829 Slow queries: 1 Opens: 538 Flush tables: 1 Open tables: 529 Queries per second avg: 13.108 [18:01:23] 2013/11/11 18:00 OK z-dat-s3-a MySQL slave Uptime: 1055 Threads: 13 Questions: 13829 Slow queries: 1 Opens: 538 Flush tables: 1 Open tables: 529 Queries per second avg: 13.108 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1409 [18:15:24] 2013/11/11 18:08 WARN z-dat-s3-a MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2117 [18:26:24] 2013/11/11 18:25 OK z-dat-s3-a MySQL slave Uptime: 2555 Threads: 20 Questions: 672945 Slow queries: 41 Opens: 12277 Flush tables: 1 Open tables: 12267 Queries per second avg: 263.383 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1714 [19:27:28] 2013/11/11 19:26 CRIT rosemary /sql/data/commonswiki DISK CRITICAL - free space: /sql/data/commonswiki 67507 MB (10% inode=99%): [21:54:35] 2013/11/11 21:54 WARN rosemary s1 replag QUERY WARNING: 'SELECT ts_rc_age()' returned 3586.000000 [21:55:35] 2013/11/11 21:55 WARN rosemary MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3583 [22:01:52] It would be nice if I could run 'byebye' without agreeing that I've moved everything. [22:02:04] I thought I could use the backup to easily move everything [22:02:07] Guess not! [22:10:45] I'll just lie. [22:16:56] legoktm: well, as long as you don't burnbridges, I suppose you'll be fine [22:19:14] Well, I was going to run it, except I can't find the source of what it will actually do so I'll hold off [22:21:41] legoktm: I think it's a wrapper for /usr/local/bin/burnbridges.php [22:22:51] which is actually broken, I think [22:23:12] exec("find /home/amette/ -maxdepth 1 -mindepth 1 -type d -exec rm -fr {} \;", $output); [22:23:18] doesn't look completely right ;-) [22:30:22] legoktm@nightshade:~$ burnbridges [22:30:23] PHP Parse error: syntax error, unexpected T_CONSTANT_ENCAPSED_STRING, expecting ',' or ';' in /usr/local/bin/burnbridges.php on line 137 [22:30:34] amette_: ^ [22:50:38] 2013/11/11 22:50 OK rosemary MySQL slave Uptime: 996363 Threads: 5 Questions: 435968871 Slow queries: 544791 Opens: 5974 Flush tables: 1 Open tables: 1326 Queries per second avg: 437.560 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1771 [22:50:39] 2013/11/11 22:50 OK rosemary s1 replag QUERY OK: 'SELECT ts_rc_age()' returned 1737.000000