[03:44:49] 2014/01/27 03:44 WARN cassia /sql DISK WARNING - free space: /sql 71311 MB (6% inode=99%): [03:45:49] 2014/01/27 03:45 CRIT cassia /sql DISK CRITICAL - free space: /sql 69896 MB (5% inode=99%): [04:02:50] 2014/01/27 04:02 WARN cassia /sql DISK WARNING - free space: /sql 71193 MB (6% inode=99%): [04:03:51] 2014/01/27 04:03 CRIT cassia /sql DISK CRITICAL - free space: /sql 69937 MB (5% inode=99%): [04:32:53] 2014/01/27 04:24 CRIT rosemary s1 replag (Service Check Timed Out) [04:32:53] 2014/01/27 04:30 CRIT web.amaranth fisheye.toolserver.org CRITICAL - Socket timeout after 21 seconds [04:34:53] 2014/01/27 04:33 WARN web.amaranth fisheye.toolserver.org HTTP WARNING: HTTP/1.1 200 OK - 272 bytes in 18.441 second response time [04:36:54] 2014/01/27 04:35 CRIT web.amaranth fisheye.toolserver.org CRITICAL - Socket timeout after 21 seconds [04:37:54] 2014/01/27 04:37 OK rosemary s1 replag QUERY OK: 'SELECT ts_rc_age()' returned 702.000000 [04:41:55] 2014/01/27 04:40 WARN web.amaranth fisheye.toolserver.org HTTP WARNING: HTTP/1.1 200 OK - 272 bytes in 15.676 second response time [04:42:55] 2014/01/27 04:41 OK web.amaranth fisheye.toolserver.org HTTP OK: HTTP/1.1 200 OK - 272 bytes in 10.384 second response time [04:49:55] 2014/01/27 04:49 WARN cassia /sql DISK WARNING - free space: /sql 72266 MB (6% inode=99%): [04:50:55] 2014/01/27 04:50 CRIT cassia /sql DISK CRITICAL - free space: /sql 70645 MB (5% inode=99%): [05:23:58] 2014/01/27 05:23 WARN cassia /sql DISK WARNING - free space: /sql 71296 MB (6% inode=99%): [05:24:58] 2014/01/27 05:24 CRIT cassia /sql DISK CRITICAL - free space: /sql 70518 MB (5% inode=99%): [05:57:59] 2014/01/27 05:57 WARN cassia /sql DISK WARNING - free space: /sql 71696 MB (6% inode=99%): [05:58:59] 2014/01/27 05:58 CRIT cassia /sql DISK CRITICAL - free space: /sql 70422 MB (5% inode=99%): [06:09:00] 2014/01/27 06:08 WARN cassia /sql DISK WARNING - free space: /sql 71258 MB (6% inode=99%): [06:10:00] 2014/01/27 06:09 CRIT cassia /sql DISK CRITICAL - free space: /sql 70327 MB (5% inode=99%): [06:53:02] 2014/01/27 06:45 CRIT yucca SSH CRITICAL - Socket timeout after 10 seconds [06:54:02] 2014/01/27 06:52 OK yucca SSH SSH OK - OpenSSH_5.5p1 Debian-6+squeeze3 (protocol 2.0) [09:03:11] 2014/01/27 09:02 WARN cassia /sql DISK WARNING - free space: /sql 71411 MB (6% inode=99%): [09:04:11] 2014/01/27 09:03 CRIT cassia /sql DISK CRITICAL - free space: /sql 70168 MB (5% inode=99%): [10:13:16] 2014/01/27 10:12 WARN cassia /sql DISK WARNING - free space: /sql 71494 MB (6% inode=99%): [10:14:16] 2014/01/27 10:13 CRIT cassia /sql DISK CRITICAL - free space: /sql 69630 MB (5% inode=99%): [10:16:05] Marlen Caemmerer * [Toolserver-l] Resetup of dewiki [10:25:17] 2014/01/27 10:19 CRIT rosemary MySQL slave Slave IO: No Slave SQL: No Seconds Behind Master: (null) [10:27:17] 2014/01/27 10:21 CRIT daphne MySQL slave Slave IO: No Slave SQL: No Seconds Behind Master: (null) [10:56:17] 2014/01/27 10:49 WARN rosemary s1 replag QUERY WARNING: 'SELECT ts_rc_age()' returned 2219.000000 [10:57:17] 2014/01/27 10:50 WARN daphne s4 replag QUERY WARNING: 'SELECT ts_rc_age()' returned 2167.000000 [11:20:17] 2014/01/27 11:19 CRIT rosemary s1 replag QUERY CRITICAL: 'SELECT ts_rc_age()' returned 3655.000000 [11:21:17] 2014/01/27 11:20 CRIT daphne s4 replag QUERY CRITICAL: 'SELECT ts_rc_age()' returned 3606.000000 [12:30:23] 2014/01/27 12:29 ?? z-dat-s7-a /sql NRPE: Call to fork() failed [12:31:23] 2014/01/27 12:30 ?? z-dat-s4-a /sql CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:33:23] 2014/01/27 12:30 CRIT z-dat-s7-a MySQL slave Can't connect to MySQL server on 'z-dat-s7-a' (110) [12:33:23] 2014/01/27 12:30 CRIT z-dat-s7-a SSH CRITICAL - Socket timeout after 10 seconds [12:34:23] 2014/01/27 12:30 CRIT z-dat-s7-a MySQL Can't connect to MySQL server on 'z-dat-s7-a' (110) [12:35:23] 2014/01/27 12:31 CRIT z-dat-s6-a MySQL Can't connect to MySQL server on 'z-dat-s6-a' (110) [12:36:23] 2014/01/27 12:29 ?? z-dat-s3-a /sql CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:36:23] 2014/01/27 12:29 ?? z-dat-s3-a /tmp CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:36:23] 2014/01/27 12:29 ?? z-dat-s6-a / CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:36:23] 2014/01/27 12:34 OK z-dat-s6-a MySQL Uptime: 6309787 Threads: 6 Questions: 1616792546 Slow queries: 544496 Opens: 2098139 Flush tables: 1 Open tables: 3964 Queries per second avg: 256.235 [12:36:23] 2014/01/27 12:29 ?? z-dat-s7-a Load avg. CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages. [12:36:24] 2014/01/27 12:35 OK z-dat-s7-a SSH SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [12:37:23] 2014/01/27 12:36 OK z-dat-s3-a /sql DISK OK - free space: /sql 131603 MB (13% inode=98%): [12:37:23] 2014/01/27 12:36 OK z-dat-s3-a /tmp DISK OK - free space: /tmp 33410 MB (99% inode=99%): [12:37:23] 2014/01/27 12:30 CRIT z-dat-s3-a MySQL Can't connect to MySQL server on 'z-dat-s3-a' (111) [12:37:23] 2014/01/27 12:30 CRIT z-dat-s3-a MySQL slave Can't connect to MySQL server on 'z-dat-s3-a' (111) [12:37:23] 2014/01/27 12:36 CRIT z-dat-s4-a /sql DISK CRITICAL - free space: /sql 0 MB (0% inode=25%): [12:37:24] 2014/01/27 12:36 OK z-dat-s6-a / DISK OK - free space: / 8632 MB (28% inode=86%): [12:37:24] 2014/01/27 12:36 WARN z-dat-s7-a /sql DISK WARNING - free space: /sql 65778 MB (9% inode=99%): [12:37:24] 2014/01/27 12:36 OK z-dat-s7-a Load avg. OK - load average: 1.22, 0.68, 0.98 [12:37:25] 2014/01/27 12:36 OK z-dat-s7-a MySQL Uptime: 5875143 Threads: 14 Questions: 1031379352 Slow queries: 377560 Opens: 4044296 Flush tables: 1 Open tables: 5878 Queries per second avg: 175.549 [12:37:26] 2014/01/27 12:36 OK z-dat-s7-a MySQL slave Uptime: 5875143 Threads: 15 Questions: 1031379353 Slow queries: 377560 Opens: 4044296 Flush tables: 1 Open tables: 5878 Queries per second avg: 175.549 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1233 [12:46:24] 2014/01/27 12:45 WARN cassia /sql DISK WARNING - free space: /sql 71095 MB (6% inode=99%): [12:47:24] 2014/01/27 12:46 CRIT cassia /sql DISK CRITICAL - free space: /sql 70483 MB (5% inode=99%): [12:57:25] 2014/01/27 12:56 OK z-dat-s3-a MySQL Uptime: 1255 Threads: 6 Questions: 210 Slow queries: 0 Opens: 96 Flush tables: 1 Open tables: 88 Queries per second avg: 0.167 [12:57:25] 2014/01/27 12:56 WARN z-dat-s3-a MySQL slave SLOW_SLAVE WARNING: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2772 [13:11:22] [[Special:Log/newusers]] create 10 * Delmar917 * (New user account) [13:33:27] 2014/01/27 13:32 OK z-dat-s3-a MySQL slave Uptime: 3415 Threads: 13 Questions: 452936 Slow queries: 121 Opens: 12594 Flush tables: 1 Open tables: 12583 Queries per second avg: 132.631 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1787 [13:36:27] 2014/01/27 13:35 WARN cassia /sql DISK WARNING - free space: /sql 71477 MB (6% inode=99%): [13:37:27] 2014/01/27 13:36 CRIT cassia /sql DISK CRITICAL - free space: /sql 70343 MB (5% inode=99%): [14:21:05] Robin Krahl * Re: [Toolserver-l] Reminder: Jan 23rd >> Office hour: Migration from Toolserver to Tool Labs [14:30:53] * closedmouth checks the calendar [16:01:35] 2014/01/27 16:00 WARN cassia /sql DISK WARNING - free space: /sql 72407 MB (6% inode=99%): [16:02:36] 2014/01/27 16:01 CRIT cassia /sql DISK CRITICAL - free space: /sql 69879 MB (5% inode=99%): [16:23:38] 2014/01/27 16:22 WARN cassia /sql DISK WARNING - free space: /sql 71829 MB (6% inode=99%): [16:24:38] 2014/01/27 16:23 CRIT cassia /sql DISK CRITICAL - free space: /sql 70821 MB (5% inode=99%): [16:50:40] 2014/01/27 16:49 WARN cassia /sql DISK WARNING - free space: /sql 71182 MB (6% inode=99%): [16:52:40] 2014/01/27 16:51 CRIT cassia /sql DISK CRITICAL - free space: /sql 70732 MB (5% inode=99%): [17:39:42] 2014/01/27 17:32 WARN wolfsbane / DISK WARNING - free space: / 5989 MB (19% inode=93%): [17:39:42] 2014/01/27 17:32 WARN wolfsbane /tmp DISK WARNING - free space: / 5990 MB (19% inode=93%): [17:59:43] 2014/01/27 17:58 WARN cassia /sql DISK WARNING - free space: /sql 71027 MB (6% inode=99%): [18:00:43] 2014/01/27 17:59 CRIT cassia /sql DISK CRITICAL - free space: /sql 70499 MB (5% inode=99%): [18:08:43] 2014/01/27 18:07 OK wolfsbane / DISK OK - free space: / 6586 MB (21% inode=93%): [18:08:43] 2014/01/27 18:07 OK wolfsbane /tmp DISK OK - free space: / 6586 MB (21% inode=93%): [18:36:46] 2014/01/27 18:35 WARN cassia /sql DISK WARNING - free space: /sql 71212 MB (6% inode=99%): [18:37:46] 2014/01/27 18:36 CRIT cassia /sql DISK CRITICAL - free space: /sql 70368 MB (5% inode=99%): [23:11:04] 2014/01/27 23:10 WARN cassia /sql DISK WARNING - free space: /sql 71219 MB (6% inode=99%): [23:12:04] 2014/01/27 23:11 CRIT cassia /sql DISK CRITICAL - free space: /sql 70391 MB (5% inode=99%):