[00:10:48] PROBLEM - RAID on mw1128 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:10:58] PROBLEM - RAID on mw1125 is CRITICAL: CHECK_NRPE: Error - Could not complete SSL handshake. [00:10:58] PROBLEM - twemproxy process on mw1118 is CRITICAL: CHECK_NRPE: Error - Could not complete SSL handshake. [00:11:08] PROBLEM - DPKG on mw1143 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:11:08] PROBLEM - RAID on mw1143 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:11:08] PROBLEM - Apache HTTP on mw1118 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:11:08] PROBLEM - DPKG on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:11:18] PROBLEM - RAID on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:11:18] PROBLEM - RAID on mw1133 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:11:38] PROBLEM - DPKG on mw1116 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:11:48] PROBLEM - RAID on mw1139 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:11:48] PROBLEM - Apache HTTP on mw1139 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:11:48] PROBLEM - RAID on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:11:48] PROBLEM - DPKG on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:11:48] PROBLEM - RAID on mw1116 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:11:48] PROBLEM - Apache HTTP on mw1116 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:11:58] RECOVERY - RAID on mw1125 is OK: OK: no RAID installed [00:11:58] PROBLEM - Apache HTTP on mw1124 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:11:59] RECOVERY - Apache HTTP on mw1118 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.074 second response time [00:11:59] RECOVERY - twemproxy process on mw1118 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [00:12:08] RECOVERY - DPKG on mw1142 is OK: All packages OK [00:12:08] RECOVERY - RAID on mw1133 is OK: OK: no RAID installed [00:12:08] PROBLEM - Apache HTTP on mw1143 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:12:18] goddamnit again [00:12:28] RECOVERY - DPKG on mw1116 is OK: All packages OK [00:12:38] RECOVERY - RAID on mw1139 is OK: OK: no RAID installed [00:12:39] RECOVERY - Apache HTTP on mw1139 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.062 second response time [00:12:39] RECOVERY - RAID on mw1116 is OK: OK: no RAID installed [00:12:39] RECOVERY - Apache HTTP on mw1116 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.043 second response time [00:12:48] PROBLEM - SSH on mw1143 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:12:48] RECOVERY - RAID on mw1128 is OK: OK: no RAID installed [00:12:48] PROBLEM - Apache HTTP on mw1142 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:13:38] RECOVERY - RAID on mw1142 is OK: OK: no RAID installed [00:13:38] RECOVERY - Apache HTTP on mw1142 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.074 second response time [00:13:48] RECOVERY - SSH on mw1143 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [00:13:58] PROBLEM - MySQL InnoDB on db1056 is CRITICAL: CRIT longest blocking idle transaction sleeps for 766 seconds [00:13:58] PROBLEM - MySQL Idle Transactions on db1056 is CRITICAL: CRIT longest blocking idle transaction sleeps for 766 seconds [00:13:58] RECOVERY - Apache HTTP on mw1143 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.103 second response time [00:14:58] RECOVERY - MySQL InnoDB on db1056 is OK: OK longest blocking idle transaction sleeps for 0 seconds [00:14:58] RECOVERY - MySQL Idle Transactions on db1056 is OK: OK longest blocking idle transaction sleeps for 0 seconds [00:14:58] RECOVERY - DPKG on mw1143 is OK: All packages OK [00:16:18] PROBLEM - SSH on mw1124 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:17:08] PROBLEM - twemproxy process on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:17:08] RECOVERY - SSH on mw1124 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [00:17:58] PROBLEM - MySQL InnoDB on db1056 is CRITICAL: CRIT longest blocking idle transaction sleeps for 827 seconds [00:17:58] PROBLEM - MySQL Idle Transactions on db1056 is CRITICAL: CRIT longest blocking idle transaction sleeps for 828 seconds [00:18:08] PROBLEM - Apache HTTP on mw1143 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:18:58] RECOVERY - MySQL InnoDB on db1056 is OK: OK longest blocking idle transaction sleeps for 0 seconds [00:18:58] RECOVERY - MySQL Idle Transactions on db1056 is OK: OK longest blocking idle transaction sleeps for 0 seconds [00:18:58] RECOVERY - Apache HTTP on mw1143 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.074 second response time [00:19:48] PROBLEM - twemproxy process on mw1143 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:20:18] PROBLEM - SSH on mw1124 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:20:38] RECOVERY - twemproxy process on mw1143 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [00:20:58] RECOVERY - RAID on mw1143 is OK: OK: no RAID installed [00:21:18] RECOVERY - SSH on mw1124 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [00:21:48] PROBLEM - Disk space on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:21:58] PROBLEM - MySQL InnoDB on db1056 is CRITICAL: CRIT longest blocking idle transaction sleeps for 1068 seconds [00:21:58] PROBLEM - MySQL Idle Transactions on db1056 is CRITICAL: CRIT longest blocking idle transaction sleeps for 1068 seconds [00:22:08] PROBLEM - RAID on mw1120 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:22:48] PROBLEM - RAID on mw1128 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:22:48] PROBLEM - DPKG on mw1118 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:22:49] PROBLEM - DPKG on mw1128 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:23:08] PROBLEM - RAID on mw1125 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:23:08] PROBLEM - DPKG on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:23:28] PROBLEM - DPKG on mw1146 is CRITICAL: CHECK_NRPE: Error - Could not complete SSL handshake. [00:23:38] PROBLEM - twemproxy process on mw1117 is CRITICAL: CHECK_NRPE: Error - Could not complete SSL handshake. [00:23:38] PROBLEM - RAID on mw1117 is CRITICAL: CHECK_NRPE: Error - Could not complete SSL handshake. [00:23:48] RECOVERY - DPKG on mw1118 is OK: All packages OK [00:23:48] RECOVERY - Disk space on mw1124 is OK: DISK OK [00:23:48] PROBLEM - RAID on mw1116 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:23:48] PROBLEM - twemproxy process on mw1143 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:23:48] PROBLEM - RAID on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:23:49] PROBLEM - RAID on mw1146 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:23:49] PROBLEM - LVS HTTP IPv4 on api.svc.eqiad.wmnet is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:23:51] PROBLEM - Apache HTTP on mw1116 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:23:58] PROBLEM - Apache HTTP on mw1193 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:23:58] PROBLEM - Apache HTTP on mw1192 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:23:58] PROBLEM - Apache HTTP on mw1190 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:23:58] RECOVERY - RAID on mw1125 is OK: OK: no RAID installed [00:23:58] RECOVERY - MySQL InnoDB on db1056 is OK: OK longest blocking idle transaction sleeps for 0 seconds [00:23:59] RECOVERY - MySQL Idle Transactions on db1056 is OK: OK longest blocking idle transaction sleeps for 0 seconds [00:23:59] RECOVERY - twemproxy process on mw1124 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [00:24:08] PROBLEM - DPKG on mw1143 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:24:08] PROBLEM - Apache HTTP on mw1146 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:24:08] PROBLEM - RAID on mw1143 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:24:08] PROBLEM - Apache HTTP on mw1198 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:24:08] PROBLEM - Apache HTTP on mw1143 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:24:09] PROBLEM - Apache HTTP on mw1118 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:24:09] PROBLEM - twemproxy process on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:24:38] PROBLEM - Disk space on mw1146 is CRITICAL: CHECK_NRPE: Error - Could not complete SSL handshake. [00:24:39] RECOVERY - RAID on mw1116 is OK: OK: no RAID installed [00:24:39] RECOVERY - LVS HTTP IPv4 on api.svc.eqiad.wmnet is OK: HTTP OK: HTTP/1.1 200 OK - 2919 bytes in 0.130 second response time [00:24:48] RECOVERY - Apache HTTP on mw1116 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 3.878 second response time [00:24:48] RECOVERY - Apache HTTP on mw1190 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.044 second response time [00:24:48] RECOVERY - Apache HTTP on mw1193 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.046 second response time [00:24:48] RECOVERY - Apache HTTP on mw1192 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.053 second response time [00:24:48] PROBLEM - Apache HTTP on mw1142 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:24:48] PROBLEM - Apache HTTP on mw1120 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:24:49] PROBLEM - Apache HTTP on mw1132 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:24:49] RECOVERY - DPKG on mw1128 is OK: All packages OK [00:24:58] PROBLEM - twemproxy process on mw1122 is CRITICAL: NRPE: Unable to read output [00:24:58] PROBLEM - Apache HTTP on mw1122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:24:58] RECOVERY - Apache HTTP on mw1198 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.050 second response time [00:24:59] RECOVERY - Apache HTTP on mw1118 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.102 second response time [00:24:59] RECOVERY - RAID on mw1120 is OK: OK: no RAID installed [00:25:08] RECOVERY - DPKG on mw1142 is OK: All packages OK [00:25:08] RECOVERY - Apache HTTP on mw1143 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 7.898 second response time [00:25:08] RECOVERY - twemproxy process on mw1142 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [00:25:18] PROBLEM - RAID on mw1133 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:25:28] RECOVERY - DPKG on mw1146 is OK: All packages OK [00:25:38] RECOVERY - Disk space on mw1146 is OK: DISK OK [00:25:39] RECOVERY - Apache HTTP on mw1120 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.055 second response time [00:25:39] RECOVERY - RAID on mw1146 is OK: OK: no RAID installed [00:25:48] RECOVERY - Apache HTTP on mw1132 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 6.435 second response time [00:25:48] RECOVERY - twemproxy process on mw1143 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [00:25:48] RECOVERY - Apache HTTP on mw1122 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.206 second response time [00:25:48] RECOVERY - RAID on mw1128 is OK: OK: no RAID installed [00:25:48] PROBLEM - DPKG on mw1133 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:25:58] RECOVERY - Apache HTTP on mw1146 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.078 second response time [00:26:28] PROBLEM - SSH on mw1133 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:26:48] RECOVERY - twemproxy process on mw1122 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [00:26:48] PROBLEM - Disk space on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:26:58] PROBLEM - MySQL InnoDB on db1056 is CRITICAL: CRIT longest blocking idle transaction sleeps for 1449 seconds [00:26:58] PROBLEM - MySQL Idle Transactions on db1056 is CRITICAL: CRIT longest blocking idle transaction sleeps for 1449 seconds [00:27:08] PROBLEM - twemproxy process on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:27:18] RECOVERY - SSH on mw1133 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [00:27:38] RECOVERY - twemproxy process on mw1117 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [00:27:48] PROBLEM - Apache HTTP on mw1134 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:27:48] PROBLEM - SSH on mw1139 is CRITICAL: Server answer: [00:27:58] PROBLEM - twemproxy process on mw1139 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:27:58] RECOVERY - MySQL InnoDB on db1056 is OK: OK longest blocking idle transaction sleeps for 0 seconds [00:27:58] RECOVERY - MySQL Idle Transactions on db1056 is OK: OK longest blocking idle transaction sleeps for 0 seconds [00:27:58] PROBLEM - DPKG on mw1139 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:27:59] PROBLEM - DPKG on mw1123 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:27:59] RECOVERY - DPKG on mw1143 is OK: All packages OK [00:28:08] PROBLEM - DPKG on mw1141 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:28:08] RECOVERY - RAID on mw1143 is OK: OK: no RAID installed [00:28:08] PROBLEM - twemproxy process on mw1133 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:28:08] PROBLEM - DPKG on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:28:08] PROBLEM - twemproxy process on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:28:08] PROBLEM - Apache HTTP on mw1195 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:28:18] PROBLEM - SSH on mw1142 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:28:18] PROBLEM - Apache HTTP on mw1203 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:28:39] RECOVERY - Apache HTTP on mw1134 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 2.018 second response time [00:28:48] PROBLEM - Apache HTTP on mw1139 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:28:48] PROBLEM - Apache HTTP on mw1204 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:28:48] RECOVERY - twemproxy process on mw1139 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [00:28:48] RECOVERY - SSH on mw1139 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [00:28:48] PROBLEM - Apache HTTP on mw1132 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:28:49] PROBLEM - Apache HTTP on mw1191 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:28:49] PROBLEM - RAID on mw1123 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:28:50] PROBLEM - Apache HTTP on mw1120 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:28:50] PROBLEM - SSH on mw1146 is CRITICAL: Server answer: [00:28:50] PROBLEM - RAID on mw1146 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:28:51] PROBLEM - RAID on mw1141 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:28:52] PROBLEM - LVS HTTP IPv4 on api.svc.eqiad.wmnet is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:28:52] RECOVERY - DPKG on mw1139 is OK: All packages OK [00:28:53] PROBLEM - SSH on mw1117 is CRITICAL: Server answer: [00:28:58] PROBLEM - Disk space on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:28:58] RECOVERY - DPKG on mw1123 is OK: All packages OK [00:28:59] RECOVERY - Apache HTTP on mw1195 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.047 second response time [00:29:08] RECOVERY - Apache HTTP on mw1203 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.056 second response time [00:29:08] PROBLEM - Apache HTTP on mw1146 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:29:08] PROBLEM - Disk space on mw1133 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:29:18] PROBLEM - Apache HTTP on mw1117 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:29:38] RECOVERY - Apache HTTP on mw1139 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.060 second response time [00:29:38] RECOVERY - Apache HTTP on mw1204 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.054 second response time [00:29:39] RECOVERY - RAID on mw1117 is OK: OK: no RAID installed [00:29:39] RECOVERY - Apache HTTP on mw1132 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.077 second response time [00:29:39] RECOVERY - Apache HTTP on mw1120 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.079 second response time [00:29:39] RECOVERY - Apache HTTP on mw1191 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.096 second response time [00:29:39] RECOVERY - RAID on mw1146 is OK: OK: no RAID installed [00:29:40] RECOVERY - LVS HTTP IPv4 on api.svc.eqiad.wmnet is OK: HTTP OK: HTTP/1.1 200 OK - 2919 bytes in 0.078 second response time [00:29:48] RECOVERY - RAID on mw1123 is OK: OK: no RAID installed [00:29:48] RECOVERY - SSH on mw1117 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [00:29:48] RECOVERY - SSH on mw1146 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [00:29:48] RECOVERY - RAID on mw1141 is OK: OK: no RAID installed [00:29:58] RECOVERY - Apache HTTP on mw1146 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.071 second response time [00:29:58] RECOVERY - twemproxy process on mw1124 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [00:30:08] RECOVERY - Apache HTTP on mw1117 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.066 second response time [00:30:08] RECOVERY - DPKG on mw1141 is OK: All packages OK [00:30:58] RECOVERY - Disk space on mw1142 is OK: DISK OK [00:31:08] RECOVERY - SSH on mw1142 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [00:31:48] RECOVERY - Disk space on mw1124 is OK: DISK OK [00:31:58] PROBLEM - Apache HTTP on mw1133 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:31:59] RECOVERY - twemproxy process on mw1133 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [00:32:08] RECOVERY - Disk space on mw1133 is OK: DISK OK [00:32:58] PROBLEM - MySQL Idle Transactions on db1056 is CRITICAL: CRIT longest blocking idle transaction sleeps for 1632 seconds [00:32:59] PROBLEM - MySQL InnoDB on db1056 is CRITICAL: CRIT longest blocking idle transaction sleeps for 1632 seconds [00:33:08] PROBLEM - twemproxy process on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:33:18] PROBLEM - SSH on mw1124 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:33:28] PROBLEM - SSH on mw1133 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:33:58] PROBLEM - Disk space on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:34:18] PROBLEM - SSH on mw1142 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:34:18] RECOVERY - SSH on mw1133 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [00:34:48] PROBLEM - Disk space on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:34:58] RECOVERY - Disk space on mw1142 is OK: DISK OK [00:35:08] RECOVERY - twemproxy process on mw1142 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [00:36:08] RECOVERY - SSH on mw1124 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [00:36:08] PROBLEM - Disk space on mw1133 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:36:58] RECOVERY - MySQL InnoDB on db1056 is OK: OK longest blocking idle transaction sleeps for 0 seconds [00:36:58] RECOVERY - MySQL Idle Transactions on db1056 is OK: OK longest blocking idle transaction sleeps for 0 seconds [00:37:08] RECOVERY - Disk space on mw1133 is OK: DISK OK [00:37:48] RECOVERY - Apache HTTP on mw1133 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.059 second response time [00:38:08] RECOVERY - twemproxy process on mw1124 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [00:38:08] PROBLEM - twemproxy process on mw1133 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:38:28] PROBLEM - SSH on mw1133 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:39:08] RECOVERY - twemproxy process on mw1133 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [00:39:28] RECOVERY - SSH on mw1133 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [00:39:34] what's going on with the apaches? [00:39:39] RECOVERY - DPKG on mw1133 is OK: All packages OK [00:39:58] PROBLEM - Disk space on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:40:08] RECOVERY - SSH on mw1142 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [00:40:58] RECOVERY - Disk space on mw1142 is OK: DISK OK [00:41:08] PROBLEM - twemproxy process on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:41:18] RECOVERY - RAID on mw1133 is OK: OK: no RAID installed [00:41:48] RECOVERY - Disk space on mw1124 is OK: DISK OK [00:41:58] PROBLEM - MySQL Idle Transactions on db1056 is CRITICAL: CRIT longest blocking idle transaction sleeps for 2220 seconds [00:41:58] PROBLEM - MySQL InnoDB on db1056 is CRITICAL: CRIT longest blocking idle transaction sleeps for 2220 seconds [00:43:58] PROBLEM - Disk space on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:44:08] PROBLEM - twemproxy process on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:44:18] PROBLEM - SSH on mw1124 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:46:48] PROBLEM - Disk space on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:47:18] PROBLEM - SSH on mw1142 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:47:58] RECOVERY - MySQL Idle Transactions on db1056 is OK: OK longest blocking idle transaction sleeps for 0 seconds [00:47:58] RECOVERY - MySQL InnoDB on db1056 is OK: OK longest blocking idle transaction sleeps for 0 seconds [00:48:18] RECOVERY - SSH on mw1142 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [00:49:48] RECOVERY - Disk space on mw1124 is OK: DISK OK [00:49:48] RECOVERY - Disk space on mw1142 is OK: DISK OK [00:51:58] PROBLEM - MySQL InnoDB on db1056 is CRITICAL: CRIT longest blocking idle transaction sleeps for 3046 seconds [00:51:58] PROBLEM - MySQL Idle Transactions on db1056 is CRITICAL: CRIT longest blocking idle transaction sleeps for 3046 seconds [00:52:08] RECOVERY - SSH on mw1124 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [00:52:18] PROBLEM - SSH on mw1142 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:52:48] PROBLEM - Disk space on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:52:58] PROBLEM - Disk space on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:54:08] RECOVERY - twemproxy process on mw1124 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [00:54:08] RECOVERY - SSH on mw1142 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [00:54:48] RECOVERY - Disk space on mw1142 is OK: DISK OK [00:55:18] PROBLEM - SSH on mw1124 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:55:58] RECOVERY - MySQL Idle Transactions on db1056 is OK: OK longest blocking idle transaction sleeps for 0 seconds [00:55:58] RECOVERY - MySQL InnoDB on db1056 is OK: OK longest blocking idle transaction sleeps for 0 seconds [00:56:38] RECOVERY - DPKG on mw1124 is OK: All packages OK [00:56:48] RECOVERY - Disk space on mw1124 is OK: DISK OK [00:57:08] PROBLEM - twemproxy process on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [00:58:08] RECOVERY - SSH on mw1124 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [00:58:58] PROBLEM - MySQL Idle Transactions on db1056 is CRITICAL: CRIT longest blocking idle transaction sleeps for 3466 seconds [00:58:58] PROBLEM - MySQL InnoDB on db1056 is CRITICAL: CRIT longest blocking idle transaction sleeps for 3466 seconds [00:59:48] PROBLEM - DPKG on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:00:58] RECOVERY - MySQL Idle Transactions on db1056 is OK: OK longest blocking idle transaction sleeps for 0 seconds [01:00:58] RECOVERY - MySQL InnoDB on db1056 is OK: OK longest blocking idle transaction sleeps for 0 seconds [01:01:08] RECOVERY - DPKG on mw1142 is OK: All packages OK [01:01:08] RECOVERY - twemproxy process on mw1142 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [01:01:58] PROBLEM - Disk space on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:02:18] PROBLEM - SSH on mw1124 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:03:08] RECOVERY - twemproxy process on mw1124 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [01:03:48] RECOVERY - Disk space on mw1142 is OK: DISK OK [01:04:08] RECOVERY - SSH on mw1124 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [01:04:08] PROBLEM - DPKG on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:06:11] RECOVERY - RAID on mw1124 is OK: OK: no RAID installed [01:06:11] PROBLEM - twemproxy process on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:06:11] PROBLEM - SSH on mw1142 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:08:01] PROBLEM - Disk space on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:08:51] RECOVERY - Disk space on mw1142 is OK: DISK OK [01:08:52] PROBLEM - Disk space on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:09:11] PROBLEM - RAID on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:09:42] RECOVERY - Disk space on mw1124 is OK: DISK OK [01:10:11] PROBLEM - twemproxy process on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:10:11] PROBLEM - SSH on mw1124 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:10:42] RECOVERY - RAID on mw1142 is OK: OK: no RAID installed [01:12:51] RECOVERY - DPKG on mw1124 is OK: All packages OK [01:12:51] RECOVERY - Apache HTTP on mw1124 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.229 second response time [01:13:11] RECOVERY - twemproxy process on mw1124 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [01:13:11] PROBLEM - RAID on mw1143 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:13:51] PROBLEM - RAID on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:14:01] RECOVERY - DPKG on mw1142 is OK: All packages OK [01:14:02] RECOVERY - twemproxy process on mw1142 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [01:14:02] RECOVERY - SSH on mw1142 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [01:14:11] PROBLEM - RAID on mw1132 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:14:21] PROBLEM - DPKG on mw1116 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:14:41] RECOVERY - RAID on mw1142 is OK: OK: no RAID installed [01:15:02] RECOVERY - RAID on mw1143 is OK: OK: no RAID installed [01:15:11] PROBLEM - Apache HTTP on mw1125 is CRITICAL: Connection timed out [01:15:21] PROBLEM - Disk space on mw1125 is CRITICAL: CHECK_NRPE: Error - Could not complete SSL handshake. [01:15:21] PROBLEM - RAID on mw1133 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:15:31] PROBLEM - RAID on mw1117 is CRITICAL: CHECK_NRPE: Error - Could not complete SSL handshake. [01:15:41] PROBLEM - Apache HTTP on mw1199 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:15:42] PROBLEM - Apache HTTP on mw1132 is CRITICAL: Connection timed out [01:15:42] PROBLEM - Apache HTTP on mw1204 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:15:42] PROBLEM - Disk space on mw1130 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:15:51] PROBLEM - Apache HTTP on mw1197 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:15:51] PROBLEM - Apache HTTP on mw1207 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:15:52] PROBLEM - RAID on mw1128 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:15:52] PROBLEM - RAID on mw1141 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:15:52] PROBLEM - LVS HTTP IPv4 on api.svc.eqiad.wmnet is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:16:01] PROBLEM - DPKG on mw1130 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:16:02] RECOVERY - RAID on mw1132 is OK: OK: no RAID installed [01:16:02] RECOVERY - Apache HTTP on mw1125 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.045 second response time [01:16:11] RECOVERY - RAID on mw1133 is OK: OK: no RAID installed [01:16:11] PROBLEM - DPKG on mw1141 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:16:11] PROBLEM - Apache HTTP on mw1130 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:16:11] PROBLEM - RAID on mw1130 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:16:11] PROBLEM - SSH on mw1130 is CRITICAL: Server answer: [01:16:12] PROBLEM - Apache HTTP on mw1141 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:16:12] PROBLEM - Apache HTTP on mw1117 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:16:21] RECOVERY - Disk space on mw1125 is OK: DISK OK [01:16:31] RECOVERY - Apache HTTP on mw1199 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.361 second response time [01:16:32] RECOVERY - Apache HTTP on mw1204 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.122 second response time [01:16:41] RECOVERY - RAID on mw1117 is OK: OK: no RAID installed [01:16:41] RECOVERY - Apache HTTP on mw1132 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.094 second response time [01:16:42] RECOVERY - Apache HTTP on mw1197 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.091 second response time [01:16:42] RECOVERY - Apache HTTP on mw1207 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.112 second response time [01:16:42] RECOVERY - LVS HTTP IPv4 on api.svc.eqiad.wmnet is OK: HTTP OK: HTTP/1.1 200 OK - 2919 bytes in 1.481 second response time [01:16:44] RECOVERY - RAID on mw1128 is OK: OK: no RAID installed [01:16:51] PROBLEM - DPKG on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:17:01] RECOVERY - DPKG on mw1130 is OK: All packages OK [01:17:01] PROBLEM - Apache HTTP on mw1124 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:17:01] RECOVERY - SSH on mw1130 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [01:17:01] RECOVERY - Apache HTTP on mw1130 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 2.972 second response time [01:17:01] RECOVERY - RAID on mw1130 is OK: OK: no RAID installed [01:17:11] RECOVERY - Apache HTTP on mw1117 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 3.038 second response time [01:17:11] RECOVERY - DPKG on mw1116 is OK: All packages OK [01:17:11] PROBLEM - twemproxy process on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:17:11] PROBLEM - DPKG on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:17:11] PROBLEM - twemproxy process on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:17:11] PROBLEM - SSH on mw1142 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:17:31] RECOVERY - Disk space on mw1130 is OK: DISK OK [01:17:51] PROBLEM - RAID on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:18:01] PROBLEM - Disk space on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:18:11] RECOVERY - SSH on mw1124 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [01:18:11] RECOVERY - twemproxy process on mw1142 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [01:18:11] PROBLEM - Apache HTTP on mw1143 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:19:11] PROBLEM - RAID on mw1143 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:19:11] PROBLEM - DPKG on mw1143 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:19:21] PROBLEM - SSH on mw1141 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:19:51] RECOVERY - Disk space on mw1142 is OK: DISK OK [01:20:51] PROBLEM - twemproxy process on mw1143 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:21:11] PROBLEM - SSH on mw1124 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:21:41] RECOVERY - twemproxy process on mw1143 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [01:21:51] RECOVERY - DPKG on mw1124 is OK: All packages OK [01:22:11] PROBLEM - twemproxy process on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:22:41] RECOVERY - RAID on mw1142 is OK: OK: no RAID installed [01:23:01] PROBLEM - Disk space on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:23:02] RECOVERY - DPKG on mw1143 is OK: All packages OK [01:23:02] RECOVERY - RAID on mw1143 is OK: OK: no RAID installed [01:23:02] RECOVERY - Apache HTTP on mw1143 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.074 second response time [01:24:51] PROBLEM - DPKG on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:24:51] PROBLEM - twemproxy process on mw1141 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:24:51] PROBLEM - Disk space on mw1124 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:24:51] RECOVERY - Disk space on mw1142 is OK: DISK OK [01:25:51] PROBLEM - RAID on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:26:01] RECOVERY - twemproxy process on mw1142 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [01:26:02] RECOVERY - DPKG on mw1142 is OK: All packages OK [01:26:11] RECOVERY - SSH on mw1142 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [01:28:51] RECOVERY - Disk space on mw1124 is OK: DISK OK [01:29:11] RECOVERY - SSH on mw1124 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [01:29:21] RECOVERY - SSH on mw1141 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [01:29:41] RECOVERY - RAID on mw1142 is OK: OK: no RAID installed [01:29:42] RECOVERY - Apache HTTP on mw1142 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.065 second response time [01:30:51] PROBLEM - RAID on mw1128 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:31:01] PROBLEM - Apache HTTP on mw1122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:31:11] PROBLEM - RAID on mw1120 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:31:11] PROBLEM - DPKG on mw1143 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:31:11] PROBLEM - RAID on mw1143 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:31:41] PROBLEM - Apache HTTP on mw1204 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:31:51] RECOVERY - RAID on mw1128 is OK: OK: no RAID installed [01:31:51] PROBLEM - Apache HTTP on mw1132 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:31:51] PROBLEM - Apache HTTP on mw1207 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:31:51] PROBLEM - Apache HTTP on mw1120 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:31:52] PROBLEM - Apache HTTP on mw1189 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:31:52] PROBLEM - DPKG on mw1133 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:32:02] RECOVERY - RAID on mw1120 is OK: OK: no RAID installed [01:32:11] PROBLEM - DPKG on mw1117 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:32:12] RECOVERY - DPKG on mw1143 is OK: All packages OK [01:32:12] RECOVERY - DPKG on mw1141 is OK: All packages OK [01:32:12] PROBLEM - SSH on mw1124 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:32:12] PROBLEM - Apache HTTP on mw1117 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:32:31] RECOVERY - Apache HTTP on mw1204 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.207 second response time [01:32:41] RECOVERY - DPKG on mw1133 is OK: All packages OK [01:32:42] RECOVERY - Apache HTTP on mw1207 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 7.085 second response time [01:32:51] PROBLEM - Disk space on mw1141 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:32:51] PROBLEM - Apache HTTP on mw1142 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:32:51] PROBLEM - RAID on mw1142 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:32:51] RECOVERY - Apache HTTP on mw1122 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 1.002 second response time [01:33:01] RECOVERY - DPKG on mw1117 is OK: All packages OK [01:33:01] RECOVERY - Apache HTTP on mw1117 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 808 bytes in 0.069 second response time [01:33:11] RECOVERY - SSH on mw1124 is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [01:33:11] RECOVERY - twemproxy process on mw1124 is OK: PROCS OK: 1 process with UID = 65534 (nobody), command name nutcracker [01:33:41] PROBLEM - SSH on mw1121 is CRITICAL: Server answer: [01:33:41] RECOVERY - RAID on mw1142 is OK: OK: no RAID installed [01:33:51] PROBLEM - Apache HTTP on mw1121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:33:51] PROBLEM - SSH on mw1116 is CRITICAL: Server answer: [01:34:01] PROBLEM - twemproxy process on mw1116 is CRITICAL: CHECK_NRPE: Error - Could not complete SSL handshake. [01:34:11] PROBLEM - Apache HTTP on mw1146 is CRITICAL: Connection timed out [01:34:11] PROBLEM - Apache HTTP on mw1201 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:34:11] PROBLEM - Apache HTTP on mw1130 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:34:11] PROBLEM - Apache HTTP on mw1194 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:34:11] PROBLEM - RAID on mw1130 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:34:11] PROBLEM - RAID on mw1121 is CRITICAL: CHECK_NRPE: Error - Could not complete SSL handshake. [01:34:21] PROBLEM - DPKG on mw1116 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:34:21] PROBLEM - RAID on mw1133 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [01:34:41]