[00:04:52] PROBLEM Total processes is now: WARNING on tools-login.pmtpa.wmflabs 10.4.0.220 output: PROCS WARNING: 153 processes [00:06:43] PROBLEM Free ram is now: WARNING on mobile-osm2.pmtpa.wmflabs 10.4.1.67 output: Warning: 19% free memory [00:13:53] RECOVERY Total processes is now: OK on parsoid-spof.pmtpa.wmflabs 10.4.0.33 output: PROCS OK: 146 processes [00:21:25] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [00:28:33] PROBLEM Free ram is now: WARNING on swift-be2.pmtpa.wmflabs 10.4.0.112 output: Warning: 19% free memory [00:41:43] RECOVERY Free ram is now: OK on mobile-osm2.pmtpa.wmflabs 10.4.1.67 output: OK: 22% free memory [00:51:23] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [01:09:43] PROBLEM Free ram is now: WARNING on mobile-osm2.pmtpa.wmflabs 10.4.1.67 output: Warning: 19% free memory [01:11:52] PROBLEM Total processes is now: CRITICAL on bots-salebot.pmtpa.wmflabs 10.4.0.163 output: PROCS CRITICAL: 253 processes [01:21:23] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [01:23:53] PROBLEM Total processes is now: WARNING on nova-precise2.pmtpa.wmflabs 10.4.1.57 output: PROCS WARNING: 152 processes [01:31:43] PROBLEM Total processes is now: CRITICAL on bots-bsql01.pmtpa.wmflabs 10.4.0.226 output: CHECK_NRPE: Socket timeout after 10 seconds. [01:32:04] PROBLEM Free ram is now: CRITICAL on bots-bsql01.pmtpa.wmflabs 10.4.0.226 output: CHECK_NRPE: Socket timeout after 10 seconds. [01:34:43] PROBLEM dpkg-check is now: CRITICAL on bots-bsql01.pmtpa.wmflabs 10.4.0.226 output: CHECK_NRPE: Socket timeout after 10 seconds. [01:35:13] PROBLEM Current Load is now: CRITICAL on bots-bsql01.pmtpa.wmflabs 10.4.0.226 output: CHECK_NRPE: Socket timeout after 10 seconds. [01:35:13] PROBLEM SSH is now: CRITICAL on bots-bsql01.pmtpa.wmflabs 10.4.0.226 output: CRITICAL - Socket timeout after 10 seconds [01:36:03] PROBLEM Disk Space is now: CRITICAL on bots-bsql01.pmtpa.wmflabs 10.4.0.226 output: CHECK_NRPE: Socket timeout after 10 seconds. [01:51:23] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [02:01:52] RECOVERY Total processes is now: OK on bots-salebot.pmtpa.wmflabs 10.4.0.163 output: PROCS OK: 124 processes [02:04:32] RECOVERY dpkg-check is now: OK on bots-bsql01.pmtpa.wmflabs 10.4.0.226 output: All packages OK [02:04:52] RECOVERY Total processes is now: OK on tools-login.pmtpa.wmflabs 10.4.0.220 output: PROCS OK: 149 processes [02:05:53] RECOVERY Disk Space is now: OK on bots-bsql01.pmtpa.wmflabs 10.4.0.226 output: DISK OK [02:06:33] PROBLEM Total processes is now: WARNING on bots-bsql01.pmtpa.wmflabs 10.4.0.226 output: PROCS WARNING: 198 processes [02:06:53] RECOVERY Free ram is now: OK on bots-bsql01.pmtpa.wmflabs 10.4.0.226 output: OK: 88% free memory [02:07:23] RECOVERY SSH is now: OK on bots-bsql01.pmtpa.wmflabs 10.4.0.226 output: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1 (protocol 2.0) [02:11:53] PROBLEM Total processes is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS WARNING: 179 processes [02:14:53] PROBLEM Total processes is now: CRITICAL on bots-salebot.pmtpa.wmflabs 10.4.0.163 output: PROCS CRITICAL: 276 processes [02:16:33] PROBLEM Total processes is now: CRITICAL on bots-bsql01.pmtpa.wmflabs 10.4.0.226 output: PROCS CRITICAL: 201 processes [02:16:43] PROBLEM Total processes is now: CRITICAL on bots-bnr1.pmtpa.wmflabs 10.4.1.68 output: PROCS CRITICAL: 201 processes [02:16:53] RECOVERY Total processes is now: OK on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS OK: 150 processes [02:20:52] PROBLEM Total processes is now: WARNING on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS WARNING: 198 processes [02:21:32] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [02:28:34] RECOVERY Free ram is now: OK on swift-be2.pmtpa.wmflabs 10.4.0.112 output: OK: 20% free memory [02:34:53] PROBLEM Total processes is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS WARNING: 178 processes [02:39:32] PROBLEM Disk Space is now: CRITICAL on documentation-small-precise.pmtpa.wmflabs 10.4.1.114 output: Connection refused by host [02:40:13] PROBLEM Free ram is now: CRITICAL on documentation-small-precise.pmtpa.wmflabs 10.4.1.114 output: Connection refused by host [02:40:53] PROBLEM Current Load is now: CRITICAL on documentation-small-precise.pmtpa.wmflabs 10.4.1.114 output: CHECK_NRPE: Error - Could not complete SSL handshake. [02:41:43] PROBLEM Total processes is now: CRITICAL on documentation-small-precise.pmtpa.wmflabs 10.4.1.114 output: CHECK_NRPE: Error - Could not complete SSL handshake. [02:42:23] PROBLEM dpkg-check is now: CRITICAL on documentation-small-precise.pmtpa.wmflabs 10.4.1.114 output: CHECK_NRPE: Error - Could not complete SSL handshake. [02:45:53] PROBLEM Current Load is now: WARNING on bots-bsql01.pmtpa.wmflabs 10.4.0.226 output: WARNING - load average: 2.51, 2.81, 16.12 [02:52:13] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [02:56:33] PROBLEM Free ram is now: WARNING on swift-be2.pmtpa.wmflabs 10.4.0.112 output: Warning: 19% free memory [02:56:43] PROBLEM Total processes is now: WARNING on bots-bnr1.pmtpa.wmflabs 10.4.1.68 output: PROCS WARNING: 168 processes [03:01:33] PROBLEM Total processes is now: WARNING on bots-bsql01.pmtpa.wmflabs 10.4.0.226 output: PROCS WARNING: 187 processes [03:10:52] RECOVERY Current Load is now: OK on bots-bsql01.pmtpa.wmflabs 10.4.0.226 output: OK - load average: 0.56, 0.53, 3.88 [03:23:03] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [03:53:04] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [04:23:43] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [04:39:43] RECOVERY Free ram is now: OK on mobile-osm2.pmtpa.wmflabs 10.4.1.67 output: OK: 22% free memory [04:53:43] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [04:56:33] RECOVERY Free ram is now: OK on swift-be2.pmtpa.wmflabs 10.4.0.112 output: OK: 20% free memory [05:07:43] PROBLEM Free ram is now: WARNING on mobile-osm2.pmtpa.wmflabs 10.4.1.67 output: Warning: 19% free memory [05:23:43] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [05:24:33] PROBLEM Free ram is now: WARNING on swift-be2.pmtpa.wmflabs 10.4.0.112 output: Warning: 19% free memory [05:53:43] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [06:23:43] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [06:28:55] PROBLEM Total processes is now: WARNING on parsoid-spof.pmtpa.wmflabs 10.4.0.33 output: PROCS WARNING: 152 processes [06:32:52] PROBLEM Total processes is now: WARNING on tools-login.pmtpa.wmflabs 10.4.0.220 output: PROCS WARNING: 154 processes [06:43:53] RECOVERY Total processes is now: OK on parsoid-spof.pmtpa.wmflabs 10.4.0.33 output: PROCS OK: 146 processes [06:51:22] PROBLEM Free ram is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: Warning: 16% free memory [06:52:52] RECOVERY Total processes is now: OK on tools-login.pmtpa.wmflabs 10.4.0.220 output: PROCS OK: 150 processes [06:53:52] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [07:06:23] RECOVERY Free ram is now: OK on bots-cb.pmtpa.wmflabs 10.4.0.44 output: OK: 20% free memory [07:14:22] PROBLEM Free ram is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: Warning: 15% free memory [07:23:52] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [07:29:33] RECOVERY Free ram is now: OK on swift-be2.pmtpa.wmflabs 10.4.0.112 output: OK: 20% free memory [07:53:52] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [07:57:33] PROBLEM Free ram is now: WARNING on swift-be2.pmtpa.wmflabs 10.4.0.112 output: Warning: 19% free memory [08:23:53] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [08:37:43] RECOVERY Free ram is now: OK on mobile-osm2.pmtpa.wmflabs 10.4.1.67 output: OK: 22% free memory [08:45:43] PROBLEM Free ram is now: WARNING on mobile-osm2.pmtpa.wmflabs 10.4.1.67 output: Warning: 19% free memory [08:53:53] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [09:25:42] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [09:56:23] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [09:57:33] RECOVERY Free ram is now: OK on swift-be2.pmtpa.wmflabs 10.4.0.112 output: OK: 20% free memory [10:01:42] PROBLEM Total processes is now: CRITICAL on bots-bnr1.pmtpa.wmflabs 10.4.1.68 output: PROCS CRITICAL: 213 processes [10:05:53] PROBLEM Total processes is now: CRITICAL on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS CRITICAL: 201 processes [10:23:23] PROBLEM Free ram is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: Warning: 17% free memory [10:24:54] PROBLEM Total processes is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS WARNING: 174 processes [10:25:33] PROBLEM Free ram is now: WARNING on swift-be2.pmtpa.wmflabs 10.4.0.112 output: Warning: 19% free memory [10:26:23] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [10:28:23] RECOVERY Free ram is now: OK on bots-cb.pmtpa.wmflabs 10.4.0.44 output: OK: 22% free memory [10:29:53] RECOVERY Total processes is now: OK on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS OK: 142 processes [10:56:23] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [11:06:23] PROBLEM Free ram is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: Warning: 18% free memory [11:07:53] PROBLEM Total processes is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS WARNING: 171 processes [11:26:23] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [11:56:23] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [12:25:33] RECOVERY Free ram is now: OK on swift-be2.pmtpa.wmflabs 10.4.0.112 output: OK: 20% free memory [12:26:23] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [12:40:42] RECOVERY Free ram is now: OK on mobile-osm2.pmtpa.wmflabs 10.4.1.67 output: OK: 22% free memory [12:48:42] PROBLEM Free ram is now: WARNING on mobile-osm2.pmtpa.wmflabs 10.4.1.67 output: Warning: 19% free memory [12:56:32] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [12:58:32] PROBLEM Free ram is now: WARNING on swift-be2.pmtpa.wmflabs 10.4.0.112 output: Warning: 19% free memory [13:18:53] PROBLEM Current Load is now: CRITICAL on ngnix-ffuqua-cache.pmtpa.wmflabs 10.4.0.80 output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:19:33] PROBLEM Disk Space is now: CRITICAL on ngnix-ffuqua-cache.pmtpa.wmflabs 10.4.0.80 output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:20:12] PROBLEM Free ram is now: CRITICAL on ngnix-ffuqua-cache.pmtpa.wmflabs 10.4.0.80 output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:20:32] RECOVERY Total processes is now: OK on bots-bnr2.pmtpa.wmflabs 10.4.0.40 output: PROCS OK: 148 processes [13:21:42] PROBLEM Total processes is now: CRITICAL on ngnix-ffuqua-cache.pmtpa.wmflabs 10.4.0.80 output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:22:22] PROBLEM dpkg-check is now: CRITICAL on ngnix-ffuqua-cache.pmtpa.wmflabs 10.4.0.80 output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:28:03] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [13:50:32] PROBLEM Free ram is now: CRITICAL on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:55:43] PROBLEM Total processes is now: CRITICAL on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:55:53] PROBLEM Current Load is now: CRITICAL on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:55:53] PROBLEM dpkg-check is now: CRITICAL on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:56:03] PROBLEM SSH is now: CRITICAL on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: CRITICAL - Socket timeout after 10 seconds [13:57:33] PROBLEM Disk Space is now: CRITICAL on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: CHECK_NRPE: Error - Could not complete SSL handshake. [13:58:09] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [14:00:32] RECOVERY Free ram is now: OK on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: OK: 95% free memory [14:00:42] RECOVERY Total processes is now: OK on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: PROCS OK: 113 processes [14:00:52] RECOVERY Current Load is now: OK on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: OK - load average: 2.27, 1.16, 0.44 [14:00:52] RECOVERY SSH is now: OK on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.1 (protocol 2.0) [14:00:52] RECOVERY dpkg-check is now: OK on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: All packages OK [14:02:32] RECOVERY Disk Space is now: OK on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: DISK OK [14:18:53] PROBLEM Current Load is now: CRITICAL on cvn-app1.pmtpa.wmflabs 10.4.1.90 output: CRITICAL - load average: 125.57, 228.02, 109.04 [14:28:03] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [14:58:34] RECOVERY Free ram is now: OK on swift-be2.pmtpa.wmflabs 10.4.0.112 output: OK: 20% free memory [14:58:52] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [15:26:32] PROBLEM Free ram is now: WARNING on swift-be2.pmtpa.wmflabs 10.4.0.112 output: Warning: 19% free memory [15:29:02] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [15:59:03] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [16:29:03] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [16:36:52] PROBLEM Total processes is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS WARNING: 178 processes [16:38:42] RECOVERY Free ram is now: OK on mobile-osm2.pmtpa.wmflabs 10.4.1.67 output: OK: 21% free memory [16:41:53] RECOVERY Total processes is now: OK on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS OK: 135 processes [17:00:43] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [17:01:43] PROBLEM Free ram is now: WARNING on mobile-osm2.pmtpa.wmflabs 10.4.1.67 output: Warning: 19% free memory [17:13:22] PROBLEM Free ram is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: Warning: 16% free memory [17:14:52] PROBLEM Total processes is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS WARNING: 191 processes [17:18:23] RECOVERY Free ram is now: OK on bots-cb.pmtpa.wmflabs 10.4.0.44 output: OK: 27% free memory [17:26:33] RECOVERY Free ram is now: OK on swift-be2.pmtpa.wmflabs 10.4.0.112 output: OK: 20% free memory [17:31:22] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [17:54:33] PROBLEM Free ram is now: WARNING on swift-be2.pmtpa.wmflabs 10.4.0.112 output: Warning: 18% free memory [18:01:22] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [18:10:53] PROBLEM Total processes is now: WARNING on bastion1.pmtpa.wmflabs 10.4.1.55 output: PROCS WARNING: 152 processes [18:15:52] RECOVERY Total processes is now: OK on bastion1.pmtpa.wmflabs 10.4.1.55 output: PROCS OK: 147 processes [18:28:52] PROBLEM Total processes is now: WARNING on bastion1.pmtpa.wmflabs 10.4.1.55 output: PROCS WARNING: 155 processes [18:31:23] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [18:38:43] PROBLEM Free ram is now: WARNING on swift-be3.pmtpa.wmflabs 10.4.0.124 output: Warning: 19% free memory [19:01:23] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [19:31:23] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [20:01:32] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [20:32:33] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [20:41:43] RECOVERY Free ram is now: OK on mobile-osm2.pmtpa.wmflabs 10.4.1.67 output: OK: 22% free memory [21:03:02] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [21:04:42] PROBLEM Free ram is now: WARNING on mobile-osm2.pmtpa.wmflabs 10.4.1.67 output: Warning: 19% free memory [21:33:03] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [21:56:42] PROBLEM Total processes is now: WARNING on bots-bnr1.pmtpa.wmflabs 10.4.1.68 output: PROCS WARNING: 198 processes [22:01:43] PROBLEM Total processes is now: CRITICAL on bots-bnr1.pmtpa.wmflabs 10.4.1.68 output: PROCS CRITICAL: 204 processes [22:03:03] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [22:06:42] PROBLEM Total processes is now: WARNING on bots-bnr1.pmtpa.wmflabs 10.4.1.68 output: PROCS WARNING: 198 processes [22:34:12] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [22:43:53] PROBLEM Total processes is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS WARNING: 163 processes [22:47:23] PROBLEM Free ram is now: WARNING on bots-cb.pmtpa.wmflabs 10.4.0.44 output: Warning: 12% free memory [22:52:23] RECOVERY Free ram is now: OK on bots-cb.pmtpa.wmflabs 10.4.0.44 output: OK: 20% free memory [22:53:53] RECOVERY Total processes is now: OK on bots-cb.pmtpa.wmflabs 10.4.0.44 output: PROCS OK: 144 processes [23:04:12] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [23:18:43] PROBLEM host: fundraising-dev-awight.pmtpa.wmflabs is DOWN address: 10.4.0.253 CRITICAL - Host Unreachable (10.4.0.253) [23:34:12] PROBLEM host: wikidata-client-test.pmtpa.wmflabs is DOWN address: 10.4.1.3 CRITICAL - Host Unreachable (10.4.1.3) [23:40:53] PROBLEM Total processes is now: WARNING on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS WARNING: 200 processes [23:45:53] PROBLEM Total processes is now: CRITICAL on bots-bnr3.pmtpa.wmflabs 10.4.0.81 output: PROCS CRITICAL: 201 processes [23:49:43] PROBLEM host: fundraising-dev-awight.pmtpa.wmflabs is DOWN address: 10.4.0.253 CRITICAL - Host Unreachable (10.4.0.253)