[00:24:38] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [00:54:38] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [01:24:38] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [01:54:38] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [02:24:38] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [02:38:28] RECOVERY Free ram is now: OK on bots-sql3 bots-sql3 output: OK: 20% free memory [02:43:29] PROBLEM Free ram is now: WARNING on puppet-lucid puppet-lucid output: Warning: 12% free memory [02:53:29] PROBLEM Free ram is now: CRITICAL on puppet-lucid puppet-lucid output: Critical: 3% free memory [02:54:49] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [03:11:29] PROBLEM Free ram is now: WARNING on bots-sql3 bots-sql3 output: Warning: 18% free memory [03:24:49] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [03:27:07] no petan today and i forgot to look here during his time [03:28:29] RECOVERY Free ram is now: OK on puppet-lucid puppet-lucid output: OK: 20% free memory [03:54:49] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [04:24:49] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [04:54:49] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [05:24:59] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [05:54:59] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [06:24:59] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [06:55:07] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [07:25:07] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [07:55:07] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [08:25:07] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [08:55:07] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [09:25:07] PROBLEM host: deployment-web2 is DOWN address: deployment-web2 CRITICAL - Host Unreachable (deployment-web2) [09:40:07] RECOVERY Free ram is now: OK on deployment-web2 deployment-web2 output: OK: 85% free memory [09:40:17] RECOVERY host: deployment-web2 is UP address: deployment-web2 PING OK - Packet loss = 0%, RTA = 1.47 ms [09:40:27] RECOVERY Current Load is now: OK on deployment-web2 deployment-web2 output: OK - load average: 0.48, 0.23, 0.09 [09:41:07] RECOVERY HTTP is now: OK on deployment-web2 deployment-web2 output: HTTP OK: HTTP/1.1 302 Found - 554 bytes in 0.007 second response time [09:41:17] RECOVERY Current Users is now: OK on deployment-web2 deployment-web2 output: USERS OK - 0 users currently logged in [09:41:37] RECOVERY Total Processes is now: OK on deployment-web2 deployment-web2 output: PROCS OK: 115 processes [09:41:42] RECOVERY dpkg-check is now: OK on deployment-web2 deployment-web2 output: All packages OK [09:42:37] RECOVERY Disk Space is now: OK on deployment-web2 deployment-web2 output: DISK OK [09:42:47] RECOVERY SSH is now: OK on deployment-web2 deployment-web2 output: SSH OK - OpenSSH_5.3p1 Debian-3ubuntu7 (protocol 2.0) [11:23:04] can someone svn up /usr/local/apache/common/live/extensions/TimedMediaHandler/ on deployment-web [14:59:20] petan, where's that memcached puppet class? [15:52:39] petan: ping [15:57:20] * jeremyb spies a petan on one of my recent edits ;) [17:40:27] PROBLEM Free ram is now: WARNING on nagios 127.0.0.1 output: Warning: 19% free memory [17:46:20] !labs [17:46:20] https://labsconsole.wikimedia.org/wiki/$1 [17:48:24] the deployment site seems to keep forgetting that I'm logged in (keeps logging me out, then on a reload remembers I'm logged in). Is this a known bug? [17:50:30] maybe it's a problem with caches... [17:50:32] which site? [17:50:42] hi-wp deployment [17:51:01] been happening for a day or two [17:53:04] * Platonides looks for his password [17:53:12] lol [17:53:24] it seems i have an account there [17:53:32] but I don't remember creating it [17:53:36] lolol [17:54:01] Hey, wait; there's no "Jimbo Wales" there yet!!! [17:54:40] what other mediawiki IRC's are there [17:56:11] seems I registered there on 9 Jan [17:56:31] what other mediawiki IRC's are there all i know are this, #mediawiki and #mediawikiIRC [17:56:56] you made up #mediawikiIRC yourself [17:57:28] woops, i keep forgetting ## not # [17:58:47] still, what other channels are there [17:59:10] and what does erroneous_nickna gives channel operator status to Platonides mean [18:00:01] for viewing all available channes do /list [18:01:05] and what does erroneous_nickna gives channel operator status to Platonides mean [18:03:18] you should change topic it talks about in the past [18:08:09] wtf, why can't I change a file rw-rw-r-- when I'm member of its group? [18:10:28] it is acting as if it was rw-r---r-- [18:11:59] http://hi.wikipedia.beta.wmflabs.org/w/api.php [18:12:01] Catchable fatal error: Argument 2 passed to ApiParseExtender::onAPIGetParamDescription() must be an array, boolean given in /usr/local/apache/common-local/live-hom/extensions/MobileFrontend/ApiParseExtender.php on line 31 [18:12:25] !bla [18:16:15] oh, now it's a 500 internal server error [18:16:20] http://hi.wikipedia.beta.wmflabs.org/w/api.php [18:16:25] Error 139 (net::ERR_TEMPORARILY_THROTTLED): Requests to the server have been temporarily throttled. [18:16:28] what's going on ? [19:10:43] hey. anyone here who can help me with credentials? [19:11:07] I can not create instances in a project I am part of: "No Nova credentials found for your account. [21:01:11] danakim: Log out of labsconsole and back in. This is a MediaWiki bug (or rather a bug in the MW->LDAP bridge) [23:03:48] PROBLEM Current Load is now: CRITICAL on driver-dev2 driver-dev2 output: CHECK_NRPE: Error - Could not complete SSL handshake. [23:04:28] PROBLEM Current Users is now: CRITICAL on driver-dev2 driver-dev2 output: CHECK_NRPE: Error - Could not complete SSL handshake. [23:05:08] PROBLEM Disk Space is now: CRITICAL on driver-dev2 driver-dev2 output: CHECK_NRPE: Error - Could not complete SSL handshake. [23:05:48] PROBLEM Free ram is now: CRITICAL on driver-dev2 driver-dev2 output: CHECK_NRPE: Error - Could not complete SSL handshake. [23:06:58] PROBLEM Total Processes is now: CRITICAL on driver-dev2 driver-dev2 output: CHECK_NRPE: Error - Could not complete SSL handshake. [23:07:38] PROBLEM dpkg-check is now: CRITICAL on driver-dev2 driver-dev2 output: CHECK_NRPE: Error - Could not complete SSL handshake. [23:33:48] RECOVERY Current Load is now: OK on driver-dev2 driver-dev2 output: OK - load average: 0.58, 0.31, 0.21 [23:34:28] RECOVERY Current Users is now: OK on driver-dev2 driver-dev2 output: USERS OK - 1 users currently logged in [23:35:08] RECOVERY Disk Space is now: OK on driver-dev2 driver-dev2 output: DISK OK [23:35:48] RECOVERY Free ram is now: OK on driver-dev2 driver-dev2 output: OK: 82% free memory [23:36:58] RECOVERY Total Processes is now: OK on driver-dev2 driver-dev2 output: PROCS OK: 101 processes [23:37:38] RECOVERY dpkg-check is now: OK on driver-dev2 driver-dev2 output: All packages OK