[00:18:42] Nomwibble [00:38:15] they ask if it's from abroad.. With ODesk, I'm not sure if it is abroad... [02:55:43] RECOVERY Puppet freshness is now: OK on deployment-nfs-memc i-000000d7 output: puppet ran at Sun Apr 8 02:55:35 UTC 2012 [03:39:33] PROBLEM Free ram is now: WARNING on utils-abogott i-00000131 output: Warning: 15% free memory [03:42:33] PROBLEM Free ram is now: WARNING on orgcharts-dev i-0000018f output: Warning: 15% free memory [03:52:43] PROBLEM Free ram is now: WARNING on nova-daas-1 i-000000e7 output: Warning: 12% free memory [03:59:33] PROBLEM Free ram is now: CRITICAL on utils-abogott i-00000131 output: Critical: 3% free memory [04:02:33] PROBLEM Free ram is now: CRITICAL on orgcharts-dev i-0000018f output: Critical: 3% free memory [04:02:43] PROBLEM Free ram is now: WARNING on test-oneiric i-00000187 output: Warning: 17% free memory [04:04:33] RECOVERY Free ram is now: OK on utils-abogott i-00000131 output: OK: 96% free memory [04:07:39] RECOVERY Free ram is now: OK on orgcharts-dev i-0000018f output: OK: 96% free memory [04:12:39] PROBLEM Free ram is now: CRITICAL on nova-daas-1 i-000000e7 output: Critical: 5% free memory [04:17:39] RECOVERY Free ram is now: OK on nova-daas-1 i-000000e7 output: OK: 93% free memory [04:22:39] PROBLEM Free ram is now: CRITICAL on test-oneiric i-00000187 output: Critical: 4% free memory [04:27:39] RECOVERY Free ram is now: OK on test-oneiric i-00000187 output: OK: 97% free memory [06:56:42] RECOVERY Disk Space is now: OK on deployment-transcoding i-00000105 output: DISK OK [07:04:32] RECOVERY Disk Space is now: OK on aggregator1 i-0000010c output: DISK OK [07:45:40] PROBLEM Puppet freshness is now: CRITICAL on puppet-lucid i-00000080 output: Puppet has not run in last 20 hours [09:27:39] PROBLEM Disk Space is now: CRITICAL on aggregator1 i-0000010c output: DISK CRITICAL - free space: / 284 MB (2% inode=93%): [10:25:49] PROBLEM Disk Space is now: WARNING on deployment-transcoding i-00000105 output: DISK WARNING - free space: / 78 MB (5% inode=53%): [14:22:00] Thehelpfulone: can u do something with all the spam we have on beta [14:22:03] english wiki [14:24:55] * Thehelpfulone looks [14:29:46] petan: how easy it is to reset deployment-prep? [14:33:34] it looks like a broken bot [14:38:55] ouch: POST /w/index.php?title=Talk:Body_odor%2B%2B%2B%2B%2B%2B%2B%2B%2B%2BResult:%2BGET-%C3%B2%C3%A0%C3%A9%C3%AC%C3%A0%C3%B3%C3%B2%C3%AE%C3%A2%2B1;%2B%C3%A8%C3%B1%C3%AF%C3%AE%C3%AB%C3%BC%C3%A7%C3%AE%C3%A2%C3%A0%C3%AD%2B%C3%AD%C3%A8%C3%AA%C3%AD%C3%A5%C3%A9%C3%AC%2B%22gaveDaubage%22;%2B%C3%AF%C3%B0%C3%A8%C3%B1%C3%B3%C3%B2%C3%B1%C3%B2%C3%A2%C3%B3%C3%A5%C3%B2%2Bnofollow;%2B%C3%B3%C3%B1%C3%AF%C3%A5%C3%B5%2B-%2B%C3%A7%C3%A0%C3%AF%C3%AE%C3%B1%C [14:39:00] what [14:39:03] Thehelpfulone: ? [14:39:37] rolling back the database is something we decided not to do [14:40:45] petan: okay, my idea was to avoid using time just deleting the endless spam and just rollback [14:42:39] or perhaps we could ask more people to become ops there [14:42:50] or fix the antibot [14:43:11] also, are you globally blocking the bots? [14:43:16] how the people deal with this on prod [14:47:18] what did those pages contain? [15:01:42] Platonides: which [15:01:45] spam [15:01:48] nothing else [15:02:05] I will create abuse filter for that [15:02:17] but I guess that proper blocking would solve it too [15:02:28] I would indef all ip's who spam [15:02:35] it's a test wiki anyway so what [15:07:45] Thehelpfulone: I meant block globally, not locally [15:07:51] but it's ok [15:08:02] with an expiry time of 48 hour [15:08:10] it could be indef as well, this isn't prod [15:08:28] my thoughts are it's a test wiki [15:08:33] yes it is [15:08:36] do we need to spend time blocking people etc? [15:08:56] it's not really harming anyone with the spam [15:09:08] if there is any way to avoid getting spammed without blocking ip's of spammers I will be happy to do that [15:09:23] robots.txt excludes us from search engines too no? [15:09:27] yes [15:09:35] if you want petan, maybe a global abuse filter [15:09:38] anyway if someone spam test.wikipedia do other people just overlook it? [15:09:43] I don't think so [15:10:00] there isn't such a tool [15:10:26] yeah that's because it's one wiki, and that one is in robots.txt I believe [15:11:15] hm, I will think of that [17:46:31] PROBLEM Puppet freshness is now: CRITICAL on puppet-lucid i-00000080 output: Puppet has not run in last 20 hours [22:38:31] PROBLEM Puppet freshness is now: CRITICAL on mobile-enwp i-000000ce output: Puppet has not run in last 20 hours