[00:05:49] PROBLEM - Puppet freshness on copper is CRITICAL: Puppet has not run in the last 10 hours [00:05:50] PROBLEM - Puppet freshness on zinc is CRITICAL: Puppet has not run in the last 10 hours [00:06:22] Alchimista, u there? [00:07:13] we at pt.wikt are in need of advice [00:07:36] we have an anon trying to vandalize for nearly 48 hours [00:11:08] TimStarling: Can you deploy a blocking js bug fix ? CR by brion [00:11:16] r106398 [00:13:55] malafaya: yap, here [00:17:59] malafaya: ainda p ai? [00:21:15] Alchimista, desculpa [00:21:18] tive visita rápida [00:21:38] no problemo. Tivemos no news um problema similar por uns tempos, creio que agora estabilizou [00:21:41] we have an anon trying to vandalize every 15 minutes [00:21:54] thet problem is that we have to sleep some time... [00:22:12] and he takes the chance to make tens of bad edits [00:22:22] currently we had to block all anons [00:22:23] sabes que podes falar pt comigo... [00:22:42] sim, mas assim falo para todos ;) [00:22:47] Ah :P [00:23:00] ...se é que lêem [00:23:08] well, on news we had an attack like that, and tryed to solve it with abusefilter [00:23:22] creio que não, n é tanto da parte tech [00:23:31] there is no simple rule for that, except block all non-autoconfs [00:23:55] seems that OP are used.. [00:24:08] what is OP? [00:24:12] ah, open proxies [00:24:24] yes, nearly all the time [00:24:38] sometimes, he doesn't, just to check if the abuse filter is off [00:25:44] you can see him trying to check the block at 23:00, 23:44 and 00:09 UTC [00:28:58] yap. seems the same one we had on ptnews [00:29:16] when did he started the vandalisms? [00:34:57] malafaya: ^ [00:35:19] yesterday [00:35:42] so i gess soon there will be a proposal to optin the global sysops :P [00:36:03] but you can use abusefilter to prevent most of those vandalisms [00:36:39] how? there's nothing typical about his edits [00:37:02] sometimes he uses swear words, sometimes he just messes up pages [00:38:27] https://pt.wiktionary.org/w/index.php?title=adjacent&diff=prev&oldid=1335326 -> bad words [00:39:02] https://pt.wiktionary.org/w/index.php?title=toxicodependente&diff=prev&oldid=1335322 -> bad words plus tipical test edits [00:39:05] bad words are actually "ok" for a dictionary, but we could limit them for anons [00:39:35] right ;) that would prevent some of the abuses. [00:40:39] there are a lot of bad words, so you may start from there. Another thing you may take in consideration is the removal of lots of info, you can also limit it to reg users [00:42:18] ok, thanks for the info [00:42:40] but i think i'll try that only after tonight :). too risky to test when no one's around [00:43:17] if you think it may help, i was coding an irc bot to report when OP and vandalisms where happening on pt.news, i can try finish him and use it on wikt [00:44:43] and if he uses a lot of OP, peraps you could talk to RonalB, he has a tool to report on wiki when an OP is used there, like this: http://pt.wikinews.org/wiki/Wikinot%C3%ADcias:Detec%C3%A7%C3%A3o_de_proxies_abertos [00:49:30] malafaya: o que fez ele à minha PU e PDU? https://pt.wiktionary.org/w/index.php?title=Utilizador%3AAlchimista&action=historysubmit&diff=1334756&oldid=962021 [01:15:33] New patchset: Lcarr; "Adding in sun JDK to contint servers" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1591 [01:15:44] New review: gerrit2; "Change did not pass lint check. You will need to send an amended patchset for this (see: https://lab..." [operations/puppet] (production); V: -1 - https://gerrit.wikimedia.org/r/1591 [01:18:51] New patchset: Lcarr; "Adding in sun JDK to contint servers" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1591 [01:19:03] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1591 [01:30:33] New patchset: Lcarr; "Adding in sun JDK to contint servers" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1591 [01:37:09] New patchset: Lcarr; "Adding in sun JDK to contint servers" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1591 [01:37:19] New review: gerrit2; "Change did not pass lint check. You will need to send an amended patchset for this (see: https://lab..." [operations/puppet] (production); V: -1 - https://gerrit.wikimedia.org/r/1591 [01:39:38] New patchset: Lcarr; "Adding in sun JDK to contint servers" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1591 [01:39:49] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1591 [01:42:08] New patchset: Lcarr; "Adding in sun JDK to contint servers" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1591 [01:42:20] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1591 [01:44:39] New review: Bhartshorne; "(no comment)" [operations/puppet] (production); V: 0 C: 1; - https://gerrit.wikimedia.org/r/1591 [01:45:02] New review: Lcarr; "(no comment)" [operations/puppet] (production); V: 0 C: 2; - https://gerrit.wikimedia.org/r/1591 [01:45:02] Change merged: Lcarr; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1591 [02:00:14] !log LocalisationUpdate failed (1.18) at Fri Dec 16 02:00:14 UTC 2011 [02:00:24] Logged the message, Master [03:11:22] PROBLEM - Puppet freshness on es1002 is CRITICAL: Puppet has not run in the last 10 hours [03:12:14] PROBLEM - SSH on maerlant is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:30:27] PROBLEM - Puppet freshness on bast1001 is CRITICAL: Puppet has not run in the last 10 hours [03:32:26] PROBLEM - Puppet freshness on fenari is CRITICAL: Puppet has not run in the last 10 hours [03:51:49] in case it is not known.. and in case the fault is not on my end (Germany, Vodafone/Arcor): commons seems to have problems with the thumbnail servers.. images fail to load [03:54:52] works again... [04:44:04] RECOVERY - SSH on maerlant is OK: SSH OK - OpenSSH_5.3p1 Debian-3ubuntu7 (protocol 2.0) [04:54:51] PROBLEM - Recursive DNS on 91.198.174.6 is CRITICAL: CRITICAL - Plugin timed out while executing system call [06:37:55] PROBLEM - Puppet freshness on snapshot4 is CRITICAL: Puppet has not run in the last 10 hours [07:12:05] !log thumbs cleaner awake for the day [07:12:15] Logged the message, Master [07:33:36] PROBLEM - Puppet freshness on ssl3002 is CRITICAL: Puppet has not run in the last 10 hours [07:33:37] PROBLEM - Puppet freshness on ssl2 is CRITICAL: Puppet has not run in the last 10 hours [07:33:37] PROBLEM - Puppet freshness on ssl3004 is CRITICAL: Puppet has not run in the last 10 hours [07:57:28] !log nikerabbit synchronized php-1.18/extensions/WebFonts/resources/ext.webfonts.js 'JS fix r106418' [07:57:37] Logged the message, Master [08:44:18] RECOVERY - Recursive DNS on 91.198.174.6 is OK: DNS OK: 0.228 seconds response time. www.wikipedia.org returns 91.198.174.225 [09:05:08] PROBLEM - MySQL slave status on es1004 is CRITICAL: CRITICAL: Slave running: expected Yes, got No [09:07:08] PROBLEM - Puppet freshness on aluminium is CRITICAL: Puppet has not run in the last 10 hours [09:07:08] PROBLEM - Puppet freshness on db1001 is CRITICAL: Puppet has not run in the last 10 hours [09:07:08] PROBLEM - Puppet freshness on db1003 is CRITICAL: Puppet has not run in the last 10 hours [09:07:08] PROBLEM - Puppet freshness on db1005 is CRITICAL: Puppet has not run in the last 10 hours [09:07:08] PROBLEM - Puppet freshness on db1002 is CRITICAL: Puppet has not run in the last 10 hours [09:22:15] dumps.wikimedia.org is down … [09:22:58] what? [09:23:10] time to use the mirror [09:23:13] bad nagios, not telling us [09:24:12] fixed [09:24:31] !log restarted lighthttpd on ds2, it had stopped (and why didn't nagios tell us? ) [09:24:39] Logged the message, Master [09:42:48] PROBLEM - Puppet freshness on db1025 is CRITICAL: Puppet has not run in the last 10 hours [09:42:48] PROBLEM - Puppet freshness on db18 is CRITICAL: Puppet has not run in the last 10 hours [09:42:48] PROBLEM - Puppet freshness on db25 is CRITICAL: Puppet has not run in the last 10 hours [09:42:48] PROBLEM - Puppet freshness on mw1004 is CRITICAL: Puppet has not run in the last 10 hours [09:42:48] PROBLEM - Puppet freshness on erzurumi is CRITICAL: Puppet has not run in the last 10 hours [09:42:48] PROBLEM - Puppet freshness on mw1010 is CRITICAL: Puppet has not run in the last 10 hours [09:42:48] PROBLEM - Puppet freshness on mw1030 is CRITICAL: Puppet has not run in the last 10 hours [09:42:49] PROBLEM - Puppet freshness on mw1045 is CRITICAL: Puppet has not run in the last 10 hours [09:42:49] PROBLEM - Puppet freshness on mw1055 is CRITICAL: Puppet has not run in the last 10 hours [09:42:50] PROBLEM - Puppet freshness on mw1100 is CRITICAL: Puppet has not run in the last 10 hours [09:42:50] PROBLEM - Puppet freshness on mw5 is CRITICAL: Puppet has not run in the last 10 hours [09:42:51] PROBLEM - Puppet freshness on nfs1 is CRITICAL: Puppet has not run in the last 10 hours [09:42:51] PROBLEM - Puppet freshness on search15 is CRITICAL: Puppet has not run in the last 10 hours [09:42:52] PROBLEM - Puppet freshness on searchidx2 is CRITICAL: Puppet has not run in the last 10 hours [09:42:52] PROBLEM - Puppet freshness on srv269 is CRITICAL: Puppet has not run in the last 10 hours [09:42:53] PROBLEM - Puppet freshness on srv276 is CRITICAL: Puppet has not run in the last 10 hours [09:54:47] PROBLEM - Puppet freshness on mw1068 is CRITICAL: Puppet has not run in the last 10 hours [09:54:47] PROBLEM - Puppet freshness on mw1072 is CRITICAL: Puppet has not run in the last 10 hours [09:54:47] PROBLEM - Puppet freshness on mw1076 is CRITICAL: Puppet has not run in the last 10 hours [09:54:47] PROBLEM - Puppet freshness on mw1138 is CRITICAL: Puppet has not run in the last 10 hours [09:54:47] PROBLEM - Puppet freshness on mw36 is CRITICAL: Puppet has not run in the last 10 hours [09:54:48] PROBLEM - Puppet freshness on mw54 is CRITICAL: Puppet has not run in the last 10 hours [09:54:48] PROBLEM - Puppet freshness on search9 is CRITICAL: Puppet has not run in the last 10 hours [09:54:49] PROBLEM - Puppet freshness on srv211 is CRITICAL: Puppet has not run in the last 10 hours [09:54:49] PROBLEM - Puppet freshness on srv238 is CRITICAL: Puppet has not run in the last 10 hours [09:54:50] PROBLEM - Puppet freshness on virt2 is CRITICAL: Puppet has not run in the last 10 hours [09:58:58] PROBLEM - Puppet freshness on db1026 is CRITICAL: Puppet has not run in the last 10 hours [09:58:58] PROBLEM - Puppet freshness on db1012 is CRITICAL: Puppet has not run in the last 10 hours [09:58:58] PROBLEM - Puppet freshness on mw1025 is CRITICAL: Puppet has not run in the last 10 hours [09:58:58] PROBLEM - Puppet freshness on db1033 is CRITICAL: Puppet has not run in the last 10 hours [09:58:58] PROBLEM - Puppet freshness on mw1046 is CRITICAL: Puppet has not run in the last 10 hours [09:58:59] PROBLEM - Puppet freshness on mw11 is CRITICAL: Puppet has not run in the last 10 hours [09:58:59] PROBLEM - Puppet freshness on mw1113 is CRITICAL: Puppet has not run in the last 10 hours [09:59:00] PROBLEM - Puppet freshness on mw1127 is CRITICAL: Puppet has not run in the last 10 hours [09:59:00] PROBLEM - Puppet freshness on srv236 is CRITICAL: Puppet has not run in the last 10 hours [09:59:01] PROBLEM - Puppet freshness on mw1149 is CRITICAL: Puppet has not run in the last 10 hours [09:59:01] PROBLEM - Puppet freshness on mw42 is CRITICAL: Puppet has not run in the last 10 hours [09:59:02] PROBLEM - Puppet freshness on mw1133 is CRITICAL: Puppet has not run in the last 10 hours [09:59:02] PROBLEM - Puppet freshness on srv263 is CRITICAL: Puppet has not run in the last 10 hours [09:59:03] PROBLEM - Puppet freshness on srv256 is CRITICAL: Puppet has not run in the last 10 hours [10:00:57] PROBLEM - Puppet freshness on mw1064 is CRITICAL: Puppet has not run in the last 10 hours [10:00:57] PROBLEM - Puppet freshness on mw1151 is CRITICAL: Puppet has not run in the last 10 hours [10:00:57] PROBLEM - Puppet freshness on mw32 is CRITICAL: Puppet has not run in the last 10 hours [10:00:57] PROBLEM - Puppet freshness on sockpuppet is CRITICAL: Puppet has not run in the last 10 hours [10:04:27] PROBLEM - Puppet freshness on db1004 is CRITICAL: Puppet has not run in the last 10 hours [10:04:27] PROBLEM - Puppet freshness on db1031 is CRITICAL: Puppet has not run in the last 10 hours [10:04:27] PROBLEM - Puppet freshness on db10 is CRITICAL: Puppet has not run in the last 10 hours [10:04:27] PROBLEM - Puppet freshness on db1039 is CRITICAL: Puppet has not run in the last 10 hours [10:04:27] PROBLEM - Puppet freshness on db44 is CRITICAL: Puppet has not run in the last 10 hours [10:04:28] PROBLEM - Puppet freshness on db33 is CRITICAL: Puppet has not run in the last 10 hours [10:04:28] PROBLEM - Puppet freshness on grosley is CRITICAL: Puppet has not run in the last 10 hours [10:21:38] RECOVERY - MySQL slave status on es1004 is OK: OK: [10:30:27] PROBLEM - Puppet freshness on maerlant is CRITICAL: Puppet has not run in the last 10 hours [10:43:20] New patchset: Dzahn; "tweak nagios to relieve spence - external command_check_interval - event_broker_options" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1592 [10:47:30] New review: Dzahn; "external command check_interval was set to "-1" which is "as often as possible". (this would influen..." [operations/puppet] (production); V: 1 C: 1; - https://gerrit.wikimedia.org/r/1592 [10:49:21] New review: Dzahn; "external command check_interval was set to "-1" which is "as often as possible". (this would influen..." [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1592 [10:49:21] Change merged: Dzahn; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1592 [11:21:14] New patchset: Dzahn; "add techblog apache_site configs - rebased - move to /misc/blogs.pp" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1433 [11:21:28] PROBLEM - Puppet freshness on copper is CRITICAL: Puppet has not run in the last 10 hours [11:21:28] PROBLEM - Puppet freshness on zinc is CRITICAL: Puppet has not run in the last 10 hours [11:22:24] New patchset: Dzahn; "add techblog apache_site configs - rebased - move to /misc/blogs.pp" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1433 [11:23:22] New review: Dzahn; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1433 [11:23:22] Change merged: Dzahn; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1433 [11:28:57] New patchset: Dzahn; "move blog.wm to blogs.pp, together with techblog" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1593 [11:29:58] New patchset: Dzahn; "move blog.wm to blogs.pp, together with techblog" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1593 [11:32:06] New patchset: Dzahn; "move blog.wm to blogs.pp, together with techblog - remove duplicate apache_site in techblog after merging configs" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1593 [11:32:42] New review: Dzahn; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1593 [11:32:42] Change merged: Dzahn; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1593 [11:34:47] New patchset: Dzahn; "fix apache_site name of techblog" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1594 [11:35:05] New review: Dzahn; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1594 [11:35:06] Change merged: Dzahn; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1594 [11:39:28] PROBLEM - Puppet freshness on spence is CRITICAL: Puppet has not run in the last 10 hours [11:42:55] New patchset: Dzahn; "add ocs.wikimania2009 apache_site configs - and basic puppet class - move to wikimanias.pp (wikimania special sites)" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1439 [11:45:21] New review: Dzahn; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1439 [11:45:21] Change merged: Dzahn; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1439 [11:54:48] New patchset: Dzahn; "add wm09schols, wm10schols, wm10reg apache_site configs & basic puppet classes, rebase, move to wikimanias.pp" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1435 [12:07:36] New patchset: Dzahn; "wm09schols, wm10schols, wm10reg apache_sites,rebase, move to wikimanias.pp, nicer class name structure, fix admin email" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1435 [12:09:09] New review: Dzahn; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1435 [12:09:09] Change merged: Dzahn; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1435 [12:14:13] New patchset: Dzahn; "consistent ServerAdmin email, somebody said software@ -> noc@, really for all though?" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1595 [12:15:27] New patchset: Dzahn; "consistent ServerAdmin email, somebody said software@ -> noc@, really for all though?" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1595 [12:28:10] New patchset: Mark Bergsma; "First step in cleaning up site.pp: folding node entries" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1596 [12:34:36] New review: Mark Bergsma; "(no comment)" [operations/puppet] (production); V: 0 C: 1; - https://gerrit.wikimedia.org/r/1596 [12:39:30] New patchset: Dzahn; "merge contacts.wm into a single apache config as well, move from misc-servers to outreach.pp, fix tabs, admin email" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1597 [12:41:04] New review: Mark Bergsma; "(no comment)" [operations/puppet] (production); V: 0 C: 2; - https://gerrit.wikimedia.org/r/1595 [12:41:04] Change merged: Mark Bergsma; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1595 [12:41:18] New review: Dzahn; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1597 [12:41:19] Change merged: Dzahn; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1597 [12:43:23] New review: Mark Bergsma; "Please fix the class name in the system_role definition" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1597 [12:44:38] New review: Mark Bergsma; "Can you rename misc/wikimanis.pp to misc/wikimania.pp, consistent with the class naming?" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1435 [12:47:47] New patchset: Dzahn; "sure, fix class name in system_role and rename wikimanias.pp to wikimania.pp" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1598 [12:50:45] New review: Dzahn; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1598 [12:50:45] Change merged: Dzahn; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1598 [12:51:08] New review: Mark Bergsma; "Perhaps I'm missing something, but how is this supposed to work then? Is SSL enabled on that vhost, ..." [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1575 [12:57:14] New patchset: Dzahn; "add outreachcivi apache_site configs, merge into one file" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1432 [12:58:39] New patchset: Dzahn; "add outreachcivi apache_site configs, merge into one file, fix tabs" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1432 [12:59:16] New review: Dzahn; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1432 [12:59:17] Change merged: Dzahn; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1432 [13:04:09] New patchset: Dzahn; "add outreachcivi to outreach.pp" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1599 [13:04:58] New review: Dzahn; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1599 [13:04:58] Change merged: Dzahn; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1599 [13:15:07] New review: Mark Bergsma; "(no comment)" [operations/puppet] (production); V: 0 C: 2; - https://gerrit.wikimedia.org/r/1596 [13:15:08] Change merged: Mark Bergsma; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1596 [13:15:37] New patchset: Dzahn; "add url.wm apache_site config, fix tabs, email (do we still use url.wm?)" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1440 [13:28:55] !log tweaked Nagios earlier today: external command_check_interval & event_broker_options (see comments in gerrit Id3b4a458) [13:29:04] Logged the message, Master [13:29:13] New patchset: Mark Bergsma; "Enforce interface gro offloading in Puppet" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1600 [13:29:37] !log catrope synchronized php-1.18/extensions/ArticleFeedbackv5/ 'Updating to trunk state' [13:29:46] Logged the message, Master [13:30:35] New patchset: Mark Bergsma; "Enforce interface gro offloading in Puppet" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1600 [13:32:42] !log Dropping and recreating AFTv5 tables on en_labswikimedia and enwiki [13:32:50] Logged the message, Mr. Obvious [13:33:05] New patchset: Dzahn; "add secure.wm apache site_config + basic puppet class, add proxy and proxy_http modules, fix tabs, move to secure.pp instead" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1436 [13:39:01] New review: Dzahn; "(no comment)" [operations/puppet] (production); V: 0 C: 1; - https://gerrit.wikimedia.org/r/1600 [13:40:34] New patchset: Mark Bergsma; "Enforce interface gro offloading in Puppet" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1600 [13:41:19] New review: Mark Bergsma; "(no comment)" [operations/puppet] (production); V: 0 C: 2; - https://gerrit.wikimedia.org/r/1600 [13:41:19] Change merged: Mark Bergsma; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1600 [13:43:51] New patchset: Mark Bergsma; "Fix syntax error" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1601 [13:44:03] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1601 [13:44:03] New review: Mark Bergsma; "(no comment)" [operations/puppet] (production); V: 0 C: 2; - https://gerrit.wikimedia.org/r/1601 [13:44:03] Change merged: Mark Bergsma; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1601 [13:44:51] New review: Dzahn; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1436 [13:47:43] New patchset: Dzahn; "add secure.wm apache site_config + basic puppet class, add proxy and proxy_http modules, fix tabs, move to secure.pp instead" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1436 [13:47:54] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1436 [13:48:26] New patchset: Dzahn; "add secure.wm apache site_config + basic puppet class, add proxy and proxy_http modules, fix tabs, move to secure.pp instead" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1436 [13:48:38] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1436 [13:48:50] New patchset: Mark Bergsma; "Remove superfluous ethtool -k parameter" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1602 [13:49:03] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1602 [13:49:08] New review: Mark Bergsma; "(no comment)" [operations/puppet] (production); V: 0 C: 2; - https://gerrit.wikimedia.org/r/1602 [13:49:08] Change merged: Mark Bergsma; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1602 [13:52:05] New patchset: Mark Bergsma; "Escape the dollar sign" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1603 [13:52:16] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1603 [13:52:21] New review: Mark Bergsma; "(no comment)" [operations/puppet] (production); V: 0 C: 2; - https://gerrit.wikimedia.org/r/1603 [13:52:22] Change merged: Mark Bergsma; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1603 [13:53:25] New review: Dzahn; "want a generic (tool?) class that ensures the ethtool package or add it to base?" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1602 [13:56:59] New review: Dzahn; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1436 [13:56:59] Change merged: Dzahn; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1436 [14:04:34] Change abandoned: Dzahn; "nobody seems to miss it - or should speak up" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1440 [14:06:31] New patchset: Mark Bergsma; "Fold interface_setting into interface_offload" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1604 [14:06:40] New review: gerrit2; "Change did not pass lint check. You will need to send an amended patchset for this (see: https://lab..." [operations/puppet] (production); V: -1 - https://gerrit.wikimedia.org/r/1604 [14:07:35] New patchset: Mark Bergsma; "Fold interface_setting into interface_offload" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1604 [14:07:46] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1604 [14:08:46] New review: Mark Bergsma; "(no comment)" [operations/puppet] (production); V: 0 C: 2; - https://gerrit.wikimedia.org/r/1604 [14:08:46] Change merged: Mark Bergsma; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1604 [14:15:15] New patchset: Dzahn; "fully puppetize check_job_queue with custom check interval" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1473 [14:15:26] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1473 [14:16:07] New review: Dzahn; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1473 [14:16:07] Change merged: Dzahn; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1473 [14:19:59] New review: Demon; "(no comment)" [test/mediawiki/core] (master); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1574 [14:19:59] Change merged: Demon; [test/mediawiki/core] (master) - https://gerrit.wikimedia.org/r/1574 [14:27:17] PROBLEM - Puppet freshness on es1002 is CRITICAL: Puppet has not run in the last 10 hours [14:40:52] New patchset: Dzahn; "do not include nagios-plugins-basic here as it conflicts with class "nrpe" - real fix should be to have "nrpe" and "nrpe-light" or something, then" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1605 [14:42:31] New review: Dzahn; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1605 [14:42:31] Change merged: Dzahn; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1605 [14:44:47] RECOVERY - Puppet freshness on sodium is OK: puppet ran at Fri Dec 16 14:43:50 UTC 2011 [14:44:48] RECOVERY - Puppet freshness on srv227 is OK: puppet ran at Fri Dec 16 14:43:52 UTC 2011 [14:44:48] RECOVERY - Puppet freshness on srv237 is OK: puppet ran at Fri Dec 16 14:43:53 UTC 2011 [14:44:48] RECOVERY - Puppet freshness on mw38 is OK: puppet ran at Fri Dec 16 14:43:57 UTC 2011 [14:44:48] RECOVERY - Puppet freshness on mw25 is OK: puppet ran at Fri Dec 16 14:43:58 UTC 2011 [14:44:48] RECOVERY - Puppet freshness on srv215 is OK: puppet ran at Fri Dec 16 14:44:02 UTC 2011 [14:44:48] RECOVERY - Puppet freshness on db1002 is OK: puppet ran at Fri Dec 16 14:44:02 UTC 2011 [14:44:49] RECOVERY - Puppet freshness on srv279 is OK: puppet ran at Fri Dec 16 14:44:04 UTC 2011 [14:44:49] RECOVERY - Puppet freshness on mw1088 is OK: puppet ran at Fri Dec 16 14:44:11 UTC 2011 [14:44:50] RECOVERY - Puppet freshness on mw32 is OK: puppet ran at Fri Dec 16 14:44:14 UTC 2011 [14:44:50] RECOVERY - Puppet freshness on locke is OK: puppet ran at Fri Dec 16 14:44:15 UTC 2011 [14:44:51] RECOVERY - Puppet freshness on mw3 is OK: puppet ran at Fri Dec 16 14:44:17 UTC 2011 [14:44:51] RECOVERY - Puppet freshness on mw21 is OK: puppet ran at Fri Dec 16 14:44:30 UTC 2011 [14:44:52] RECOVERY - Puppet freshness on db49 is OK: puppet ran at Fri Dec 16 14:44:31 UTC 2011 [14:44:52] RECOVERY - Puppet freshness on srv297 is OK: puppet ran at Fri Dec 16 14:44:38 UTC 2011 [14:44:53] RECOVERY - Puppet freshness on mw4 is OK: puppet ran at Fri Dec 16 14:44:38 UTC 2011 [14:44:53] RECOVERY - Puppet freshness on grosley is OK: puppet ran at Fri Dec 16 14:44:40 UTC 2011 [14:44:54] RECOVERY - Puppet freshness on mw47 is OK: puppet ran at Fri Dec 16 14:44:44 UTC 2011 [14:45:47] RECOVERY - Puppet freshness on srv187 is OK: puppet ran at Fri Dec 16 14:44:48 UTC 2011 [14:45:48] RECOVERY - Puppet freshness on srv189 is OK: puppet ran at Fri Dec 16 14:44:55 UTC 2011 [14:45:48] RECOVERY - Puppet freshness on db47 is OK: puppet ran at Fri Dec 16 14:45:01 UTC 2011 [14:45:48] RECOVERY - Puppet freshness on srv298 is OK: puppet ran at Fri Dec 16 14:45:12 UTC 2011 [14:45:48] RECOVERY - Puppet freshness on mw1006 is OK: puppet ran at Fri Dec 16 14:45:16 UTC 2011 [14:45:48] RECOVERY - Puppet freshness on db27 is OK: puppet ran at Fri Dec 16 14:45:19 UTC 2011 [14:45:48] RECOVERY - Puppet freshness on es1004 is OK: puppet ran at Fri Dec 16 14:45:25 UTC 2011 [14:45:49] RECOVERY - Puppet freshness on db1022 is OK: puppet ran at Fri Dec 16 14:45:26 UTC 2011 [14:45:49] RECOVERY - Puppet freshness on db1044 is OK: puppet ran at Fri Dec 16 14:45:27 UTC 2011 [14:45:50] RECOVERY - Puppet freshness on snapshot2 is OK: puppet ran at Fri Dec 16 14:45:27 UTC 2011 [14:45:50] RECOVERY - Puppet freshness on mw30 is OK: puppet ran at Fri Dec 16 14:45:41 UTC 2011 [14:46:17] PROBLEM - Puppet freshness on bast1001 is CRITICAL: Puppet has not run in the last 10 hours [14:46:27] RECOVERY - Puppet freshness on srv272 is OK: puppet ran at Fri Dec 16 14:45:49 UTC 2011 [14:46:28] RECOVERY - Puppet freshness on mw1043 is OK: puppet ran at Fri Dec 16 14:45:55 UTC 2011 [14:46:28] RECOVERY - Puppet freshness on mw14 is OK: puppet ran at Fri Dec 16 14:45:55 UTC 2011 [14:46:28] RECOVERY - Puppet freshness on mw1085 is OK: puppet ran at Fri Dec 16 14:45:58 UTC 2011 [14:46:28] RECOVERY - Puppet freshness on srv256 is OK: puppet ran at Fri Dec 16 14:45:59 UTC 2011 [14:46:28] RECOVERY - Puppet freshness on db21 is OK: puppet ran at Fri Dec 16 14:46:06 UTC 2011 [14:46:28] RECOVERY - Puppet freshness on mw5 is OK: puppet ran at Fri Dec 16 14:46:08 UTC 2011 [14:46:29] RECOVERY - Puppet freshness on mw35 is OK: puppet ran at Fri Dec 16 14:46:08 UTC 2011 [14:46:37] RECOVERY - Puppet freshness on search7 is OK: puppet ran at Fri Dec 16 14:46:17 UTC 2011 [14:46:38] RECOVERY - Puppet freshness on mw6 is OK: puppet ran at Fri Dec 16 14:46:20 UTC 2011 [14:46:47] RECOVERY - Puppet freshness on mw1 is OK: puppet ran at Fri Dec 16 14:46:27 UTC 2011 [14:46:48] RECOVERY - Puppet freshness on srv241 is OK: puppet ran at Fri Dec 16 14:46:27 UTC 2011 [14:46:48] RECOVERY - Puppet freshness on srv207 is OK: puppet ran at Fri Dec 16 14:46:33 UTC 2011 [14:46:48] RECOVERY - Puppet freshness on mw43 is OK: puppet ran at Fri Dec 16 14:46:38 UTC 2011 [14:46:48] RECOVERY - Puppet freshness on srv273 is OK: puppet ran at Fri Dec 16 14:46:43 UTC 2011 [14:46:48] RECOVERY - Puppet freshness on db35 is OK: puppet ran at Fri Dec 16 14:46:43 UTC 2011 [14:46:48] RECOVERY - Puppet freshness on mw1052 is OK: puppet ran at Fri Dec 16 14:46:46 UTC 2011 [14:46:49] RECOVERY - Puppet freshness on db1039 is OK: puppet ran at Fri Dec 16 14:46:46 UTC 2011 [14:47:48] RECOVERY - Puppet freshness on mw1093 is OK: puppet ran at Fri Dec 16 14:46:48 UTC 2011 [14:47:48] RECOVERY - Puppet freshness on mw1121 is OK: puppet ran at Fri Dec 16 14:46:55 UTC 2011 [14:47:48] RECOVERY - Puppet freshness on mw1072 is OK: puppet ran at Fri Dec 16 14:46:56 UTC 2011 [14:47:48] RECOVERY - Puppet freshness on srv244 is OK: puppet ran at Fri Dec 16 14:47:18 UTC 2011 [14:47:48] RECOVERY - Puppet freshness on mw19 is OK: puppet ran at Fri Dec 16 14:47:22 UTC 2011 [14:47:48] RECOVERY - Puppet freshness on db1048 is OK: puppet ran at Fri Dec 16 14:47:27 UTC 2011 [14:47:48] RECOVERY - Puppet freshness on srv221 is OK: puppet ran at Fri Dec 16 14:47:27 UTC 2011 [14:47:49] RECOVERY - Puppet freshness on srv286 is OK: puppet ran at Fri Dec 16 14:47:35 UTC 2011 [14:47:49] RECOVERY - Puppet freshness on mw1131 is OK: puppet ran at Fri Dec 16 14:47:39 UTC 2011 [14:47:50] RECOVERY - Puppet freshness on gilman is OK: puppet ran at Fri Dec 16 14:47:39 UTC 2011 [14:47:50] RECOVERY - Puppet freshness on db1030 is OK: puppet ran at Fri Dec 16 14:47:47 UTC 2011 [14:48:17] PROBLEM - Puppet freshness on fenari is CRITICAL: Puppet has not run in the last 10 hours [14:48:27] RECOVERY - Puppet freshness on mw1063 is OK: puppet ran at Fri Dec 16 14:47:49 UTC 2011 [14:48:27] RECOVERY - Puppet freshness on srv236 is OK: puppet ran at Fri Dec 16 14:47:51 UTC 2011 [14:48:27] RECOVERY - Puppet freshness on mw9 is OK: puppet ran at Fri Dec 16 14:47:56 UTC 2011 [14:48:27] RECOVERY - Puppet freshness on snapshot1 is OK: puppet ran at Fri Dec 16 14:47:56 UTC 2011 [14:48:27] RECOVERY - Puppet freshness on srv220 is OK: puppet ran at Fri Dec 16 14:47:59 UTC 2011 [14:48:28] RECOVERY - Puppet freshness on srv288 is OK: puppet ran at Fri Dec 16 14:48:04 UTC 2011 [14:48:28] RECOVERY - Puppet freshness on snapshot3 is OK: puppet ran at Fri Dec 16 14:48:10 UTC 2011 [14:48:29] RECOVERY - Puppet freshness on mw55 is OK: puppet ran at Fri Dec 16 14:48:12 UTC 2011 [14:48:37] RECOVERY - Puppet freshness on db1038 is OK: puppet ran at Fri Dec 16 14:48:18 UTC 2011 [14:48:37] RECOVERY - Puppet freshness on search19 is OK: puppet ran at Fri Dec 16 14:48:18 UTC 2011 [14:48:37] RECOVERY - Puppet freshness on srv199 is OK: puppet ran at Fri Dec 16 14:48:18 UTC 2011 [14:48:37] RECOVERY - Puppet freshness on srv282 is OK: puppet ran at Fri Dec 16 14:48:22 UTC 2011 [14:48:37] RECOVERY - Puppet freshness on mw42 is OK: puppet ran at Fri Dec 16 14:48:25 UTC 2011 [14:48:47] RECOVERY - Puppet freshness on mw1078 is OK: puppet ran at Fri Dec 16 14:48:32 UTC 2011 [14:48:48] RECOVERY - Puppet freshness on search18 is OK: puppet ran at Fri Dec 16 14:48:34 UTC 2011 [14:48:48] RECOVERY - Puppet freshness on srv251 is OK: puppet ran at Fri Dec 16 14:48:35 UTC 2011 [14:48:48] RECOVERY - Puppet freshness on srv213 is OK: puppet ran at Fri Dec 16 14:48:40 UTC 2011 [14:48:48] RECOVERY - Puppet freshness on mw1044 is OK: puppet ran at Fri Dec 16 14:48:43 UTC 2011 [14:49:47] RECOVERY - Puppet freshness on db1007 is OK: puppet ran at Fri Dec 16 14:48:50 UTC 2011 [14:49:48] RECOVERY - Puppet freshness on srv285 is OK: puppet ran at Fri Dec 16 14:48:50 UTC 2011 [14:49:48] RECOVERY - Puppet freshness on virt2 is OK: puppet ran at Fri Dec 16 14:49:02 UTC 2011 [14:49:48] RECOVERY - Puppet freshness on mw1146 is OK: puppet ran at Fri Dec 16 14:49:08 UTC 2011 [14:49:48] RECOVERY - Puppet freshness on db1008 is OK: puppet ran at Fri Dec 16 14:49:22 UTC 2011 [14:49:48] RECOVERY - Puppet freshness on mw56 is OK: puppet ran at Fri Dec 16 14:49:26 UTC 2011 [14:49:48] RECOVERY - Puppet freshness on mw1154 is OK: puppet ran at Fri Dec 16 14:49:27 UTC 2011 [14:49:49] RECOVERY - Puppet freshness on srv295 is OK: puppet ran at Fri Dec 16 14:49:43 UTC 2011 [14:49:49] RECOVERY - Puppet freshness on mw68 is OK: puppet ran at Fri Dec 16 14:49:45 UTC 2011 [14:50:47] RECOVERY - Puppet freshness on db1001 is OK: puppet ran at Fri Dec 16 14:49:48 UTC 2011 [14:50:48] RECOVERY - Puppet freshness on mw1124 is OK: puppet ran at Fri Dec 16 14:49:50 UTC 2011 [14:50:48] RECOVERY - Puppet freshness on db1016 is OK: puppet ran at Fri Dec 16 14:49:51 UTC 2011 [14:50:48] RECOVERY - Puppet freshness on mw70 is OK: puppet ran at Fri Dec 16 14:49:54 UTC 2011 [14:50:48] RECOVERY - Puppet freshness on mw1034 is OK: puppet ran at Fri Dec 16 14:50:03 UTC 2011 [14:50:48] RECOVERY - Puppet freshness on mw61 is OK: puppet ran at Fri Dec 16 14:50:08 UTC 2011 [14:50:48] RECOVERY - Puppet freshness on srv226 is OK: puppet ran at Fri Dec 16 14:50:13 UTC 2011 [14:50:49] RECOVERY - Puppet freshness on mw1029 is OK: puppet ran at Fri Dec 16 14:50:33 UTC 2011 [14:50:49] RECOVERY - Puppet freshness on db18 is OK: puppet ran at Fri Dec 16 14:50:44 UTC 2011 [14:50:50] RECOVERY - Puppet freshness on db46 is OK: puppet ran at Fri Dec 16 14:50:45 UTC 2011 [14:51:47] RECOVERY - Puppet freshness on db1020 is OK: puppet ran at Fri Dec 16 14:50:47 UTC 2011 [14:51:48] RECOVERY - Puppet freshness on virt3 is OK: puppet ran at Fri Dec 16 14:50:49 UTC 2011 [14:51:48] RECOVERY - Puppet freshness on mw1116 is OK: puppet ran at Fri Dec 16 14:50:49 UTC 2011 [14:51:48] RECOVERY - Puppet freshness on srv269 is OK: puppet ran at Fri Dec 16 14:50:50 UTC 2011 [14:51:48] RECOVERY - Puppet freshness on search14 is OK: puppet ran at Fri Dec 16 14:50:54 UTC 2011 [14:53:47] RECOVERY - Puppet freshness on srv292 is OK: puppet ran at Fri Dec 16 14:52:48 UTC 2011 [14:53:48] RECOVERY - Puppet freshness on db36 is OK: puppet ran at Fri Dec 16 14:52:51 UTC 2011 [14:53:48] RECOVERY - Puppet freshness on db48 is OK: puppet ran at Fri Dec 16 14:52:52 UTC 2011 [14:53:48] RECOVERY - Puppet freshness on srv287 is OK: puppet ran at Fri Dec 16 14:52:53 UTC 2011 [14:53:48] RECOVERY - Puppet freshness on mw1003 is OK: puppet ran at Fri Dec 16 14:52:59 UTC 2011 [14:53:48] RECOVERY - Puppet freshness on mw1100 is OK: puppet ran at Fri Dec 16 14:53:16 UTC 2011 [14:53:48] RECOVERY - Puppet freshness on mw1130 is OK: puppet ran at Fri Dec 16 14:53:19 UTC 2011 [14:53:49] RECOVERY - Puppet freshness on db34 is OK: puppet ran at Fri Dec 16 14:53:21 UTC 2011 [14:53:49] RECOVERY - Puppet freshness on mw1105 is OK: puppet ran at Fri Dec 16 14:53:24 UTC 2011 [14:53:50] RECOVERY - Puppet freshness on mw1064 is OK: puppet ran at Fri Dec 16 14:53:28 UTC 2011 [14:53:50] RECOVERY - Puppet freshness on srv214 is OK: puppet ran at Fri Dec 16 14:53:29 UTC 2011 [14:53:51] RECOVERY - Puppet freshness on mw1028 is OK: puppet ran at Fri Dec 16 14:53:33 UTC 2011 [14:53:51] RECOVERY - Puppet freshness on db17 is OK: puppet ran at Fri Dec 16 14:53:41 UTC 2011 [14:53:52] RECOVERY - Puppet freshness on mw1014 is OK: puppet ran at Fri Dec 16 14:53:43 UTC 2011 [14:53:52] RECOVERY - Puppet freshness on mw15 is OK: puppet ran at Fri Dec 16 14:53:44 UTC 2011 [14:53:53] RECOVERY - Puppet freshness on db37 is OK: puppet ran at Fri Dec 16 14:53:45 UTC 2011 [14:54:47] RECOVERY - Puppet freshness on mw1007 is OK: puppet ran at Fri Dec 16 14:53:52 UTC 2011 [14:54:48] RECOVERY - Puppet freshness on mw1117 is OK: puppet ran at Fri Dec 16 14:53:56 UTC 2011 [14:54:48] RECOVERY - Puppet freshness on mw1032 is OK: puppet ran at Fri Dec 16 14:54:00 UTC 2011 [14:54:48] RECOVERY - Puppet freshness on es3 is OK: puppet ran at Fri Dec 16 14:54:03 UTC 2011 [14:54:48] RECOVERY - Puppet freshness on srv201 is OK: puppet ran at Fri Dec 16 14:54:05 UTC 2011 [14:54:48] RECOVERY - Puppet freshness on srv265 is OK: puppet ran at Fri Dec 16 14:54:08 UTC 2011 [14:54:48] RECOVERY - Puppet freshness on mw1015 is OK: puppet ran at Fri Dec 16 14:54:13 UTC 2011 [14:54:49] RECOVERY - Puppet freshness on mw1082 is OK: puppet ran at Fri Dec 16 14:54:18 UTC 2011 [14:54:49] RECOVERY - Puppet freshness on mw62 is OK: puppet ran at Fri Dec 16 14:54:21 UTC 2011 [14:54:50] RECOVERY - Puppet freshness on mw1035 is OK: puppet ran at Fri Dec 16 14:54:23 UTC 2011 [14:54:50] RECOVERY - Puppet freshness on mw1048 is OK: puppet ran at Fri Dec 16 14:54:24 UTC 2011 [14:54:51] RECOVERY - Puppet freshness on mw1040 is OK: puppet ran at Fri Dec 16 14:54:34 UTC 2011 [14:54:51] RECOVERY - Puppet freshness on mw1091 is OK: puppet ran at Fri Dec 16 14:54:39 UTC 2011 [14:54:52] RECOVERY - Puppet freshness on srv263 is OK: puppet ran at Fri Dec 16 14:54:40 UTC 2011 [14:54:52] RECOVERY - Puppet freshness on mw1125 is OK: puppet ran at Fri Dec 16 14:54:40 UTC 2011 [14:54:53] RECOVERY - Puppet freshness on srv254 is OK: puppet ran at Fri Dec 16 14:54:41 UTC 2011 [14:55:47] RECOVERY - Puppet freshness on mw29 is OK: puppet ran at Fri Dec 16 14:54:49 UTC 2011 [14:55:48] RECOVERY - Puppet freshness on srv219 is OK: puppet ran at Fri Dec 16 14:54:50 UTC 2011 [14:55:48] RECOVERY - Puppet freshness on mw39 is OK: puppet ran at Fri Dec 16 14:54:52 UTC 2011 [14:55:48] RECOVERY - Puppet freshness on srv196 is OK: puppet ran at Fri Dec 16 14:54:52 UTC 2011 [14:55:48] RECOVERY - Puppet freshness on db9 is OK: puppet ran at Fri Dec 16 14:54:55 UTC 2011 [14:55:48] RECOVERY - Puppet freshness on ms5 is OK: puppet ran at Fri Dec 16 14:55:02 UTC 2011 [14:55:48] RECOVERY - Puppet freshness on search4 is OK: puppet ran at Fri Dec 16 14:55:04 UTC 2011 [14:55:49] RECOVERY - Puppet freshness on srv293 is OK: puppet ran at Fri Dec 16 14:55:07 UTC 2011 [14:55:49] RECOVERY - Puppet freshness on mw1051 is OK: puppet ran at Fri Dec 16 14:55:15 UTC 2011 [14:55:50] RECOVERY - Puppet freshness on db1043 is OK: puppet ran at Fri Dec 16 14:55:19 UTC 2011 [14:55:50] RECOVERY - Puppet freshness on db1012 is OK: puppet ran at Fri Dec 16 14:55:24 UTC 2011 [14:55:51] RECOVERY - Puppet freshness on db42 is OK: puppet ran at Fri Dec 16 14:55:31 UTC 2011 [14:55:51] RECOVERY - Puppet freshness on mw1030 is OK: puppet ran at Fri Dec 16 14:55:39 UTC 2011 [14:56:48] RECOVERY - Puppet freshness on nfs2 is OK: puppet ran at Fri Dec 16 14:55:55 UTC 2011 [14:56:48] RECOVERY - Puppet freshness on srv255 is OK: puppet ran at Fri Dec 16 14:56:07 UTC 2011 [14:56:48] RECOVERY - Puppet freshness on mw1159 is OK: puppet ran at Fri Dec 16 14:56:09 UTC 2011 [14:56:48] RECOVERY - Puppet freshness on search9 is OK: puppet ran at Fri Dec 16 14:56:10 UTC 2011 [14:56:48] RECOVERY - Puppet freshness on mw1008 is OK: puppet ran at Fri Dec 16 14:56:10 UTC 2011 [14:56:48] RECOVERY - Puppet freshness on db16 is OK: puppet ran at Fri Dec 16 14:56:16 UTC 2011 [14:56:48] RECOVERY - Puppet freshness on mw1089 is OK: puppet ran at Fri Dec 16 14:56:17 UTC 2011 [14:56:49] RECOVERY - Puppet freshness on db1026 is OK: puppet ran at Fri Dec 16 14:56:24 UTC 2011 [14:56:49] RECOVERY - Puppet freshness on db1046 is OK: puppet ran at Fri Dec 16 14:56:30 UTC 2011 [14:56:50] RECOVERY - Puppet freshness on db1024 is OK: puppet ran at Fri Dec 16 14:56:34 UTC 2011 [14:56:50] RECOVERY - Puppet freshness on mw1141 is OK: puppet ran at Fri Dec 16 14:56:34 UTC 2011 [14:56:51] RECOVERY - Puppet freshness on aluminium is OK: puppet ran at Fri Dec 16 14:56:38 UTC 2011 [14:56:51] RECOVERY - Puppet freshness on mw1115 is OK: puppet ran at Fri Dec 16 14:56:40 UTC 2011 [14:56:52] RECOVERY - Puppet freshness on mw1140 is OK: puppet ran at Fri Dec 16 14:56:41 UTC 2011 [14:57:47] RECOVERY - Puppet freshness on srv195 is OK: puppet ran at Fri Dec 16 14:56:50 UTC 2011 [14:57:48] RECOVERY - Puppet freshness on mw1090 is OK: puppet ran at Fri Dec 16 14:56:58 UTC 2011 [14:57:48] RECOVERY - Puppet freshness on mw58 is OK: puppet ran at Fri Dec 16 14:57:00 UTC 2011 [14:57:48] RECOVERY - Puppet freshness on mw1060 is OK: puppet ran at Fri Dec 16 14:57:01 UTC 2011 [14:57:48] RECOVERY - Puppet freshness on db1017 is OK: puppet ran at Fri Dec 16 14:57:02 UTC 2011 [14:57:48] RECOVERY - Puppet freshness on mw1142 is OK: puppet ran at Fri Dec 16 14:57:11 UTC 2011 [14:57:48] RECOVERY - Puppet freshness on db1003 is OK: puppet ran at Fri Dec 16 14:57:11 UTC 2011 [14:57:49] RECOVERY - Puppet freshness on mw1001 is OK: puppet ran at Fri Dec 16 14:57:15 UTC 2011 [14:57:49] RECOVERY - Puppet freshness on mw1065 is OK: puppet ran at Fri Dec 16 14:57:15 UTC 2011 [14:57:50] RECOVERY - Puppet freshness on storage3 is OK: puppet ran at Fri Dec 16 14:57:22 UTC 2011 [14:57:50] RECOVERY - Puppet freshness on mw1024 is OK: puppet ran at Fri Dec 16 14:57:26 UTC 2011 [14:57:51] RECOVERY - Puppet freshness on mw63 is OK: puppet ran at Fri Dec 16 14:57:26 UTC 2011 [14:57:51] RECOVERY - Puppet freshness on mw1144 is OK: puppet ran at Fri Dec 16 14:57:27 UTC 2011 [14:57:52] RECOVERY - Puppet freshness on mw1004 is OK: puppet ran at Fri Dec 16 14:57:32 UTC 2011 [14:57:52] RECOVERY - Puppet freshness on srv240 is OK: puppet ran at Fri Dec 16 14:57:34 UTC 2011 [14:57:53] RECOVERY - Puppet freshness on srv280 is OK: puppet ran at Fri Dec 16 14:57:35 UTC 2011 [14:57:53] RECOVERY - Puppet freshness on mw45 is OK: puppet ran at Fri Dec 16 14:57:35 UTC 2011 [14:57:54] RECOVERY - Puppet freshness on db30 is OK: puppet ran at Fri Dec 16 14:57:40 UTC 2011 [14:57:54] RECOVERY - Puppet freshness on srv270 is OK: puppet ran at Fri Dec 16 14:57:46 UTC 2011 [14:58:47] RECOVERY - Puppet freshness on mw57 is OK: puppet ran at Fri Dec 16 14:57:47 UTC 2011 [14:58:48] RECOVERY - Puppet freshness on mw1155 is OK: puppet ran at Fri Dec 16 14:57:48 UTC 2011 [14:58:48] RECOVERY - Puppet freshness on db1009 is OK: puppet ran at Fri Dec 16 14:57:48 UTC 2011 [14:58:48] RECOVERY - Puppet freshness on srv262 is OK: puppet ran at Fri Dec 16 14:57:49 UTC 2011 [14:58:48] RECOVERY - Puppet freshness on mw1037 is OK: puppet ran at Fri Dec 16 14:57:52 UTC 2011 [14:58:48] RECOVERY - Puppet freshness on mw1104 is OK: puppet ran at Fri Dec 16 14:57:54 UTC 2011 [14:58:48] RECOVERY - Puppet freshness on srv198 is OK: puppet ran at Fri Dec 16 14:57:54 UTC 2011 [14:58:49] RECOVERY - Puppet freshness on mw1020 is OK: puppet ran at Fri Dec 16 14:57:57 UTC 2011 [14:58:49] RECOVERY - Puppet freshness on mw1057 is OK: puppet ran at Fri Dec 16 14:58:02 UTC 2011 [14:58:50] RECOVERY - Puppet freshness on mw1127 is OK: puppet ran at Fri Dec 16 14:58:08 UTC 2011 [14:58:50] RECOVERY - Puppet freshness on search3 is OK: puppet ran at Fri Dec 16 14:58:10 UTC 2011 [14:58:51] RECOVERY - Puppet freshness on srv228 is OK: puppet ran at Fri Dec 16 14:58:19 UTC 2011 [14:58:51] RECOVERY - Puppet freshness on mw1041 is OK: puppet ran at Fri Dec 16 14:58:19 UTC 2011 [14:58:52] RECOVERY - Puppet freshness on mw1126 is OK: puppet ran at Fri Dec 16 14:58:20 UTC 2011 [14:58:52] RECOVERY - Puppet freshness on mw27 is OK: puppet ran at Fri Dec 16 14:58:23 UTC 2011 [14:58:53] RECOVERY - Puppet freshness on db29 is OK: puppet ran at Fri Dec 16 14:58:38 UTC 2011 [14:58:53] RECOVERY - Puppet freshness on ms1004 is OK: puppet ran at Fri Dec 16 14:58:43 UTC 2011 [14:59:48] RECOVERY - Puppet freshness on mw1077 is OK: puppet ran at Fri Dec 16 14:59:13 UTC 2011 [14:59:48] RECOVERY - Puppet freshness on srv247 is OK: puppet ran at Fri Dec 16 14:59:27 UTC 2011 [14:59:48] RECOVERY - Puppet freshness on mw34 is OK: puppet ran at Fri Dec 16 14:59:28 UTC 2011 [14:59:48] RECOVERY - Puppet freshness on db1035 is OK: puppet ran at Fri Dec 16 14:59:32 UTC 2011 [14:59:48] RECOVERY - Puppet freshness on db40 is OK: puppet ran at Fri Dec 16 14:59:34 UTC 2011 [14:59:48] RECOVERY - Puppet freshness on mw1149 is OK: puppet ran at Fri Dec 16 14:59:34 UTC 2011 [14:59:48] RECOVERY - Puppet freshness on mw1076 is OK: puppet ran at Fri Dec 16 14:59:34 UTC 2011 [14:59:49] RECOVERY - Puppet freshness on mw1069 is OK: puppet ran at Fri Dec 16 14:59:40 UTC 2011 [14:59:49] RECOVERY - Puppet freshness on mw1156 is OK: puppet ran at Fri Dec 16 14:59:42 UTC 2011 [14:59:50] RECOVERY - Puppet freshness on mw10 is OK: puppet ran at Fri Dec 16 14:59:44 UTC 2011 [15:00:48] RECOVERY - Puppet freshness on mw18 is OK: puppet ran at Fri Dec 16 14:59:52 UTC 2011 [15:00:48] RECOVERY - Puppet freshness on db1042 is OK: puppet ran at Fri Dec 16 15:00:04 UTC 2011 [15:00:48] RECOVERY - Puppet freshness on srv271 is OK: puppet ran at Fri Dec 16 15:00:10 UTC 2011 [15:00:48] RECOVERY - Puppet freshness on es4 is OK: puppet ran at Fri Dec 16 15:00:11 UTC 2011 [15:00:48] RECOVERY - Puppet freshness on sockpuppet is OK: puppet ran at Fri Dec 16 15:00:13 UTC 2011 [15:00:48] RECOVERY - Puppet freshness on mw1129 is OK: puppet ran at Fri Dec 16 15:00:14 UTC 2011 [15:00:48] RECOVERY - Puppet freshness on mw8 is OK: puppet ran at Fri Dec 16 15:00:15 UTC 2011 [15:00:49] RECOVERY - Puppet freshness on mw33 is OK: puppet ran at Fri Dec 16 15:00:22 UTC 2011 [15:00:49] RECOVERY - Puppet freshness on mw28 is OK: puppet ran at Fri Dec 16 15:00:44 UTC 2011 [15:00:50] RECOVERY - Puppet freshness on mw52 is OK: puppet ran at Fri Dec 16 15:00:46 UTC 2011 [15:01:47] RECOVERY - Puppet freshness on nfs1 is OK: puppet ran at Fri Dec 16 15:00:47 UTC 2011 [15:01:48] RECOVERY - Puppet freshness on mw1002 is OK: puppet ran at Fri Dec 16 15:00:50 UTC 2011 [15:01:48] RECOVERY - Puppet freshness on es2 is OK: puppet ran at Fri Dec 16 15:00:52 UTC 2011 [15:01:48] RECOVERY - Puppet freshness on srv203 is OK: puppet ran at Fri Dec 16 15:00:57 UTC 2011 [15:01:48] RECOVERY - Puppet freshness on mw1068 is OK: puppet ran at Fri Dec 16 15:00:58 UTC 2011 [15:01:48] RECOVERY - Puppet freshness on mw1114 is OK: puppet ran at Fri Dec 16 15:01:01 UTC 2011 [15:01:48] RECOVERY - Puppet freshness on mw1123 is OK: puppet ran at Fri Dec 16 15:01:05 UTC 2011 [15:01:49] RECOVERY - Puppet freshness on mw11 is OK: puppet ran at Fri Dec 16 15:01:08 UTC 2011 [15:01:49] RECOVERY - Puppet freshness on srv258 is OK: puppet ran at Fri Dec 16 15:01:15 UTC 2011 [15:01:50] RECOVERY - Puppet freshness on srv276 is OK: puppet ran at Fri Dec 16 15:01:17 UTC 2011 [15:01:50] RECOVERY - Puppet freshness on mw1083 is OK: puppet ran at Fri Dec 16 15:01:18 UTC 2011 [15:01:51] RECOVERY - Puppet freshness on mw1135 is OK: puppet ran at Fri Dec 16 15:01:23 UTC 2011 [15:01:51] RECOVERY - Puppet freshness on srv277 is OK: puppet ran at Fri Dec 16 15:01:29 UTC 2011 [15:01:52] RECOVERY - Puppet freshness on db33 is OK: puppet ran at Fri Dec 16 15:01:32 UTC 2011 [15:01:52] RECOVERY - Puppet freshness on mw1073 is OK: puppet ran at Fri Dec 16 15:01:38 UTC 2011 [15:01:53] RECOVERY - Puppet freshness on db1014 is OK: puppet ran at Fri Dec 16 15:01:40 UTC 2011 [15:01:53] RECOVERY - Puppet freshness on mw1148 is OK: puppet ran at Fri Dec 16 15:01:42 UTC 2011 [15:02:48] RECOVERY - Puppet freshness on mw1097 is OK: puppet ran at Fri Dec 16 15:01:51 UTC 2011 [15:02:48] RECOVERY - Puppet freshness on mw1118 is OK: puppet ran at Fri Dec 16 15:01:56 UTC 2011 [15:02:48] RECOVERY - Puppet freshness on mw73 is OK: puppet ran at Fri Dec 16 15:02:01 UTC 2011 [15:02:48] RECOVERY - Puppet freshness on mw48 is OK: puppet ran at Fri Dec 16 15:02:02 UTC 2011 [15:02:48] RECOVERY - Puppet freshness on search20 is OK: puppet ran at Fri Dec 16 15:02:03 UTC 2011 [15:02:48] RECOVERY - Puppet freshness on mw17 is OK: puppet ran at Fri Dec 16 15:02:09 UTC 2011 [15:02:48] RECOVERY - Puppet freshness on mw1009 is OK: puppet ran at Fri Dec 16 15:02:10 UTC 2011 [15:02:49] RECOVERY - Puppet freshness on mw41 is OK: puppet ran at Fri Dec 16 15:02:11 UTC 2011 [15:02:49] RECOVERY - Puppet freshness on mw1098 is OK: puppet ran at Fri Dec 16 15:02:13 UTC 2011 [15:02:50] RECOVERY - Puppet freshness on mw67 is OK: puppet ran at Fri Dec 16 15:02:23 UTC 2011 [15:02:50] RECOVERY - Puppet freshness on srv192 is OK: puppet ran at Fri Dec 16 15:02:26 UTC 2011 [15:02:51] RECOVERY - Puppet freshness on db1025 is OK: puppet ran at Fri Dec 16 15:02:27 UTC 2011 [15:02:51] RECOVERY - Puppet freshness on mw66 is OK: puppet ran at Fri Dec 16 15:02:35 UTC 2011 [15:02:52] RECOVERY - Puppet freshness on mw1106 is OK: puppet ran at Fri Dec 16 15:02:39 UTC 2011 [15:03:17] does all wikis share the same what you see isn't what you get? i.e. preview before seeing the result? Is there a wiki with wysiswyg? [15:03:47] RECOVERY - Puppet freshness on db23 is OK: puppet ran at Fri Dec 16 15:02:51 UTC 2011 [15:03:48] RECOVERY - Puppet freshness on srv267 is OK: puppet ran at Fri Dec 16 15:02:55 UTC 2011 [15:03:48] RECOVERY - Puppet freshness on mw36 is OK: puppet ran at Fri Dec 16 15:03:08 UTC 2011 [15:03:48] RECOVERY - Puppet freshness on srv224 is OK: puppet ran at Fri Dec 16 15:03:10 UTC 2011 [15:03:48] RECOVERY - Puppet freshness on srv239 is OK: puppet ran at Fri Dec 16 15:03:14 UTC 2011 [15:03:48] RECOVERY - Puppet freshness on srv204 is OK: puppet ran at Fri Dec 16 15:03:17 UTC 2011 [15:03:48] RECOVERY - Puppet freshness on srv249 is OK: puppet ran at Fri Dec 16 15:03:17 UTC 2011 [15:03:49] RECOVERY - Puppet freshness on srv278 is OK: puppet ran at Fri Dec 16 15:03:17 UTC 2011 [15:03:49] RECOVERY - Puppet freshness on srv294 is OK: puppet ran at Fri Dec 16 15:03:22 UTC 2011 [15:03:50] RECOVERY - Puppet freshness on mw1096 is OK: puppet ran at Fri Dec 16 15:03:23 UTC 2011 [15:03:50] RECOVERY - Puppet freshness on db1031 is OK: puppet ran at Fri Dec 16 15:03:24 UTC 2011 [15:03:51] RECOVERY - Puppet freshness on mw1119 is OK: puppet ran at Fri Dec 16 15:03:37 UTC 2011 [15:03:51] RECOVERY - Puppet freshness on mw1134 is OK: puppet ran at Fri Dec 16 15:03:37 UTC 2011 [15:03:52] RECOVERY - Puppet freshness on mw1023 is OK: puppet ran at Fri Dec 16 15:03:38 UTC 2011 [15:03:52] RECOVERY - Puppet freshness on srv191 is OK: puppet ran at Fri Dec 16 15:03:39 UTC 2011 [15:03:53] RECOVERY - Puppet freshness on db39 is OK: puppet ran at Fri Dec 16 15:03:41 UTC 2011 [15:03:53] RECOVERY - Puppet freshness on db10 is OK: puppet ran at Fri Dec 16 15:03:42 UTC 2011 [15:03:54] RECOVERY - Puppet freshness on mw1084 is OK: puppet ran at Fri Dec 16 15:03:43 UTC 2011 [15:03:54] RECOVERY - Puppet freshness on search15 is OK: puppet ran at Fri Dec 16 15:03:45 UTC 2011 [15:04:47] RECOVERY - Puppet freshness on db1013 is OK: puppet ran at Fri Dec 16 15:03:50 UTC 2011 [15:04:48] RECOVERY - Puppet freshness on srv225 is OK: puppet ran at Fri Dec 16 15:03:50 UTC 2011 [15:04:48] RECOVERY - Puppet freshness on hume is OK: puppet ran at Fri Dec 16 15:03:53 UTC 2011 [15:04:48] RECOVERY - Puppet freshness on mw49 is OK: puppet ran at Fri Dec 16 15:03:54 UTC 2011 [15:04:48] RECOVERY - Puppet freshness on mw1111 is OK: puppet ran at Fri Dec 16 15:04:07 UTC 2011 [15:04:48] RECOVERY - Puppet freshness on mw1153 is OK: puppet ran at Fri Dec 16 15:04:15 UTC 2011 [15:04:48] RECOVERY - Puppet freshness on es1 is OK: puppet ran at Fri Dec 16 15:04:17 UTC 2011 [15:04:49] RECOVERY - Puppet freshness on search6 is OK: puppet ran at Fri Dec 16 15:04:25 UTC 2011 [15:04:49] RECOVERY - Puppet freshness on mw1132 is OK: puppet ran at Fri Dec 16 15:04:26 UTC 2011 [15:04:50] RECOVERY - Puppet freshness on srv243 is OK: puppet ran at Fri Dec 16 15:04:28 UTC 2011 [15:04:50] RECOVERY - Puppet freshness on virt4 is OK: puppet ran at Fri Dec 16 15:04:31 UTC 2011 [15:04:51] RECOVERY - Puppet freshness on db44 is OK: puppet ran at Fri Dec 16 15:04:31 UTC 2011 [15:04:51] RECOVERY - Puppet freshness on search16 is OK: puppet ran at Fri Dec 16 15:04:33 UTC 2011 [15:04:52] RECOVERY - Puppet freshness on srv202 is OK: puppet ran at Fri Dec 16 15:04:38 UTC 2011 [15:04:52] RECOVERY - Puppet freshness on mw12 is OK: puppet ran at Fri Dec 16 15:04:45 UTC 2011 [15:04:53] RECOVERY - Puppet freshness on mw1011 is OK: puppet ran at Fri Dec 16 15:04:47 UTC 2011 [15:05:48] RECOVERY - Puppet freshness on srv257 is OK: puppet ran at Fri Dec 16 15:04:48 UTC 2011 [15:05:48] RECOVERY - Puppet freshness on mw1160 is OK: puppet ran at Fri Dec 16 15:04:53 UTC 2011 [15:05:48] RECOVERY - Puppet freshness on db12 is OK: puppet ran at Fri Dec 16 15:04:55 UTC 2011 [15:05:48] RECOVERY - Puppet freshness on mw1075 is OK: puppet ran at Fri Dec 16 15:04:57 UTC 2011 [15:05:48] RECOVERY - Puppet freshness on mw1143 is OK: puppet ran at Fri Dec 16 15:04:59 UTC 2011 [15:05:48] RECOVERY - Puppet freshness on mw1062 is OK: puppet ran at Fri Dec 16 15:05:04 UTC 2011 [15:05:48] RECOVERY - Puppet freshness on srv248 is OK: puppet ran at Fri Dec 16 15:05:04 UTC 2011 [15:05:49] RECOVERY - Puppet freshness on mw1101 is OK: puppet ran at Fri Dec 16 15:05:12 UTC 2011 [15:05:49] RECOVERY - Puppet freshness on mw20 is OK: puppet ran at Fri Dec 16 15:05:16 UTC 2011 [15:05:50] RECOVERY - Puppet freshness on mw1059 is OK: puppet ran at Fri Dec 16 15:05:22 UTC 2011 [15:05:50] RECOVERY - Puppet freshness on searchidx2 is OK: puppet ran at Fri Dec 16 15:05:30 UTC 2011 [15:05:51] RECOVERY - Puppet freshness on mw59 is OK: puppet ran at Fri Dec 16 15:05:46 UTC 2011 [15:06:48] RECOVERY - Puppet freshness on db24 is OK: puppet ran at Fri Dec 16 15:05:49 UTC 2011 [15:06:48] RECOVERY - Puppet freshness on stafford is OK: puppet ran at Fri Dec 16 15:05:50 UTC 2011 [15:06:48] RECOVERY - Puppet freshness on db1019 is OK: puppet ran at Fri Dec 16 15:05:50 UTC 2011 [15:06:48] RECOVERY - Puppet freshness on srv238 is OK: puppet ran at Fri Dec 16 15:05:55 UTC 2011 [15:06:48] RECOVERY - Puppet freshness on srv233 is OK: puppet ran at Fri Dec 16 15:06:01 UTC 2011 [15:06:48] RECOVERY - Puppet freshness on mw44 is OK: puppet ran at Fri Dec 16 15:06:03 UTC 2011 [15:06:48] RECOVERY - Puppet freshness on professor is OK: puppet ran at Fri Dec 16 15:06:10 UTC 2011 [15:06:49] RECOVERY - Puppet freshness on srv299 is OK: puppet ran at Fri Dec 16 15:06:11 UTC 2011 [15:06:49] RECOVERY - Puppet freshness on srv188 is OK: puppet ran at Fri Dec 16 15:06:16 UTC 2011 [15:06:50] RECOVERY - Puppet freshness on mw13 is OK: puppet ran at Fri Dec 16 15:06:22 UTC 2011 [15:06:50] RECOVERY - Puppet freshness on mw72 is OK: puppet ran at Fri Dec 16 15:06:23 UTC 2011 [15:06:51] RECOVERY - Puppet freshness on db1010 is OK: puppet ran at Fri Dec 16 15:06:26 UTC 2011 [15:06:51] RECOVERY - Puppet freshness on mw1058 is OK: puppet ran at Fri Dec 16 15:06:29 UTC 2011 [15:06:52] RECOVERY - Puppet freshness on mw1158 is OK: puppet ran at Fri Dec 16 15:06:30 UTC 2011 [15:06:52] RECOVERY - Puppet freshness on srv290 is OK: puppet ran at Fri Dec 16 15:06:34 UTC 2011 [15:06:53] RECOVERY - Puppet freshness on srv268 is OK: puppet ran at Fri Dec 16 15:06:37 UTC 2011 [15:06:53] RECOVERY - Puppet freshness on srv234 is OK: puppet ran at Fri Dec 16 15:06:38 UTC 2011 [15:06:54] RECOVERY - Puppet freshness on mw1122 is OK: puppet ran at Fri Dec 16 15:06:41 UTC 2011 [15:07:47] RECOVERY - Puppet freshness on mw1128 is OK: puppet ran at Fri Dec 16 15:06:51 UTC 2011 [15:07:48] RECOVERY - Puppet freshness on db1004 is OK: puppet ran at Fri Dec 16 15:06:56 UTC 2011 [15:07:48] RECOVERY - Puppet freshness on srv274 is OK: puppet ran at Fri Dec 16 15:07:03 UTC 2011 [15:07:48] RECOVERY - Puppet freshness on mw1087 is OK: puppet ran at Fri Dec 16 15:07:05 UTC 2011 [15:07:48] RECOVERY - Puppet freshness on mw1013 is OK: puppet ran at Fri Dec 16 15:07:12 UTC 2011 [15:07:48] RECOVERY - Puppet freshness on mw1151 is OK: puppet ran at Fri Dec 16 15:07:18 UTC 2011 [15:07:48] RECOVERY - Puppet freshness on mw64 is OK: puppet ran at Fri Dec 16 15:07:20 UTC 2011 [15:07:49] RECOVERY - Puppet freshness on db1033 is OK: puppet ran at Fri Dec 16 15:07:26 UTC 2011 [15:07:49] RECOVERY - Puppet freshness on mw1147 is OK: puppet ran at Fri Dec 16 15:07:26 UTC 2011 [15:07:50] RECOVERY - Puppet freshness on mw1017 is OK: puppet ran at Fri Dec 16 15:07:36 UTC 2011 [15:07:50] RECOVERY - Puppet freshness on srv264 is OK: puppet ran at Fri Dec 16 15:07:39 UTC 2011 [15:07:51] RECOVERY - Puppet freshness on mw51 is OK: puppet ran at Fri Dec 16 15:07:46 UTC 2011 [15:07:51] RECOVERY - Puppet freshness on srv205 is OK: puppet ran at Fri Dec 16 15:07:46 UTC 2011 [15:08:48] RECOVERY - Puppet freshness on mw1074 is OK: puppet ran at Fri Dec 16 15:07:54 UTC 2011 [15:08:48] RECOVERY - Puppet freshness on srv209 is OK: puppet ran at Fri Dec 16 15:07:57 UTC 2011 [15:08:48] RECOVERY - Puppet freshness on mw2 is OK: puppet ran at Fri Dec 16 15:08:00 UTC 2011 [15:08:48] RECOVERY - Puppet freshness on mw1016 is OK: puppet ran at Fri Dec 16 15:08:00 UTC 2011 [15:08:48] RECOVERY - Puppet freshness on mw40 is OK: puppet ran at Fri Dec 16 15:08:13 UTC 2011 [15:08:48] RECOVERY - Puppet freshness on search12 is OK: puppet ran at Fri Dec 16 15:08:15 UTC 2011 [15:08:48] RECOVERY - Puppet freshness on srv291 is OK: puppet ran at Fri Dec 16 15:08:18 UTC 2011 [15:08:49] RECOVERY - Puppet freshness on db1040 is OK: puppet ran at Fri Dec 16 15:08:20 UTC 2011 [15:08:49] RECOVERY - Puppet freshness on mw1086 is OK: puppet ran at Fri Dec 16 15:08:20 UTC 2011 [15:08:50] RECOVERY - Puppet freshness on mw53 is OK: puppet ran at Fri Dec 16 15:08:21 UTC 2011 [15:08:50] RECOVERY - Puppet freshness on mw16 is OK: puppet ran at Fri Dec 16 15:08:22 UTC 2011 [15:08:51] RECOVERY - Puppet freshness on db1027 is OK: puppet ran at Fri Dec 16 15:08:26 UTC 2011 [15:08:51] RECOVERY - Puppet freshness on mw1136 is OK: puppet ran at Fri Dec 16 15:08:27 UTC 2011 [15:08:52] RECOVERY - Puppet freshness on emery is OK: puppet ran at Fri Dec 16 15:08:28 UTC 2011 [15:08:52] RECOVERY - Puppet freshness on db1028 is OK: puppet ran at Fri Dec 16 15:08:34 UTC 2011 [15:08:53] RECOVERY - Puppet freshness on db5 is OK: puppet ran at Fri Dec 16 15:08:37 UTC 2011 [15:08:53] RECOVERY - Puppet freshness on mw1145 is OK: puppet ran at Fri Dec 16 15:08:38 UTC 2011 [15:08:54] RECOVERY - Puppet freshness on db13 is OK: puppet ran at Fri Dec 16 15:08:39 UTC 2011 [15:09:48] RECOVERY - Puppet freshness on mw65 is OK: puppet ran at Fri Dec 16 15:08:48 UTC 2011 [15:09:48] RECOVERY - Puppet freshness on mw1070 is OK: puppet ran at Fri Dec 16 15:08:50 UTC 2011 [15:09:48] RECOVERY - Puppet freshness on mw1019 is OK: puppet ran at Fri Dec 16 15:08:52 UTC 2011 [15:09:48] RECOVERY - Puppet freshness on mw69 is OK: puppet ran at Fri Dec 16 15:08:53 UTC 2011 [15:09:48] RECOVERY - Puppet freshness on mw1067 is OK: puppet ran at Fri Dec 16 15:09:00 UTC 2011 [15:09:48] RECOVERY - Puppet freshness on srv283 is OK: puppet ran at Fri Dec 16 15:09:02 UTC 2011 [15:09:48] RECOVERY - Puppet freshness on mw1095 is OK: puppet ran at Fri Dec 16 15:09:04 UTC 2011 [15:09:49] RECOVERY - Puppet freshness on ms1002 is OK: puppet ran at Fri Dec 16 15:09:09 UTC 2011 [15:09:49] RECOVERY - Puppet freshness on mw1102 is OK: puppet ran at Fri Dec 16 15:09:14 UTC 2011 [15:09:50] RECOVERY - Puppet freshness on db1021 is OK: puppet ran at Fri Dec 16 15:09:18 UTC 2011 [15:09:50] RECOVERY - Puppet freshness on mw37 is OK: puppet ran at Fri Dec 16 15:09:19 UTC 2011 [15:09:51] RECOVERY - Puppet freshness on srv260 is OK: puppet ran at Fri Dec 16 15:09:22 UTC 2011 [15:09:51] RECOVERY - Puppet freshness on srv235 is OK: puppet ran at Fri Dec 16 15:09:29 UTC 2011 [15:09:52] RECOVERY - Puppet freshness on search13 is OK: puppet ran at Fri Dec 16 15:09:30 UTC 2011 [15:09:52] RECOVERY - Puppet freshness on srv252 is OK: puppet ran at Fri Dec 16 15:09:31 UTC 2011 [15:09:53] RECOVERY - Puppet freshness on srv190 is OK: puppet ran at Fri Dec 16 15:09:32 UTC 2011 [15:09:53] RECOVERY - Puppet freshness on mw24 is OK: puppet ran at Fri Dec 16 15:09:37 UTC 2011 [15:09:54] RECOVERY - Puppet freshness on search1 is OK: puppet ran at Fri Dec 16 15:09:38 UTC 2011 [15:09:54] RECOVERY - Puppet freshness on mw1112 is OK: puppet ran at Fri Dec 16 15:09:39 UTC 2011 [15:09:55] RECOVERY - Puppet freshness on mw1050 is OK: puppet ran at Fri Dec 16 15:09:39 UTC 2011 [15:09:55] RECOVERY - Puppet freshness on es1001 is OK: puppet ran at Fri Dec 16 15:09:41 UTC 2011 [15:10:48] RECOVERY - Puppet freshness on srv212 is OK: puppet ran at Fri Dec 16 15:09:47 UTC 2011 [15:10:48] RECOVERY - Puppet freshness on mw1047 is OK: puppet ran at Fri Dec 16 15:09:55 UTC 2011 [15:10:48] RECOVERY - Puppet freshness on db8 is OK: puppet ran at Fri Dec 16 15:09:56 UTC 2011 [15:10:48] RECOVERY - Puppet freshness on db32 is OK: puppet ran at Fri Dec 16 15:10:01 UTC 2011 [15:10:48] RECOVERY - Puppet freshness on srv275 is OK: puppet ran at Fri Dec 16 15:10:03 UTC 2011 [15:10:48] RECOVERY - Puppet freshness on mw1005 is OK: puppet ran at Fri Dec 16 15:10:04 UTC 2011 [15:10:48] RECOVERY - Puppet freshness on db11 is OK: puppet ran at Fri Dec 16 15:10:14 UTC 2011 [15:10:49] RECOVERY - Puppet freshness on srv229 is OK: puppet ran at Fri Dec 16 15:10:16 UTC 2011 [15:10:49] RECOVERY - Puppet freshness on mw1031 is OK: puppet ran at Fri Dec 16 15:10:17 UTC 2011 [15:10:50] RECOVERY - Puppet freshness on mw1025 is OK: puppet ran at Fri Dec 16 15:10:25 UTC 2011 [15:10:50] RECOVERY - Puppet freshness on mw7 is OK: puppet ran at Fri Dec 16 15:10:29 UTC 2011 [15:10:51] RECOVERY - Puppet freshness on srv194 is OK: puppet ran at Fri Dec 16 15:10:29 UTC 2011 [15:10:51] RECOVERY - Puppet freshness on db1006 is OK: puppet ran at Fri Dec 16 15:10:44 UTC 2011 [15:10:58] RECOVERY - Puppet freshness on mw1152 is OK: puppet ran at Fri Dec 16 15:10:47 UTC 2011 [15:11:08] RECOVERY - Puppet freshness on db1045 is OK: puppet ran at Fri Dec 16 15:10:48 UTC 2011 [15:11:08] RECOVERY - Puppet freshness on mw1138 is OK: puppet ran at Fri Dec 16 15:10:49 UTC 2011 [15:11:08] RECOVERY - Puppet freshness on mw1049 is OK: puppet ran at Fri Dec 16 15:10:53 UTC 2011 [15:11:08] RECOVERY - Puppet freshness on srv218 is OK: puppet ran at Fri Dec 16 15:10:55 UTC 2011 [15:11:18] RECOVERY - Puppet freshness on mw1055 is OK: puppet ran at Fri Dec 16 15:11:04 UTC 2011 [15:11:18] RECOVERY - Puppet freshness on db1041 is OK: puppet ran at Fri Dec 16 15:11:05 UTC 2011 [15:11:27] RECOVERY - Puppet freshness on mw1036 is OK: puppet ran at Fri Dec 16 15:11:08 UTC 2011 [15:11:28] RECOVERY - Puppet freshness on srv245 is OK: puppet ran at Fri Dec 16 15:11:09 UTC 2011 [15:11:28] RECOVERY - Puppet freshness on srv289 is OK: puppet ran at Fri Dec 16 15:11:10 UTC 2011 [15:11:28] RECOVERY - Puppet freshness on srv197 is OK: puppet ran at Fri Dec 16 15:11:15 UTC 2011 [15:11:28] RECOVERY - Puppet freshness on db1029 is OK: puppet ran at Fri Dec 16 15:11:15 UTC 2011 [15:11:28] RECOVERY - Puppet freshness on es1003 is OK: puppet ran at Fri Dec 16 15:11:16 UTC 2011 [15:11:38] RECOVERY - Puppet freshness on srv223 is OK: puppet ran at Fri Dec 16 15:11:18 UTC 2011 [15:11:38] RECOVERY - Puppet freshness on db26 is OK: puppet ran at Fri Dec 16 15:11:19 UTC 2011 [15:11:38] RECOVERY - Puppet freshness on search11 is OK: puppet ran at Fri Dec 16 15:11:20 UTC 2011 [15:11:38] RECOVERY - Puppet freshness on srv208 is OK: puppet ran at Fri Dec 16 15:11:23 UTC 2011 [15:11:48] RECOVERY - Puppet freshness on mw1053 is OK: puppet ran at Fri Dec 16 15:11:33 UTC 2011 [15:11:48] RECOVERY - Puppet freshness on mw1157 is OK: puppet ran at Fri Dec 16 15:11:42 UTC 2011 [15:11:48] RECOVERY - Puppet freshness on db41 is OK: puppet ran at Fri Dec 16 15:11:46 UTC 2011 [15:12:17] RECOVERY - Puppet freshness on db1034 is OK: puppet ran at Fri Dec 16 15:11:47 UTC 2011 [15:12:18] RECOVERY - Puppet freshness on mw1042 is OK: puppet ran at Fri Dec 16 15:11:49 UTC 2011 [15:12:18] RECOVERY - Puppet freshness on db1018 is OK: puppet ran at Fri Dec 16 15:11:51 UTC 2011 [15:12:18] RECOVERY - Puppet freshness on search8 is OK: puppet ran at Fri Dec 16 15:11:57 UTC 2011 [15:12:18] RECOVERY - Puppet freshness on db1015 is OK: puppet ran at Fri Dec 16 15:12:04 UTC 2011 [15:12:27] RECOVERY - Puppet freshness on srv296 is OK: puppet ran at Fri Dec 16 15:12:07 UTC 2011 [15:12:28] RECOVERY - Puppet freshness on srv210 is OK: puppet ran at Fri Dec 16 15:12:09 UTC 2011 [15:12:28] RECOVERY - Puppet freshness on mw31 is OK: puppet ran at Fri Dec 16 15:12:15 UTC 2011 [15:12:28] RECOVERY - Puppet freshness on mw1022 is OK: puppet ran at Fri Dec 16 15:12:16 UTC 2011 [15:12:38] RECOVERY - Puppet freshness on mw1137 is OK: puppet ran at Fri Dec 16 15:12:23 UTC 2011 [15:12:38] RECOVERY - Puppet freshness on mw1080 is OK: puppet ran at Fri Dec 16 15:12:25 UTC 2011 [15:12:48] RECOVERY - Puppet freshness on mw1109 is OK: puppet ran at Fri Dec 16 15:12:27 UTC 2011 [15:12:48] RECOVERY - Puppet freshness on mw1150 is OK: puppet ran at Fri Dec 16 15:12:31 UTC 2011 [15:12:48] RECOVERY - Puppet freshness on srv301 is OK: puppet ran at Fri Dec 16 15:12:32 UTC 2011 [15:12:48] RECOVERY - Puppet freshness on mw1133 is OK: puppet ran at Fri Dec 16 15:12:41 UTC 2011 [15:12:48] RECOVERY - Puppet freshness on mw74 is OK: puppet ran at Fri Dec 16 15:12:45 UTC 2011 [15:13:48] RECOVERY - Puppet freshness on srv216 is OK: puppet ran at Fri Dec 16 15:12:53 UTC 2011 [15:13:48] RECOVERY - Puppet freshness on mw1071 is OK: puppet ran at Fri Dec 16 15:12:54 UTC 2011 [15:13:48] RECOVERY - Puppet freshness on srv211 is OK: puppet ran at Fri Dec 16 15:12:55 UTC 2011 [15:13:48] RECOVERY - Puppet freshness on mw1079 is OK: puppet ran at Fri Dec 16 15:12:55 UTC 2011 [15:13:48] RECOVERY - Puppet freshness on search10 is OK: puppet ran at Fri Dec 16 15:12:58 UTC 2011 [15:13:48] RECOVERY - Puppet freshness on srv300 is OK: puppet ran at Fri Dec 16 15:12:58 UTC 2011 [15:13:48] RECOVERY - Puppet freshness on search2 is OK: puppet ran at Fri Dec 16 15:13:00 UTC 2011 [15:13:49] RECOVERY - Puppet freshness on srv259 is OK: puppet ran at Fri Dec 16 15:13:01 UTC 2011 [15:13:49] RECOVERY - Puppet freshness on mw1045 is OK: puppet ran at Fri Dec 16 15:13:07 UTC 2011 [15:13:50] RECOVERY - Puppet freshness on mw1110 is OK: puppet ran at Fri Dec 16 15:13:12 UTC 2011 [15:13:50] RECOVERY - Puppet freshness on mw1012 is OK: puppet ran at Fri Dec 16 15:13:13 UTC 2011 [15:13:51] RECOVERY - Puppet freshness on srv246 is OK: puppet ran at Fri Dec 16 15:13:15 UTC 2011 [15:13:51] RECOVERY - Puppet freshness on db22 is OK: puppet ran at Fri Dec 16 15:13:21 UTC 2011 [15:13:52] RECOVERY - Puppet freshness on mw71 is OK: puppet ran at Fri Dec 16 15:13:29 UTC 2011 [15:13:52] RECOVERY - Puppet freshness on mw54 is OK: puppet ran at Fri Dec 16 15:13:39 UTC 2011 [15:17:08] PROBLEM - SSH on maerlant is CRITICAL: Server answer: [15:55:40] Change abandoned: Dzahn; "fixed meanwhile, already running for a while" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1453 [16:01:26] New review: Dzahn; "yes, generic::webserver::php5': ssl => 'true', but true about the certificate, lemme add that" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1575 [16:11:36] New patchset: Dzahn; "planet - use star.wmf ssl cert, move to own file, remove hard-coded IP,.." [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1606 [16:11:48] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1606 [16:16:24] !log thumb cleaner to bed for the night. definitely need an alarm clock for this... good thing it's only got about 4 days of backlog left [16:16:32] Logged the message, Master [16:42:53] New patchset: Dzahn; "blogs - consistent class names" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1607 [16:43:05] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1607 [17:53:17] PROBLEM - Puppet freshness on snapshot4 is CRITICAL: Puppet has not run in the last 10 hours [17:53:30] don't look at me [17:56:58] * mark looks at apergos [18:13:57] wasn't that a good ignore on my part? :-) [18:30:50] @add #wikimedia-labs [18:49:18] PROBLEM - Puppet freshness on ssl2 is CRITICAL: Puppet has not run in the last 10 hours [18:49:18] PROBLEM - Puppet freshness on ssl3002 is CRITICAL: Puppet has not run in the last 10 hours [18:49:18] PROBLEM - Puppet freshness on ssl3004 is CRITICAL: Puppet has not run in the last 10 hours [18:51:57] Ryan_Lane: Are you around ? [18:52:48] RECOVERY - Puppet freshness on ssl3002 is OK: puppet ran at Fri Dec 16 18:52:00 UTC 2011 [18:55:48] RECOVERY - Puppet freshness on ssl2 is OK: puppet ran at Fri Dec 16 18:55:22 UTC 2011 [18:56:26] Is there a IPv6-entry for upload.esams.wikimedia.org for some regions of the world defined? We have a mail in OTRS whose writen complains that he gets an IPv6-entry, but I can not reproduce this here [18:56:35] writen → writer [18:57:54] DaBPunkt: I will check for you [18:59:03] DaBPunkt: yes, upload is the one vip that hands out ipv6 on some occasions [19:00:03] mu574n9: yes/no. what's up? [19:00:09] LeslieCarr: May I ask on what occasions? And why is the webserver not listing on ipv6 then? [19:01:12] Ryan_Lane: Long time. [19:01:20] indeed [19:01:22] Ryan_Lane: I moved to a new city [19:01:26] cool [19:01:35] I am now working on a Devops role. [19:02:08] With cfengine3, now playing around with orchestra. [19:02:28] Got a chance to learn python for couple of months. [19:02:34] What's news here ? [19:02:48] RECOVERY - Puppet freshness on ssl3003 is OK: puppet ran at Fri Dec 16 19:02:30 UTC 2011 [19:03:11] well, labs is up [19:03:16] we're hanging out in #wikimedia-labs [19:03:51] DaBPunkt: if the dns server detects that you are in europe and coming in via ipv6 [19:04:17] the real question would be why upload isn't listening on ipv6 :) [19:05:02] I am there too. [19:07:59] LeslieCarr: Whom should I ask for that? Can you handle that, should I filll in a bugreport, have I to beg for a RT-Ticket? ;) [19:08:24] hehe, um, how about i email and ask mark - do you have an email of the ticket/bug report you can send me ? [19:08:59] LeslieCarr: #2011121610003398, but it is in german of course [19:09:39] DaBPunkt: i am a dumb american and only speak english… mark is dutch so might know enough german to get by [19:10:04] DaBPunkt: I actually don't have an otrs login, believe it or not [19:10:27] there's already a translation. [19:10:48] RECOVERY - Puppet freshness on ssl3004 is OK: puppet ran at Fri Dec 16 19:09:55 UTC 2011 [19:10:58] (it's been added as a note) [19:11:01] !log catrope synchronized wmf-config/InitialiseSettings.php 'Set AFTv4 lottery odds to 100% on en_labswikimedia' [19:11:04] LeslieCarr: I gues for the part "Let the webserver listen on IPv6" there is no mail needed, because it IS a problem if to offer a ipv6-adress, but do not use it :-) [19:11:10] Logged the message, Master [19:12:08] DaBPunkt: definitely [19:17:43] !log catrope synchronized php-1.18/resources/startup.js 'touch' [19:17:52] Logged the message, Master [19:38:46] New patchset: Lcarr; "Adding libswt into jdk bundle for android development" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1608 [19:38:59] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1608 [19:39:27] New review: Lcarr; "(no comment)" [operations/puppet] (production); V: 0 C: 2; - https://gerrit.wikimedia.org/r/1608 [19:39:27] Change merged: Lcarr; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1608 [19:52:36] New patchset: Bhartshorne; "pulling out some of the inheretence and default + override stuff in swift config due to puppet bug 6539" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1609 [19:52:48] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1609 [19:56:59] New review: Bhartshorne; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1609 [19:57:00] Change merged: Bhartshorne; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1609 [20:08:04] New patchset: Bhartshorne; "correcting copper and zinc's swift class" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1610 [20:08:16] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1610 [20:08:50] New review: Bhartshorne; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1610 [20:08:50] Change merged: Bhartshorne; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1610 [20:10:47] RECOVERY - Puppet freshness on copper is OK: puppet ran at Fri Dec 16 20:10:26 UTC 2011 [20:34:47] New patchset: Hashar; "move misc::contint::* to misc/contint.pp" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1611 [20:34:59] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1611 [20:36:47] RECOVERY - Puppet freshness on zinc is OK: puppet ran at Fri Dec 16 20:36:32 UTC 2011 [21:19:34] New patchset: Hashar; "basic class to install Android SDK prerequisites" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1612 [21:19:44] New review: gerrit2; "Change did not pass lint check. You will need to send an amended patchset for this (see: https://lab..." [operations/puppet] (production); V: -1 - https://gerrit.wikimedia.org/r/1612 [21:24:34] New patchset: Hashar; "basic class to install Android SDK prerequisites" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1612 [21:24:43] New review: gerrit2; "Change did not pass lint check. You will need to send an amended patchset for this (see: https://lab..." [operations/puppet] (production); V: -1 - https://gerrit.wikimedia.org/r/1612 [21:25:44] New patchset: Hashar; "basic class to install Android SDK prerequisites" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1612 [21:25:56] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1612 [21:46:18] PROBLEM - Puppet freshness on maerlant is CRITICAL: Puppet has not run in the last 10 hours [21:47:08] New patchset: Bhartshorne; "setting real auth tokens for pmtpa test cluster" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1613 [21:47:19] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1613 [22:02:38] bugzilla seems sluggish/broken to me [22:02:50] can't load https://bugzilla.wikimedia.org/query.cgi?format=advanced while logged in [22:02:57] in another browser it loads not logged in, but i can't log in [22:03:02] browser just spins forever [22:03:25] * derp offers brion a plate of bacon. [22:03:31] bacon makes everything better [22:03:48] brion, darling, any idea of IPv6 implementation for wiki? [22:04:01] RobH, can you give it a quick poke? check its db isn't overloaded maybe? thx [22:04:12] already working issue, its db9 [22:04:17] thx yay [22:04:20] talking in operations ;] [22:04:24] ah excellent [22:04:45] derp, as in ipv6 network access to wikipedia sites? i'm not familiar with the progress on that myself [22:04:59] ah ok [22:05:24] I can switch between ipv6 / ipv4 at home without a tunnel and it works fine :) [22:05:51] derp: It would be a start if the hosts which have already IPv6_entries in DNS would LISTEN to it ;) [22:05:59] :) [22:06:17] Right now, on ipv4 [22:06:26] brb, switching [22:07:23] there we go [22:07:24] :) [22:07:26] 2001:0:4137:9e76:343b:2c95:b9cf:a1d8 [22:08:28] ah ok [22:10:53] yep [22:13:48] oh our lvs and pyball systems don't yet support ipv6 [22:13:52] we need to upgrade them [22:14:46] :) [22:18:20] is the otrs still on db9 or is it moved already? [22:22:04] DaBPunkt: moved [22:22:08] so its still up ;] [22:22:10] ok, tnx [22:23:09] (maybe someone should put that in the "news"-box on the OTRS-startpage ;-) (or remove it complete)) [22:23:11] sadly, we are actually in the process of the 'migrate stuff off db9' project [22:34:35] bz is back in bisnessssss [22:41:29] New patchset: Hashar; "directory to host WikipediaMobile nighties" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1615 [22:41:42] New patchset: Hashar; "move misc::contint::* to misc/contint.pp" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1611 [22:41:55] New patchset: Hashar; "basic class to install Android SDK prerequisites" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1612 [22:42:00] hmm [22:42:06] not sure why they got pushed again [22:42:08] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1615 [22:42:08] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1611 [22:42:08] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1612 [22:44:33] hashar: gerrit is on db9 which was having problems [22:44:52] that would explain the issue with RT and bugzilla so [22:45:03] the above were resubmitted because I had to rebase my changes [22:45:19] (I think) [22:50:10] Hi guys - are you having any problems with bits.wikimedia.org??? I keep getting error messages when going through WP. [22:50:14] "http://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=jquery.ui.button%2Ccore%2Cdialog%2Cdraggable%2Cmouse%2Cposition%2Cresizable%2Cwidget&skin=monobook&version=20110919T161434Z&*:14" [22:50:47] Firefox tells me the script has either stopped responding entirely, or is running very slowly, and asks if I wish to disable it. [22:55:18] PROBLEM - Puppet freshness on spence is CRITICAL: Puppet has not run in the last 10 hours [23:03:37] gn8 folks