[00:00:46] http://xkcd.com/583/ [00:00:47] HAH [00:00:55] NCR [00:00:57] *CNR [00:05:47] !log added myself as a labsconsole sysop, added myself to the packaging project, created a "php" instance for building php-related packages [00:05:55] Logged the message, Master [00:32:47] ori-l: http://wikitech.wikimedia.org/index.php?title=How_to_deploy_code&action=historysubmit&diff=50062&oldid=50050 [00:34:49] Reedy: might be of use to you as well [00:35:54] feel free to move to a more permanent location if it's actually useful [00:36:05] kaldari: I don't scap often, but when I do scap, I scap to all sites ;) [00:58:53] PROBLEM - Puppet freshness on ms-be1006 is CRITICAL: Puppet has not run in the last 10 hours [00:58:53] PROBLEM - Puppet freshness on ms-be1009 is CRITICAL: Puppet has not run in the last 10 hours [01:07:47] !log on virt0: enabling mysql virtual log to debug "detached session" errors on nova [01:07:56] Logged the message, Master [01:08:20] s/virtual/general [01:39:58] PROBLEM - MySQL Slave Delay on db1025 is CRITICAL: CRIT replication delay 187 seconds [01:40:43] PROBLEM - MySQL Slave Delay on storage3 is CRITICAL: CRIT replication delay 228 seconds [01:48:04] PROBLEM - Misc_Db_Lag on storage3 is CRITICAL: CHECK MySQL REPLICATION - lag - CRITICAL - Seconds_Behind_Master : 671s [01:54:31] RECOVERY - MySQL Slave Delay on db1025 is OK: OK replication delay 9 seconds [01:58:16] RECOVERY - Misc_Db_Lag on storage3 is OK: CHECK MySQL REPLICATION - lag - OK - Seconds_Behind_Master : 12s [01:58:17] RECOVERY - MySQL Slave Delay on storage3 is OK: OK replication delay 14 seconds [02:04:53] PROBLEM - Puppet freshness on singer is CRITICAL: Puppet has not run in the last 10 hours [02:22:22] !log LocalisationUpdate completed (1.20wmf9) at Fri Aug 10 02:22:22 UTC 2012 [02:22:39] Logged the message, Master [02:39:58] !log LocalisationUpdate completed (1.20wmf8) at Fri Aug 10 02:39:57 UTC 2012 [02:40:08] Logged the message, Master [05:16:52] PROBLEM - Puppet freshness on spence is CRITICAL: Puppet has not run in the last 10 hours [05:26:56] PROBLEM - Puppet freshness on labstore1 is CRITICAL: Puppet has not run in the last 10 hours [05:30:50] PROBLEM - Puppet freshness on ms-be1007 is CRITICAL: Puppet has not run in the last 10 hours [05:30:50] PROBLEM - Puppet freshness on ms-be1011 is CRITICAL: Puppet has not run in the last 10 hours [05:30:50] PROBLEM - Puppet freshness on ms-be1010 is CRITICAL: Puppet has not run in the last 10 hours [06:03:37] Why is the thumbnail server distorting this image https://commons.wikimedia.org/wiki/File:WendyDavis.jpeg [06:07:53] PROBLEM - Puppet freshness on hume is CRITICAL: Puppet has not run in the last 10 hours [06:10:32] there's a few other images like that. lemme see if I can find the report in bugzilla, you should add your example to it [06:12:40] https://bugzilla.wikimedia.org/show_bug.cgi?id=24854 I think [06:13:27] if after reading the report you think that's not the same issue, go ahead and file a new report [06:14:42] mutante: ping [06:14:49] wikitech is kinda borked [06:14:51] > (cur | prev) 2012-08-10T00:27:32 Kaldari (Talk | contribs) (26,118 bytes) (<sectionlink>Step 2: get the code on fenari) (undo) [06:14:59] (see "sectionlink") [06:15:06] seems to link to the right place though [06:16:28] could someone pastebin the contents of fenari:/home/kaldari/bin/cleartestwiki ? (from http://wikitech.wikimedia.org/index.php?title=How_to_deploy_code&diff=50062&oldid=50050 ) [06:18:44] http://p.defau.lt/?agQ5epKjGutJw2_2xTFHTg [06:19:51] jeremyb: [06:20:24] apergos: danke [06:21:48] yw [07:42:55] PROBLEM - Puppet freshness on neon is CRITICAL: Puppet has not run in the last 10 hours [08:05:51] PROBLEM - Puppet freshness on ocg3 is CRITICAL: Puppet has not run in the last 10 hours [08:10:21] PROBLEM - HTTP on kaulen is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:11:33] RECOVERY - HTTP on kaulen is OK: HTTP OK HTTP/1.1 200 OK - 461 bytes in 0.001 seconds [08:45:56] PROBLEM - Puppet freshness on ms-be1003 is CRITICAL: Puppet has not run in the last 10 hours [09:00:55] PROBLEM - Puppet freshness on virt1001 is CRITICAL: Puppet has not run in the last 10 hours [09:15:55] PROBLEM - Puppet freshness on virt1002 is CRITICAL: Puppet has not run in the last 10 hours [09:30:54] PROBLEM - Puppet freshness on virt1003 is CRITICAL: Puppet has not run in the last 10 hours [10:59:52] PROBLEM - Puppet freshness on ms-be1006 is CRITICAL: Puppet has not run in the last 10 hours [10:59:52] PROBLEM - Puppet freshness on ms-be1009 is CRITICAL: Puppet has not run in the last 10 hours [11:44:51] PROBLEM - MySQL Slave Delay on db33 is CRITICAL: CRIT replication delay 182 seconds [11:45:00] PROBLEM - MySQL Replication Heartbeat on db33 is CRITICAL: CRIT replication delay 181 seconds [11:46:21] RECOVERY - MySQL Slave Delay on db33 is OK: OK replication delay 0 seconds [11:46:30] RECOVERY - MySQL Replication Heartbeat on db33 is OK: OK replication delay 0 seconds [12:05:52] PROBLEM - Puppet freshness on singer is CRITICAL: Puppet has not run in the last 10 hours [13:24:15] hi, I am trying to translate in French the description of the problem of August 6 [13:24:18] but given I am not very knowlegeable in network I wonder what is the "folded fiber segment" [13:29:47] Seb35: maybe guillom can help? [13:29:53] i also had no idea what that meant [13:30:47] I don't know either. But there's this handy free encyclopedia online, I forgot its name. [13:30:56] Bing [13:31:05] Right! That's it. [13:31:15] Oh, Encyclopedia Britannica [13:31:16] Wait, they stopped that :( [13:34:57] I wikipedized and googled but didn’t found what it is [13:35:34] I guess it is some segment inside the datacenter given the context but absolutely not sure [13:35:54] mark: ^ can you clarify what "folded fiber segment" means? [13:37:41] just a piece of fiber that was bent so much that no light went through [13:38:36] yay for minimum bend radiuses (radi?) [13:38:40] ok :) [13:39:40] Does that classify it as "fucked" then? :p [13:39:48] mark: some of the news mentioned mobile relied on API. that's totally wrong, right? [13:40:17] not really [13:40:20] the mobile apps do [13:40:34] * action=mobileview * [13:40:35] Returns data needed for mobile views [13:40:40] Reedy: for more than search? [13:40:40] https://www.mediawiki.org/wiki/Extension:MobileFrontend#action.3Dmobileview [13:40:55] Yeah, it's their action=parse [13:41:28] huh [13:42:16] * jeremyb thought the mobile apps were more closely tied to the mobile web sites [13:42:27] (e.g. en.m.wp.o) [13:42:48] anyway, danke [13:44:03] mmmm. 8.51GB bz2 file becomes a 37.0GB XML file [13:44:30] is that a surprise? [13:44:53] Not really [15:17:52] PROBLEM - Puppet freshness on spence is CRITICAL: Puppet has not run in the last 10 hours [15:27:55] PROBLEM - Puppet freshness on labstore1 is CRITICAL: Puppet has not run in the last 10 hours [15:31:58] PROBLEM - Puppet freshness on ms-be1010 is CRITICAL: Puppet has not run in the last 10 hours [15:31:58] PROBLEM - Puppet freshness on ms-be1007 is CRITICAL: Puppet has not run in the last 10 hours [15:31:58] PROBLEM - Puppet freshness on ms-be1011 is CRITICAL: Puppet has not run in the last 10 hours [16:08:51] PROBLEM - Puppet freshness on hume is CRITICAL: Puppet has not run in the last 10 hours [16:56:12] Request: GET http://en.wikipedia.org/wiki/Chronological_summary_of_the_2012_Summer_Olympics, from 10.64.0.131 via cp1014.eqiad.wmnet (squid/2.7.STABLE9) to () [16:56:12] Error: ERR_CANNOT_FORWARD, errno [No Error] at Fri, 10 Aug 2012 16:55:27 GMT [16:56:14] Who broke it? [16:56:29] (breaks intermittently) [16:56:32] Wait, it's back up now. [16:56:34] Huh/ [16:56:35] there was a problem with editing a second ago [16:56:35] nvm [16:56:36] why? [16:56:46] anyone else spotted the prob ? [16:57:00] It dumped me on a "Wikimedia Error" page. [16:57:16] Nathan2055: with which error? [16:57:16] I did get a loss of session data error a minute ago. [16:57:34] I didn't see... [16:57:43] (okay, now I look stupid) [16:58:31] are you all talking about enwiki? [16:58:34] Yes. [16:58:38] Hey, look [16:58:46] Copyed from -en-help [16:58:48] [12:55] knick I just tried editing another page and got an error message... I am a wiki newbie! [16:58:56] Not just me. [16:59:58] <^demon> jeremyb: I'm wondering if it's just a single server or two flapping. I can't seem to hit it with any regularity. [17:00:00] does anyone remember the variable for changing the extensions location? [17:00:46] Uh, guys. ukwiki service disruption was just noted. [17:01:11] Nathan2055: where? [17:01:17] can you guys get headers? [17:01:22] http://status.wikimedia.org [17:01:30] oh, yeah, that we saw, thanks [17:01:31] ukwiki is noted as being down [17:01:44] does anyone remember the variable for changing the extensions location? [17:02:14] OrenBo: can you try #mediawiki ? [17:02:20] Nathan2055: actually it's back if you check the graphs [17:02:34] This is the tech channel for Wikimedia wikis, OrenBo. [17:02:44] jeremyb: Huh, you're right. [17:04:15] thanks I got it [17:27:15] ukwiki is back up, mostly [17:43:57] PROBLEM - Puppet freshness on neon is CRITICAL: Puppet has not run in the last 10 hours [18:06:54] PROBLEM - Puppet freshness on ocg3 is CRITICAL: Puppet has not run in the last 10 hours [18:10:59] !log Payments cluster updated to 133b2b6fb5 [18:11:08] Logged the message, Master [18:46:58] PROBLEM - Puppet freshness on ms-be1003 is CRITICAL: Puppet has not run in the last 10 hours [19:01:58] PROBLEM - Puppet freshness on virt1001 is CRITICAL: Puppet has not run in the last 10 hours [19:16:57] PROBLEM - Puppet freshness on virt1002 is CRITICAL: Puppet has not run in the last 10 hours [19:31:57] PROBLEM - Puppet freshness on virt1003 is CRITICAL: Puppet has not run in the last 10 hours [21:00:54] PROBLEM - Puppet freshness on ms-be1006 is CRITICAL: Puppet has not run in the last 10 hours [21:00:54] PROBLEM - Puppet freshness on ms-be1009 is CRITICAL: Puppet has not run in the last 10 hours [21:20:43] !log preilly synchronized php-1.20wmf8/extensions/MobileFrontend 'update for zero' [21:20:54] Logged the message, Master [21:21:12] !log update to zero needed to support carrier testing [21:21:16] !log preilly synchronized php-1.20wmf9/extensions/MobileFrontend 'update for zero' [21:21:22] Logged the message, Master [21:21:30] !log update to zero needed to support carrier testing [21:21:31] Logged the message, Master [21:21:40] Logged the message, Master [21:22:23] !log preilly synchronized php-1.20wmf8/extensions/ZeroRatedMobileAccess 'update for landing page' [21:22:32] Logged the message, Master [21:22:58] !log preilly synchronized php-1.20wmf9/extensions/ZeroRatedMobileAccess 'update for landing page' [21:23:07] Logged the message, Master [22:06:55] PROBLEM - Puppet freshness on singer is CRITICAL: Puppet has not run in the last 10 hours [23:46:01] Hello, What is Wikimedia? [23:46:58] something not worth waiting a few sec for an answer apparently