[00:15:50] what is the URL to purge? [00:16:44] i guess http://bits.wikimedia.org/be.wikimedia.org/load.php?debug=false&lang=en&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=20120423T214913Z [00:16:59] but i think it's not actually cached. (the headers say miss) [00:18:04] example broken page: http://be.wikimedia.org/wiki/Special:Contributions/Bradford [00:29:50] the documented command does not appear to work [02:13:59] !log LocalisationUpdate completed (1.20wmf1) at Mon Apr 30 02:13:59 UTC 2012 [02:14:04] Logged the message, Master [16:51:20] robla: branching 1.20wmf2 today? [16:53:01] chrismcmahon: that's the plan. Reedy? [16:53:17] You tell me ;)) [16:53:40] plsbranchkthxbai :) [16:53:45] I suppose we should have a quick triage of unreviewed revisions, and see if there's anything that we should merge with first [16:55:01] <^demon> Also double check anything that's been committed to 1.20wmf1 (we should have a tool to automate this check...) so we know if needs backporting to master still, etc. [16:55:20] <^demon> I know there were some dumps hacks that snuck in. Need to make sure they're in master or ready to be re-applied. [16:56:05] https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/core.git;a=log;h=refs/heads/wmf/1.20wmf1 [16:59:34] Not too many revs to look at [17:00:45] I'm AFK for dinner in afew mins [17:02:08] Reedy: did you want to hand off the branching exercise? [17:02:43] There's no reason to do so, unless we're in a rush [17:02:56] I'll be back within an hour or so, then about all evening [17:03:31] Certainly when it's branched, help finding revisions we need to deal with would be good [17:03:44] Same if people want to merge anything back into trunk before I merge [17:03:53] not a huge rush, I guess. I would like to have the deployment happen during the deployment window [17:05:08] So it's due to start in an hour then? [17:05:17] Sounds like the timing is about right [17:05:20] yup [17:05:32] * robla goes onto call now [17:25:41] alright, no scaping for a while plz :) [17:28:18] csteipp, roblaBUSY - is it ok to add bad user stories to the OAuth draft? Like things that are already happening and shouldn't (and add a new "malicious dev" character to the cast)? [17:28:33] DarTar: sure [17:28:37] cool [17:29:07] DarTar: Yup, please! Would love to make sure we get bad stuff fixed. [17:29:18] ok I'll start adding some stuff today [17:43:12] hrm [17:43:14] wiki down? [17:43:16] PHP fatal error in /usr/local/apache/common-local/wmf-config/CommonSettings.php line 2403: [17:43:17] require() [function.require]: Failed opening required '/usr/local/apache/common-local/php-1.20wmf1/../wmf-config/ExtensionMessages-1.20wmf1-*.php' (include_path='/usr/local/apache/common-local/php-1.20wmf1/extensions/OggHandler/PEAR/File_Ogg:/usr/local/apache/common-local/php-1.20wmf1:/usr/local/apache/common-local/php-1.20wmf1/lib:/usr/local/lib/php:/usr/share/php') [17:43:18] yes [17:43:21] I see [17:43:22] Who pulled the plug [17:43:23] ? [17:43:29] !log updating multiversion code [17:43:32] Logged the message, Master [17:43:35] * AaronSchulz reverts the cdb [17:44:15] !log aaron synchronized wikiversions.cdb [17:44:17] Logged the message, Master [17:44:32] and back up [17:44:42] thanks [17:45:24] german wikipedia is down [17:45:33] Still? [17:45:34] no longer :) [17:45:37] oh now it's up [17:45:39] heh [17:45:39] Reedy: will the magic word bug be solved/former patch added until 1.20wmf2 in branched? https://bugzilla.wikimedia.org/show_bug.cgi?id=31576 [17:45:49] there was a php error [17:46:15] mmovchin, it's been reported a couple of minutes before you joined :) [17:46:19] down all server? [17:46:20] * AaronSchulz ran the wrong sync script, I was trying to just sync on box ;) [17:46:25] *one box [17:46:59] pill: I was to late. Like always. [17:47:00] :P [17:47:24] Merlissimo: we can merge that back into 1.20wmf1 and into 1.20wmf2 [17:49:30] RoanKattouw: ok, testing via getMWVersion looks OK, it must be the * handling [17:49:31] * Reedy kicks fenari [17:49:52] star handling? [17:49:54] * RoanKattouw is confused [17:49:58] Oooh that [17:50:01] Reedy: that would be ok, but on every new wmf branch that was created in the pasted it was added after the bug was live again. How can we make sure that sb. thinks of this when e.g. wmf3 is branched and _before_ is goes live? [17:50:26] Fix the actual bug that causes it, so the hack isn't needed? [17:51:58] AaronSchulz: are these FileRepo.php live hacks yours? [17:58:45] !log reedy synchronized php-1.20wmf1/includes/MagicWord.php 'https://gerrit.wikimedia.org/r/6135' [17:58:48] Logged the message, Master [17:59:34] Are you branching extensions too? [17:59:56] We will be, yes [18:00:06] Everything that's submitted will be going live [18:00:24] Oh, I would like to have this change -> https://gerrit.wikimedia.org/r/#change,5949 merged for Proofread extension. [18:00:51] Need to get someone to review it [18:00:53] !log aaron synchronized multiversion [18:00:55] Logged the message, Master [18:08:36] ok, good, new cdb has empty diff against old one [18:09:06] !log aaron rebuilt wikiversions.cdb and synchronized wikiversions files: Sanity run after script changes. [18:09:08] Logged the message, Master [18:09:29] Beau's rev looks like one that we should take....it fixes an exception that should block deployment on Wednesday [18:09:57] Reedy: are you deploying to test? Looks like I'm done fiddling with multiversion now [18:10:20] * apergos is curious about the changes [18:11:13] * AaronSchulz goes back to filebackend land [18:12:32] I guess we can backport it if we can't find a reviewer [18:13:24] robla: you can always unmerge my change which introduced the issue in the first place ;) [18:14:41] Beau_: Gerrit change #? [18:14:47] 5949 [18:14:54] * AaronSchulz wonders who is deploying [18:15:02] !g 5949 [18:15:09] * RoanKattouw isn't deploying anything [18:15:13] * robla assumes Reedy is preparing the branch [18:15:16] https://gerrit.wikimedia.org/r/#change,5949 [18:15:20] Thanks Beau_ [18:15:37] it's Beau_bot ;-) [18:15:56] Approved and merged [18:16:02] thanks [18:18:50] Need to check on/merge (into trunk) the life hacks Chad mentioned [18:19:28] !log asher synchronized wmf-config/db.php 'adding db59,60 to s1 with low weights' [18:19:30] Logged the message, Master [18:20:09] http://etherpad.wikimedia.org/120wmf2deployment [18:23:17] The only export related revisions I can see are in trunk [18:23:29] I count 3 commits to be merged into 1.20wmf2 [18:37:02] Ok, so, we have a basic branch [18:37:03] https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/core.git;a=log;h=refs/heads/wmf/1.20wmf2 [18:37:27] 3 commits to merge into it.. [18:48:02] Reedy: are you merging those, or waiting on something? [18:48:18] I'm waiting for it to clone extensions etc [18:48:27] ah [18:49:05] then 3 cherry picks, push/git review, pull, copy [18:59:58] Grrrrr [19:00:47] "You have more than one commit that you are about to submit." [19:00:52] Why won't defaultremote=origin work? [19:01:59] Oh.. ! [remote rejected] HEAD -> refs/for/master/wmf/1.20wmf2 (missing Change-Id in commit message) [19:08:14] RoanKattouw: http://etherpad.wikimedia.org/120wmf2deployment [19:09:01] Reedy: What are you doing and why? [19:09:03] * RoanKattouw is confused [19:09:22] Trying to commit a cherry picked (not from Gerrit) change into 1.20wmf1 and submit it [19:09:43] OK [19:09:51] Is this change just queued for review, or has it already been approved? [19:10:08] neither [19:10:17] Oh is this a cherry-pick of a cherry-pick? [19:10:35] A revision that was pushed into 1.20wmf1 bypassing review, and you wanna cherry-pick it into 1.20wmf2? [19:10:36] No [19:11:27] https://gerrit.wikimedia.org/r/#q,6142,n,z [19:11:46] Ah good [19:11:58] Stupid thing [19:12:02] Shouldn't happen in future ;) [19:12:22] OK [19:12:30] I'll need to rebase 6144 so it doesn't depend on 6142 [19:12:43] bleh [19:12:48] Not a big deal [19:12:50] because they were submitted to different branches [19:13:05] yeah [19:13:24] we should get variations of these things into master probably [19:13:47] even if it means adding $wgUglyWMFHacks [19:14:23] is it possible to get the width and/or haight of a file using magic words [19:14:37] I don't think there is much point.. The difficulty here was last time I directly committed it [19:14:46] It's in gerrit this time, so cherry-pick will be much nicer [19:15:15] I want to determine if the file is widescreen or not [19:15:21] There isn't AFAIK [19:15:21] or if it is large enough [19:15:44] the max resolution is displayed already though [19:15:57] what holds that value? [19:16:08] Reedy: I think the point would be to have it in master offending everyone maybe motivating someone to actually fix the root problem ;-) [19:16:22] Reedy: I wonder if the SquidUpdate hack in HTMLCacheUpdate is still necessary, it was a "temporary" hack instated in March 2011 [19:16:38] many "temporary" hacks :) [19:16:48] Something to deal with later! No point complicating the deployment [19:16:54] agreed [19:17:02] Time to upset NFS [19:17:04] I'll reply to Tim's original e-mail asking him if it can be disabled [19:18:13] !log Syncing php-1.20wmf2 files from NFS to apaches. Likely to upset NFS (or the uplink for the switch nfs is on) for a little while... [19:18:15] Logged the message, Master [19:19:05] Reedy: was https://gerrit.wikimedia.org/r/#change,6144 intended for master or wmf/1.20wmf2 ? [19:19:23] ...'cause by my reading it went into master [19:19:24] 1.20wmf2 [19:19:46] Yeah that went into master [19:19:54] Was that supposed to go into 1.20wmf2 instead? [19:19:55] I guess it was [19:19:56] Yup [19:19:58] * RoanKattouw reverts and fixes [19:20:22] The outstanding commits are: [19:20:22] 6f528ec (HEAD, wmf/1.20wmf2) Forward port of https://www.mediawiki.org/wiki/Special:Code/MediaWiki/105964 [19:20:28] Rarrgh [19:21:50] * robla was going to be impressed that hashar already had jenkins running on 1.20wmf2 [19:22:01] Reedy: Sorry about that [19:22:28] I think it was my fault ;) [19:22:33] Only partly [19:22:38] I "fixed" it without checking [19:22:40] So I made it worse [19:23:13] "no new changes" [19:23:14] boo [19:23:41] nobody is perfect [19:24:04] Reedy: OK I can fix this but your clone is gonna be in a slightly weird state [19:24:17] I'll delete the branch locally, it's not a big deal [19:24:43] Aha [19:24:47] You didn't change .gitreview [19:26:12] !log reedy synchronized php-1.20wmf2 'Syncing 1.20wmf2 for deployment' [19:26:15] Logged the message, Master [19:26:36] Not too bad, only mw60 full [19:27:13] * RoanKattouw waits for Jenkins to finish verifying 6148 and 6149 [19:28:14] !log reedy synchronized php-1.20wmf2 'Syncing 1.20wmf1 live hack revisions' [19:28:17] Logged the message, Master [19:29:06] Reedy: OK all done [19:30:26] !log reedy synchronized php-1.20wmf2 'Syncing 1.20wmf2 live hack revisions' [19:30:28] Logged the message, Master [19:30:55] Is checkoutMediaWiki supposed to be correct now? [19:31:39] Looks upto date.. [19:31:54] AaronSchulz: ^ Is checkoutMediaWiki supposed to be correct now? [19:33:00] Reedy: in terms of symlinks? yes [19:33:02] RoanKattouw: "static-1.20wmf2" thats your changes, right? [19:33:16] it should all "just work" now [19:33:16] Look fine [19:33:35] Yes, static-1.20wmf2 is my stuff [19:34:06] Certainly makes more sense now :p [19:36:16] Right then, time to run scap and break test2wiki [19:39:23] ah....what fun is breaking test2wiki, can't we break them all? [19:40:29] We can do enwiki first next time [19:40:34] Blah, gerrit spam [19:43:44] Hmm, that's strange [19:43:46] Updating ExtensionMessages-1.20wmf2.php... [19:43:46] Loading data from /home/wikipedia/common/php-1.20wmf1/extensions/AbuseFilter/AbuseFilter.php [19:43:46] Loading data from /home/wikipedia/common/php-1.20wmf1/extensions/AntiBot/AntiBot.php [19:45:58] !log reedy synchronizing Wikimedia installation... : Pushing out new symlinks etc, moving test2wiki to 1.20wmf2 [19:46:01] Logged the message, Master [19:47:26] https://test2.wikipedia.org/wiki/Special:Version [19:47:29] LocalSettings.php not found. [19:49:17] !log reedy synchronized php-1.20wmf2/LocalSettings.php 'Pushing LocalSettings.php' [19:49:19] Logged the message, Master [19:49:58] !log reedy synchronized wmf-config/ExtensionMessages-1.20wmf2.php 'Syncing file' [19:50:00] Logged the message, Master [19:50:03] why didn't scap push those 2 out? :/ [19:52:52] Updating ExtensionMessages-1.20wmf2.php... [19:52:52] Loading data from /home/wikipedia/common/php-1.20wmf2/extensions/AbuseFilter/AbuseFilter.php [19:52:54] Better 2nd time round [19:53:56] I've been experimenting with 'git shortlog' but before I go too much further, is anyone else working on RELEASE NOTES for 1.20wmf2? [19:55:10] !log reedy synchronizing Wikimedia installation... : Rebuiild l10n for 1.20wmf2 [19:55:14] Logged the message, Master [19:55:36] chrismcmahon: making a diff of https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/core.git;a=blob;f=RELEASE-NOTES-1.20;h=0e8cdc09e6163154a6182e301a37aebdb244cc41;hb=refs/heads/wmf/1.20wmf1 and https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/core.git;a=blob;f=RELEASE-NOTES-1.20;h=0e8cdc09e6163154a6182e301a37aebdb244cc41;hb=refs/heads/wmf/1.20wmf2 should give you some of it [19:55:55] Well, not those links [19:56:06] * Reedy tries again [19:56:19] https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/core.git;a=blob;f=RELEASE-NOTES-1.20;h=0e8cdc09e6163154a6182e301a37aebdb244cc41;hb=wmf/1.20wmf1 [19:56:26] https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/core.git;a=blob;f=RELEASE-NOTES-1.20;h=bd8f775511332803c4b63260c706edddebcf9e07;hb=wmf/1.20wmf2 [19:56:35] 13 lines more [19:57:47] robla: scap would seem to be somewhat broken [19:57:54] It's not copying files to the apaches.. [19:58:05] Reedy: Yeah it's inexplicably broken [19:58:11] As in I've read the source and cannot figure out why [19:58:14] You need to run: [19:58:15] Already known? [19:58:15] that's kinda its job, isn't it [19:58:24] I'm just using sync-dir instead [19:58:29] AaronSchulz: ^ [19:58:32] (to push all the l10n cache out) [19:58:54] Though, it probably means we have weird things not actually out on t eh cluster [19:59:50] What's happening to fenari? [19:59:55] Reedy: so you rebuild the cache on home and are syncing? [20:00:02] It doesn't seem to be responding to ssh very well [20:00:14] probably means NFS is complaining [20:00:49] AaronSchulz: yeah. Scap had built them, it just hadn't pushed them [20:01:01] http://ganglia.wikimedia.org/latest/graph_all_periods.php?c=Miscellaneous%20pmtpa&m=load_one&r=hour&s=by%20name&hc=4&mc=2&st=1335815996&g=network_report&z=large&c=Miscellaneous%20pmtpa [20:01:10] Oh, I wonder what those misc out peaks are from... [20:01:11] * Reedy hides [20:01:49] * RoanKattouw is still waiting on 'ssh fenari' to complete [20:02:22] Reedy: What you'd run to push them is ddsh -F30 -cM -g mediawiki-installation -o -oSetupTimeout=10 '/usr/bin/scap-1' [20:02:36] Isn't that exact line in scap already? [20:02:47] Yse [20:02:51] yes, but it's not the skins part [20:02:51] Awesome [20:02:54] But scap mysteriously stops right before that line [20:03:01] Or rather before the line before [20:03:24] Like it does get to echo 'Copying style sheets to apaches...' but it doesn't reach echo 'Copying code to apaches...' [20:03:24] I <3 Computers [20:03:42] fenari is slightly more responsive now [20:03:49] load average: 26.76, 17.69, 8.75 [20:05:12] In better news, test2 should load load average: 26.76, 17.69, 8.75 [20:05:16] https://test2.wikipedia.org/wiki/Special:Version [20:06:24] No .git metadata copied [20:06:44] Failed to load resource: the server responded with a status of 404 (Not Found) https://bits.wikimedia.org/static-1.20wmf2/skins/vector/images/search-ltr.png?303-4 [20:06:44] Failed to load resource: the server responded with a status of 404 (Not Found) https://bits.wikimedia.org/static-1.20wmf2/skins/common/images/poweredby_mediawiki_88x31.png [20:06:47] I wonder why [20:06:50] * Reedy grins [20:08:23] With more frequent releases, this is more reason to get those uplinks upgraded.. [20:17:38] Reedy so we will have weekly updates now <3 [20:18:29] !log reedy synchronized php-1.20wmf2/cache/ 'Synching whole cache directory' [20:18:31] Logged the message, Master [20:24:55] !log Running ddsh -F30 -cM -g mediawiki-installation -o -oSetupTimeout=10 '/usr/bin/scap-1' in the hope it syncs all the files that would be nice to be on the app servers [20:24:57] Logged the message, Master [20:32:21] Oooh [20:32:26] Who broke Special:Version? https://test2.wikipedia.org/wiki/Special:Version [20:32:56] fine locally [20:32:58] * Reedy ignores [20:34:53] Quite a few apparently full apaches [20:36:58] Fatal error: Unsupported operand types in /usr/local/apache/common-local/php-1.20wmf2/languages/Language.php on line 362 [20:37:03] $this->namespaceNames = $wgExtraNamespaces + $this->namespaceNames + $validNamespaces; [20:37:46] presumably transient [20:50:43] robla: all looks ok bar bits caching 404s for resources [20:53:27] Reedy: root cause for Special:Version issue on test2? [20:54:34] Not sure [20:54:47] It's fine on git head locally [20:55:39] It looks to be missing some CSS rules [20:55:47] So it's possibly related to bits being a PITA [20:56:01] Reedy: not sure if you saw it, there is a wmf2 change (bug 35685) api.php URL and other entry point URLs are now listed on Special:Version [20:56:12] Yup [20:56:37] * chrismcmahon snatches at straws :) [20:56:59] !log disabled all Jenkins jobs on Aluminium in prep for db1008 reboot [20:57:02] Logged the message, Master [20:57:09] Reedy: I got a bunch of sudo deny emails from scap-2 [20:57:33] Yeah, ignore/delete them [20:57:45] Reedy: what are the 404 issues? [20:57:47] I failed [20:57:59] robla: cached 404s [20:58:00] https://bits.wikimedia.org/static-1.20wmf2/skins/common/images/poweredby_mediawiki_88x31.png [20:58:07] vs https://bits.wikimedia.org/static-1.20wmf2/skins/common/images/poweredby_mediawiki_88x31.png?4 [20:58:17] Seems our varnish purge script, doesn't work [20:58:27] hrm...that's an issue [20:59:45] !log reedy synchronized php-1.20wmf2/resources/startup.js 'touch' [20:59:47] Logged the message, Master [21:00:43] !log put CiviCRM on Aluminium in maintenance mode for db1008 reboot [21:00:46] Logged the message, Master [21:02:08] How long does varnish cache stuff for? [21:02:33] A long long long time [21:02:47] * RoanKattouw runs purge-varnish 'static-1.20wmf2' [21:02:52] Oooh you have me https:// before [21:02:55] Of course that didn't work [21:03:56] you mean past 2038? ;) [21:11:37] !log CiviCRM back to normal after db1008 reboot [21:11:40] Logged the message, Master [21:11:59] !log re-enabled Jenkins jobs on Aluminium after db1008 reboot [21:12:02] Logged the message, Master [21:22:12] Hi, a question [21:22:12] http://i.imgur.com/aumUT.png [21:22:18] ^ any explanation? Can it be fixed? [21:23:56] Chrome v. 18.0.1025.162 m on Windows 7, etc. [21:23:56] Do you only see that on diffs? [21:23:56] or only there? [21:24:15] Only there [21:31:24] I've just had /win 6 [21:31:26] urgh, fail with irssi... sorry [21:31:29] I've just had an interesting error while creating an account for someone via email. It looks like the account was created from the logs, but the error I got gives me concern. [21:31:33] http://helpmebot.org.uk/~stwalkerster/mw%20error.png [21:34:49] https://en.wikipedia.org/w/index.php?title=Wikipedia_talk:AutoWikiBrowser/Bugs&diff=prev&oldid=477299226 is the URL [21:35:39] Oh [21:35:52] Reedy: no problem there, just the editpage code includes that content [21:37:35] I see it also [21:37:35] IT's something on the page I think