[00:17:21] mwalker & awight, my scap is complete [00:17:32] thanks :) [00:20:16] so, pgerhes and I are starting our centralnotice deploy [00:20:45] gn8 folks [05:10:16] I'm on http://test2.wikipedia.org/wiki/Special:UploadWizard using IE 9.0, and the Upload Wizard won't load for me. [05:10:45] valeriej: elaborate? [05:10:51] what exactly does happen? [05:11:59] * jeremyb has not his own IE9 to play with [05:12:20] I just see the title "Upload Wizard" and a loading animation. [05:13:55] The buttons 'Select a media file to donate' and 'Add image files from Flickr' are on the page when I load it on Firefox. [05:15:21] grrrrrrr, what is this

Namespaces

thing? is new? [05:16:49] valeriej: is there some kind of error console you can check for JS errors? [05:16:57] valeriej: works in IE9 on commons? [05:17:26] and what about on test.wp ? [05:17:53] testwiki and test2wiki are 1.21wmf5, commons is 1.21wmf4 [05:18:45] I can upload a file on http://commons.wikimedia.org/wiki/Special:UploadWizard, yes. [05:19:33] what about test.wikipedia ? [05:19:47] and error console [05:19:52] i have to run shortly [05:21:44] The page loads on test.wikipedia. I selected a file to upload, but the file gets stuck on 'Getting file information and previews...' [05:22:31] hrmmmm [05:22:42] valeriej: will you be around tomorrow? [05:23:00] And by 'stuck' I mean the file won't upload. It displays that message with a loading animation. That was the problem I was trying to test before the page loaded wird. I mention it and the end of my post here: https://bugzilla.wikimedia.org/show_bug.cgi?id=30924#c4 [05:23:11] Yes, I can be. What time will you be around? [05:24:08] valeriej: idk, but there's a couple people that recently were fixing other things with the upload wizard and I'd like to get them on at the same time [05:24:28] valeriej: are you usually online in the day? [05:24:47] * jeremyb opens the link [05:25:05] andre__: ^ [05:26:38] * jeremyb is sleepy [16:36:15] ack. Education Program extension is broken again. "PHP fatal error in /usr/local/apache/common-local/php-1.21wmf4/extensions/EducationProgram/includes/EPTimelineGroup.php line 365: [16:36:15] Call to a member function getId() on a non-object " [17:20:24] chrismcmahon, jeremyb told me to look for you. I am having problems using the Upload Wizard with IE 9. At first, I could not upload a file using the wizard. I would get stuck at 'Getting file information and previews...'. At some point last night when I was trying to replicate the error, the Upload Wizard would not even load. The page loads for me now, but sometimes it won't. When that happens I'll reload and after a coup [17:20:46] Oh, and I'm using the Upload Wizard on test.wp and test2.wp. [17:20:55] I'm not having problems on commons. [17:20:59] hi valeriej [17:21:12] Hello. Sorry for the info dump. [17:21:35] valeriej: you were truncated at When that happens I'll reload and after a coup [17:22:37] Oh... "When that happens I'll reload and after a couple of tries, the page will load again." [17:23:02] chrismcmahon: do ya'll have IE9? [17:23:11] I have error messages from the console, for both situations. [17:23:19] ah, what are those? [17:24:06] When the page doesn't load : LOG: Exception thrown by ext.uploadWizard: 'UploadWizardConfig' is undefined [17:24:07] ReferenceError: 'UploadWizardConfig' is undefined [17:25:05] �nd when the page loads, I select a file to upload: ""SCRIPT5007: Unable to get value of the property 'date': object is null or undefined [17:25:05] load.php?debug=false&lang=en&modules=jquery%2Cmediawiki%2CSpinner%7Cjquery.triggerQueueCallback%2CloadingSpinner%2CmwEmbedUtil%7Cmw.MwEmbedSupport&only=scripts&skin=vector&version=20121126T175337Z, line 45 character 75 [17:25:56] chrismcmahon: ^ [17:26:17] I tried it in debug mode was pointed to line 143. This condition was highlighted "if(dateObj==undefined&&this.upload.file.date!=undefined)". [17:27:27] marktraceur: ^^ [17:28:10] * marktraceur is excited to get highlighted in #wm-tech [17:29:40] valeriej: I'm trying to reproduce your issue [17:30:37] Ok. [17:35:32] valeriej: for me right now in IE9 UW is hanging at "Getting file information and previews..." and doesn't seem to return. You seem to have more information than I can get, may I ask that you create an issue for this in Bugzilla? https://bugzilla.wikimedia.org/enter_bug.cgi [17:35:54] chrismcmahon: Maybe you should be running developer tools to get the log message? [17:36:44] marktraceur: the only IE9 I have is on a hosted service and non-IE browsers seem to be OK [17:37:30] chrismcmahon: IE should *come* with dev tools. I think. [17:37:46] Yes, I used the dev tools to get the error messages. [17:37:51] * marktraceur remembers something about F12 [17:39:03] marktraceur: so it does, thanks [17:39:33] valeriej: Plan B: I'll make a bugzilla issue and you can comment on it if you want. [17:39:46] Ok. Sounds good. [17:40:58] is there somebody here who cares about issues on live WMF sites that are breaking pages for all anonymous viewers? [17:41:06] disclaimer: that site is not english wikipedia [17:41:07] https://bugzilla.wikimedia.org/show_bug.cgi?id=41832#c6 [17:41:32] reported almost a month ago, still broken [17:41:47] it would be great if someone looked at it [17:51:18] valeriej: thanks! I think I've captured everything in https://bugzilla.wikimedia.org/show_bug.cgi?id=42512 [17:53:06] Thanks. I don't see the issue in Firefox, btw. I'll add that in a comment. [18:03:35] Reedy: refresh my memory? today's 1.21wmf5 deploy is *not* going to commons, correct? [18:03:57] It should be [18:04:11] Reedy: in that case, I think https://bugzilla.wikimedia.org/show_bug.cgi?id=42512 blocks it [18:04:25] lol [18:04:26] awesome [18:06:29] Reedy: sure am glad I asked then, and thanks very much to valeriej. We actually have an automated browser test that is failing on this problem, but other failures in the suite that we're cleaning up masked it. [18:07:42] Glad I could help. [18:26:31] andre__: bug 41832 has new comments since you last commented... /me wonders if you've seen them [18:31:41] chrismcmahon: where can i see these tests run that you say were failing on the same problem? [18:31:47] chrismcmahon: and where is the remote IE9 ? [18:31:51] tesla? [18:32:17] i suppose we could theoretically run windows on nova too ;-P [18:34:31] bbl [19:05:23] jeremyb, yeah saw them, but thanks [19:34:26] ^demon: thanks for filling in on this deployment [19:35:21] ^demon: after we deploy to the wikis, let's also enable TemplateSandbox on all of the wmf5 wikis [19:39:53] <^demon> Argh. [19:40:06] argh? [19:40:14] <^demon> mutante: I'm still getting the rsa key thing :\ [19:41:12] ^demon: where? i am not aware [19:41:25] <^demon> Oh wait, I meant notpeter. [19:41:25] <^demon> Sorry [19:41:27] <^demon> :) [19:41:33] ah,ok [19:42:42] <^demon> notpeter: I cleared out my known_hosts just to make sure. [19:42:45] <^demon> Still no luck :( [19:43:39] wtf? [19:43:41] is anyone lese? [19:43:45] *else [19:44:33] <^demon> Nobody else complaining :( [19:44:41] ^demon: what's the rsa key thing? getting into fenari, or doing stuff on fenari, or something else entirely? [19:44:59] <^demon> fenari's complaining when I try to rsync out to the mw* boxes. [19:46:03] there are known_hosts in your home and in /etc/ssh/ too [19:46:16] you probably need to clean from the global one [19:46:37] should tell you where the offending key is [19:47:12] <^demon> /home/demon/.ssh/known_hosts is empty [19:48:03] any "Offending key in .. ?"? [19:48:07] * robla steps out to grab lunch....should be back in 10 min or so [19:48:22] <^demon> It doesn't say. Here's the output. http://p.defau.lt/?tE6yqICaQsCy240gYsfZIQ [19:50:24] ^demon: not sure if you saw this earlier, but I made https://bugzilla.wikimedia.org/show_bug.cgi?id=42512 a blocker for deploying wmf5 to commons [19:51:10] <^demon> No, I hadn't. [19:51:54] <^demon> If UploadWizard is broken, we should change the submodule for 1.21wmf5 [19:52:02] <^demon> Rather than having the broken version deployed. [19:52:35] huh, oh, weird [19:52:48] so that's not key mismatch, that it not having anything... [19:52:48] weird [19:54:07] for host in $(cat /etc/dsh/group/apaches); do ssh-keygen -R $host; done [19:55:16] did yo ujust do that, mutante ? [19:55:21] yes [19:55:36] ^demon: https://bugzilla.wikimedia.org/show_bug.cgi?id=41832 is also listed as a blocker but I'm not sure it truly is [19:55:44] * robla is back [19:55:46] ah, ok [19:55:54] that should def work.... [19:55:55] notpeter: now i am connecting via SSH to say "yes" once [19:56:09] but..ehm.. i actually need to go to driving test [19:56:17] chrismcmahon: i added it earlier. it breaks rendering on live sites for all anonymous users. [19:56:32] ah crap....forgot to check the blockers [19:56:35] and has been for a month, and nobody cared, so i did this toi maybe raise awareness or something. [19:57:02] it's not enwiki, so i can only assume nobody cared enough to check [19:57:18] you can probably go on with the deployment (no sarcasm here) [19:57:33] but it would be nice if somebody checked that out. :( [20:00:38] MatmaRex: understood. please don't use the blocker that way again, though, ok? [20:01:15] ^demon: run this: for host in $(cat /etc/dsh/group/apaches); do ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no $host -C 'exit'; done [20:02:11] ^demon: notpeter: arr..or not ... please check again i need to run [20:02:26] but this should definitely readd the host keys [20:02:31] robla: how else do i get someone to look at it? [20:02:34] (honest question) [20:02:46] i tried upping the priority and it was ignored [20:02:53] i tried mentioning it here and it was ignored as well [20:03:24] MatmaRex: talk to andre__ about the best escalation strategy [20:03:25] and, as i said, it's breaking the rendering right now, for all anonymous users of Polish wikipedia, and probably other wikis with flaggedrevs enabled [20:03:42] MatmaRex, I'm looking at it right now [20:04:02] I just assigned the bug itself to AaronSchulz [20:04:05] MatmaRex, however what confuses me is that this is the case since at least 2010 [20:04:11] as per https://bugzilla.wikimedia.org/show_bug.cgi?id=15748#c10 [20:04:21] MatmaRex, so there must be something else that this is suddenly an issue [20:04:49] andre__: that bug is marked a fixed, so it might have been reintroduces by something? [20:05:05] andre__: i've heard repors about it since approx. the time the bug was created [20:05:10] heard reports [20:05:50] also sorry for the guerilla tactics i used, but honestly, i'm just feeling ignored here all the time, and this looks like an issue that should be fixed immediately, not after a month of rotting... [20:06:31] it's like nobody cares about anything except for the English Wikipedia [20:06:53] MatmaRex: it is more so about what extensions have maintainers and which ones don't [20:06:58] FR doesn't have one [20:06:59] MatmaRex: sorry that it took that. I'm working with andre__ on the correct escalation strategy for just this type of issue [20:07:43] MatmaRex: dunno if you're following wikitech-l, but that's the reason why the priority standardization takes on some urgency from my perspective [20:07:45] AaronSchulz: to me it looks like maybe reverting the changes if there's nobody around who's able to fix new bugs :/ [20:07:58] robla: yeah, i am, neat bikeshedding by everyone there [20:10:07] <^demon> mutante: I tried said command...same errors [20:11:53] MatmaRex: if you have a prefered color for the bikeshed I'd love to know it :P [20:12:58] andre__: i'm not really in position to discuss it since i'm not using any of those fields, i just fix bugs which affect myself [20:13:26] well, except for when i'm raising priority to >=high and severity to >=major, and i usually change both in the same way ;) [20:13:40] MatmaRex, heh, okay. I wasn't too serious. But really, thanks for bumping that ticket! [20:14:47] though the missing flags issue on pl.wp reminded me rather of https://bugzilla.wikimedia.org/show_bug.cgi?id=41113 [20:14:58] the pleasure's all mine :) [20:15:37] andre__: that one looked almost the same, but it was something completely different afaik [20:16:00] that one was visible for logged-in users, and on wikis without flaggedrevs, right? [20:16:03] I'm also afraid of that. [20:16:08] meanwhile, we have https://bugzilla.wikimedia.org/show_bug.cgi?id=42512 which is probably a blocker [20:16:11] yes, IIRC [20:16:21] the IE9 one? [20:16:30] heh, nice timing. was just looking at it. Sigh. [20:16:34] anyone know Nischay's IRC handle? [20:17:09] nm....looks like he's Nischay, but not on this channel [20:17:23] assuming it's 01:45AM for him, unlikely to find him online [20:22:38] ^demon: are you any closer to figuring out your deploy issues? [20:22:48] <^demon> No. [20:22:54] <^demon> I tried mutante's suggestion, but it didn't work. [21:06:11] I assume someone's already working on fixing the mess? [21:07:27] (the mess being giant scary headers everywhere they shouldn't be) [21:09:39] ^demon: ^ [21:10:13] <^demon> Link? Screenshot? [21:10:16] <^demon> I'm not seeing anything [21:10:28] So far I've only seen it with Twinkle links [21:10:35] https://www.wikidata.org/wiki/File:Wikidata_glitch.png [21:13:00] I thought I saw a bug for that somewhere [21:13:40] giant scary headers are gone, sidebar is still a bit messed up [21:14:48] the bug appears to be everywhere except wikipedia. commons. and mediawiki [21:17:21] sidebar issue: https://www.wikidata.org/wiki/File:432423423.jpg.png [21:17:29] YairRand: headers that look like http://bug-attachment.wikimedia.org/attachment.cgi?id=11424 by any chance? [21:17:42] yes [21:38:35] hm, broken sidebars seem to have spread to commons... [21:42:56] there are various comment threads about the recent bug(s) popping up in many places... [21:47:17] aaand it's messed up again, at least on http:// [21:49:21] the sites, they is borked. [22:09:51] YairRand, just got that on foundationwiki [22:18:11] does the new mw version require that all sidebar

s be replaced with

s? [22:19:54] I still can't tell how much of the new stuff is bugs and how much was deliberate... [22:21:03] and if they do need to be replaced, that would have been useful info before all the js broke as a result. grumble grumble... [22:23:46] would anyone be terribly offended if fundraising did a syncdir now? [22:26:12] js is taking forever to load. Common.js won't run until about 5 seconds after the page finishes loading. [22:31:02] TimStarling: any idea why https://wikimediafoundation.org/wiki/Home isn't rendering properly ? [22:31:25] see discussion in um [22:31:33] <^demon> I'm going to roll the wikis back to 1.21wmf4. [22:31:37] <^demon> Nobody seems to be fixing it. [22:31:58] that works for me too [22:32:01] cool [22:32:07] who did the deployment ? [22:32:11] ^demon: I will hold deploying CN until you are done [22:32:22] YairRand, ok, someone just reported the issue on wikitech-l [22:32:22] <^demon> I did. [22:32:48] cool [22:33:17] LeslieCarr: you mean with "namespaces", "personal tools", etc. visible? [22:33:30] yep [22:33:36] though it looks like ^demon is taking care of it [22:33:50] that's been broken since yesterday, at least on meta [22:33:58] and mediawiki.org [22:34:03] and test [22:34:17] <^demon> Rolled back today's deploy. [22:34:26] <^demon> So should only be hitting mw.org, test, etc. [22:36:09] yep reload cleared it n foundationwiki [22:36:21] ^demon: can I go ahead and sync-dir CentralNotice then? [22:36:32] great, everything's being broken in all sorts of entertaining ways... [22:36:34] srv261 has issues with stylesheet delivery [22:36:43] page layout is broken [22:36:46] <^demon> pgehres: No. Let's please wait for a bit. [22:37:24] ^demon: okay, no hurry [22:39:06] Danny_B|backup: if it looks like http://bug-attachment.wikimedia.org/attachment.cgi?id=11424 more or less then it's the deploy, now rolled back; are you seeing something else? [22:39:36] exactly [22:39:40] <^demon> Ok, so it seems to be showing up fine, even on wikis that are on wmf5 [22:39:49] thx for saving me work with reporting the bug [22:40:04] σθρε [22:40:07] er, sure [22:40:31] <^demon> Wonder if it was meta that's causing it. [22:40:44] <^demon> Something served from meta that's breaking other wiki styles. [22:42:25] what would foundationwiki pages pull from there? [22:42:46] as a logged out user I don't see a central notice or banner or anything [22:43:56] <^demon> generic central notice crap? [22:43:59] <^demon> Even if you don't have a notice [22:44:03] could be [22:44:38] I should have viewed page source on the broken page [22:45:13] <^demon> Let's take this to #mediawiki. Krinkle & kaldari are discussing it there. It's UI stuff, so I'll defer to them. [22:45:20] k [22:45:28] when is 1.21wmf5 going to be deployed again? [22:45:34] <^demon> Don't know yet. [22:45:39] <^demon> Probably won't be tonight. [22:45:56] <^demon> Unless someone comes up with a fix soon. [22:46:00] ugh [22:46:20] that had way too many bugs [22:46:58] all sidebar sections added by site js broke as a result of that update [22:47:40] and site js was, by the way, taking more than 5 seconds to run [22:48:20] wasn't that just cached css? [23:27:59] ^demon: think we're safe to sync centralnotice now? [23:29:43] <^demon> mwalker: As long as kaldari and Krinkle are done syncing, sure. [23:30:16] watch the mediawiki channel [23:30:48] yay another IRC window to have open :) [23:31:14] <^demon> #mediawiki is the place to hang out :) [23:31:25] <^demon> It was the busy channel before we forked into 8 dozen channels :) [23:31:33] that and this one [23:31:38] <^demon> Indeed. [23:31:46] * apergos still misses having this be the busy tech channel [23:32:20] (so I'd wait a little bitmor for them to give thumbs up generally on the fix) [23:32:22] usually deploy conversations happen here, today's in #mediawiki was a little bit accidental [23:36:11] [23:37:14] gn8 folks [23:40:21] mwalker: both wmf4 and wmf5 are pending merge in gerrit for whenever kaldar_i is done [23:49:55] saper, FYI I edited https://www.mediawiki.org/wiki/Git/Workflow#Prepare_to_work_with_gerrit :)