with a class that is display none and display block from Group-sysop.css [05:35:23] (that's the easiest way anyway) [05:35:48] Thx [05:36:02] if we can revert https://gerrit.wikimedia.org/r/#/c/30597/ but try [05:36:16] RobH_: thanks. Message rolled back [05:36:30] now to beat some editors for not actually testing before screaming about the sky falling [05:36:30] my latest patch for the content model stuff, i think that's the best solution for immediately [05:36:33] and wmf3 only for wikidata [05:37:54] https://gerrit.wikimedia.org/r/#/c/31179/ should suffice for wikidata, as it's used now [05:38:08] * marktraceur wonders if the bug is getting fixed already :) [05:38:17] marktraceur: yes [05:38:22] Excellent [05:38:29] Ironholds: You mis-tab-completed above. [05:38:35] oops [05:38:35] robla, rather. [05:39:03] commons has wmf2 now, which is pre-bug i'm 99.9% sure [05:39:19] Well that's no reason not to panic. [05:39:55] aude: Commons is rollbacked then? I can do test and clear sitenotice if it's good. [05:40:21] Dcoetzee: https://commons.wikimedia.org/wiki/Special:Version [05:40:23] wmf2 [05:40:30] I don't think Commons ever had wmf3. [05:40:41] i understand it is [05:41:09] Brooke: think it did [05:41:12] https://gerrit.wikimedia.org/r/#/c/31046/1/wikiversions.dat [05:41:26] Brooke: It had the bug, that's for sure [05:41:30] although if there were some bugs before then.... [05:41:37] If it didn't have wmf3, then wmf2 is borked [05:41:51] Hmm, ignore me, then. [05:42:07] 04:47 logmsgbot_: aaron rebuilt wikiversions.cdb and synchronized wikiversions files: reverted wmf3 deployment made earlier today. [05:42:10] Dcoetzee: when did https://bugzilla.wikimedia.org/show_bug.cgi?id=41606 happen? [05:42:21] aude: I'll pinpoint it [05:42:25] thanks [05:42:33] * aude didn't hear anything like that before today [05:42:47] aude: 18:22, 31 October 2012‎ [05:42:47] pretty likely a wmf3 problem [05:42:47] and have been checking at the village pumps, etc. with wmf2 [05:42:50] agree [05:42:56] (UTC) [05:43:07] http://wikitech.wikimedia.org/view/Server_admin_log [05:43:07] i think that's after wmf3 [05:43:15] https://www.mediawiki.org/wiki/MediaWiki_1.21/Roadmap#Schedule_for_the_deployments [05:44:47] 18:07 logmsgbot_: reedy rebuilt wikiversions.cdb and synchronized wikiversions files: special, closed and private to 1.21wmf3 [05:44:55] I think "commons" falls under "special" [05:45:17] Yes, . [05:45:21] yes [05:45:31] it's all the non wikipedias that were in the change earlier [05:45:41] The other failure occured at 22:56, 31 October 2012 UTC [05:45:53] So they were definitely all after 18:07 [05:46:04] As far as I know [05:46:39] Dcoetzee: that's helpful [05:46:48] yup, thanks! [05:47:00] aude: are you up late or up early? :) [05:47:19] early :) [05:47:30] it's normal, often [05:47:59] otherwise, trying to get a new branch of wikibase ready [05:48:25] glad you're here for this...thanks for jumping on it! [05:48:27] sure [05:48:34] (and thanks Aaron|home!) [05:49:11] Roan said earlier "Today was the regularly scheduled biweekly deploy to Commons" [05:49:43] yeah, it was. [05:50:45] such is how things go with the more continuous deployment, is suppose [05:51:06] quick from putting stuff in master to deploying it [05:51:49] nice that reverting only means reverting to a couple weeks prior rather than the hand wrenching we used to do [05:51:50] * aude wants more test cases to catch stuff [05:51:59] yeah [05:52:37] jenkins is false security when so much key, core stuff has few tests [05:53:05] But he's so cute in his butler suit! [05:53:15] :) [05:53:45] ^ colloquy [07:41:50] TimStarling: is the job queue lazying again? [07:42:13] since 5 UTC [08:21:03] !log Job queue runners almost idle again since 5 UTC; en.wiki queue at 145k, zh.wiki 260k, Commons 60k now [08:21:11] Logged the message, Master [09:00:42] Nemo_bis: almost certainly the job queue thing has to do with the revert from wmf3 [09:00:57] I guess folsk wil be able t patch it up later when they show up [10:48:20] Anyone available to help me deploy code? [10:55:30] Hydriz: going to lunch soon sorry :/ [10:55:41] nevermind, its just wiki config changes [10:58:59] Hydriz: is that in Gerrit? [10:59:35] yep [11:00:06] ahhh the google news site map extension [11:00:14] :) [11:00:32] I don't think there is any SQL changes? or I am wrong... [11:00:39] luckily that ext is already deployed [11:00:53] hehe [11:01:28] But I also need someone to review this: https://gerrit.wikimedia.org/r/#/c/30333/ [11:01:50] added myself on review of the Extension:GoogleNewsSitemap [11:01:54] might deploy it after lunch [11:02:07] I don't like to alter the cluster when I know I must leave soon ;-] [11:02:29] sure :) [11:05:51] and Dereckson asked me to do some namespaces update ;-] [11:05:58] so I guess I will handle a few shell requests this afternoon [11:06:09] thats good :) [11:06:18] plus some labs stuff, if you want [11:06:21] lunch time for now, will be back in roughly 2 hours [13:47:13] Anyone know why http://en.wikipedia.org/w/skins/monobook/main.css doesn't seem to match what's in git? Line 22 has "z-index: 0" instead of "z-index: 2", which leads to [[en:WP:VPT#New color on the edit tabs?]] [14:20:09] are you looking at the correct b ranch ? [14:21:26] anomie: that is a live hack on our cluster I guess [14:36:49] say ^demon, do you know is there a plan to reattempt wmf3 today? [14:36:52] I guess it was rolled back yesterday [14:37:32] <^demon> I don't know, actually. I don't do much deployment stuff other than one-off firefighting these days. Reedy would know, but he's idle. [14:37:34] ok [14:37:38] I'll check later, thanks [14:51:57] I didn't even know about the rollback. Reedy should have updated https://www.mediawiki.org/wiki/MediaWiki_1.21/Roadmap [14:53:38] What rollback? [14:53:53] wasn't there a revert of wmf3 yesterday late? [14:54:07] at least, I thought I read something to that effect in the channel [14:54:21] (now youre going to ask me which channel, and I'm going to have to hunt around) [14:54:24] 10 hours ago [14:54:27] ok [14:54:30] It's in the mediawiki-config git log [14:54:41] https://bugzilla.wikimedia.org/show_bug.cgi?id=41615 [14:55:10] so the job runners... on the lam again today :-/ [14:55:12] http://ganglia.wikimedia.org/latest/?r=hour&cs=&ce=&s=by+name&c=Jobrunners%2520pmtpa&tab=m&vn= [14:55:27] mw12 only looks somewhat loaded cause I've been running non priority jobs over there by hand [14:55:34] not that it's doing much but whatever [14:55:47] I figure that's connected with the rollback, logiaclly [14:55:49] er [14:55:50] logically [14:56:09] http://ganglia.wikimedia.org/latest/?r=day&cs=&ce=&m=load_one&s=by+name&c=Jobrunners+pmtpa&h=&host_regex=&max_graphs=0&tab=m&vn=&sh=1&z=small&hc=4 [14:56:50] going to drop of lawyer paperwork, back in a bit (hopefully 30 min but if I have to wait it could be longer) [14:56:58] Reedy: when you do a 1.21wmf* deploy or rollback, it's really useful to me & Andre if you update the status at https://www.mediawiki.org/wiki/MediaWiki_1.21/Roadmap so we actually know that it happened [14:57:11] I didn't do the rollback [14:57:11] I was in bed [14:57:14] As,eep [14:57:15] it was AaronSchulz? [14:57:26] Indeed [14:57:28] Granted, I did forget to mark it {{done}} [14:57:42] * sumanah will slightly complain to Aaron, then [14:59:14] apergos: 04:47 logmsgbot_: aaron rebuilt wikiversions.cdb and synchronized wikiversions files: reverted wmf3 deployment made earlier today. from http://wikitech.wikimedia.org/view/Server_admin_log [15:19:14] no waiting, got to do a drop and run but he might call me later, we'll see [15:31:40] hi i am nasir, i need a help [15:34:42] nasir8891: what kind of help do you need? [15:35:13] sumanah: thanks for the reply [15:35:41] i want to add a few extensions and gadgets in Bengali wikipedia [15:36:14] there is a test place available at http://bn.wikipedia.beta.wmflabs.org [15:36:51] but i want a kind of replica with all the settings and gadgets of Bengali wiki [15:37:06] the contents are not required [15:37:20] so what should be the procedure to get this [15:37:23] ? [15:38:25] nasir8891: I think http://bn.wikipedia.beta.wmflabs.org IS the current replica [15:38:42] nasir8891: you could play there and add user scripts [15:39:33] nasir8891: you could also set up a development environment on your own computer fairly easily via this method http://www.gossamer-threads.com/lists/wiki/wikitech/304935 and look at the configuration settings of Bengali Wikipedia to copy them [15:40:56] sumanah: in the preferences page of http://bn.wikipedia.beta.wmflabs.org only one gadget is available [15:41:37] sumanah: that is why i thought all the other settigns might not be available there [15:43:04] nasir8891: hm. I see that https://bn.wikipedia.org/wiki/%E0%A6%AE%E0%A6%BF%E0%A6%A1%E0%A6%BF%E0%A6%AF%E0%A6%BC%E0%A6%BE%E0%A6%89%E0%A6%87%E0%A6%95%E0%A6%BF:Gadgets-definition and http://bn.wikipedia.beta.wmflabs.org/wiki/%E0%A6%AE%E0%A6%BF%E0%A6%A1%E0%A6%BF%E0%A6%AF%E0%A6%BC%E0%A6%BE%E0%A6%89%E0%A6%87%E0%A6%95%E0%A6%BF:Gadgets-definition are different, yeah [15:43:09] sumanah: another thing is i am an admin of Bengali Wiki but not in http://bn.wikipedia.beta.wmflabs.org, so where should i ask this permission , that i can install new gadgets [15:43:26] nasir8891: #wikimedia-labs and the labs-l mailing list are good places to ask [15:45:03] sumanah: i tried to contact via #wikimedia-labs few hours ago, but did not get any reply, and i came here [15:49:12] nasir8891: the mailing list is better [15:49:43] sumanah: ok. thank [15:49:53] good luck and thanks for contributing! [16:11:59] AaronSchulz: you there? [16:26:42] j^: mdale: Reedy is going to be hleping out instead [16:29:05] chrismcmahon: you ready to do some quick testing? [16:29:21] robla: yessir [16:29:43] Reedy: so we need to update TMH submodule to latest master (2e18020b51ef10e4db00b7ad75b4297d7af6013c) and after that set wmgUseTimedMediaHandler to true for enwiki in wmf-config/InitialiseSettings.php [16:29:57] Yup, I'm just doing that :) [16:33:05] excellent [16:37:17] Right, just confirmed the code on both branches on the cluster is up to date [16:42:45] robla: chrismcmahon, j^ mdale It's live on enwiki [16:43:37] thanks Reedy, looking at Folgers.ogv first [16:44:12] * robla looks [16:45:37] PHP fatal error in /usr/local/apache/common-local/php-1.21wmf2/includes/Message.php line 632: [16:45:37] Call to a member function getText() on a non-object [16:45:51] gah, do we need to go back to wmf3 first? [16:45:51] Let me grab a stack trace [16:46:33] I was submitting an edit [16:46:47] robla: That's PageTriage [16:46:52] http://en.wikipedia.org/wiki/User:JanGerber/commonstest looks ok here [16:46:56] greeeeat [16:47:02] hi folks [16:47:24] do you think that the recent deletion bug might still have influence on other things? [16:47:43] Such as? [16:47:43] e.g. I cannot see the deleted revision of https://commons.wikimedia.org/w/index.php?title=Special:Undelete&target=File%3AMail+fr%C3%A5n+ssb.jpg [16:47:47] robla: https://bugzilla.wikimedia.org/show_bug.cgi?id=41642 [16:47:52] (Commons adminship required) [16:48:13] Reedy: was test2 also updated with your run or just en? test2 is 1.21wmf3 [16:48:25] I did both [16:49:08] hi. trying to save edits at http://en.wikipedia.org/wiki/If_I_Had_to_Do_It_All_Over_Again and i get message "[3eb5bf63] 2012-11-01 16:48:49: Fatal exception of type UsageException" [16:49:38] Reedy: let's back out [16:50:06] looking at the stack trace, it looks like there's some TMH in there [16:50:33] strange (and complicated) error msgs in IE7, but the video plays [16:50:36] Oh, yes.. [16:50:39] not page triage [16:50:40] where did I see that [16:51:16] it's a ginormous trace, so I'm sure everything is in there ;-) [16:51:20] http://en.wikipedia.org/wiki/Polar_bear also looks ok here [16:51:32] how do you get the error? [16:51:42] j^: try creating a new page with a couple videos on it [16:51:48] john__: just normally via your browser? It looks to be ArticleFeedback related [16:52:05] j^: http://en.wikipedia.org/wiki/User:RobLa-WMF/Video_testing was the page I created [16:52:19] Reedy: are you able to look into that file? [16:52:26] odder: not now [16:52:30] Reedy: I'd like to oversight it as soon as I see what's in it [16:52:35] Reedy: sure [16:52:40] Reedy: timing seems too coincidental to me not to be TMH related. if I saw it, and someone is coming onto IRC with same problem, it's probably widespread [16:52:42] just normally via my browser [16:53:36] errors looked like they were cached. vids playing in IE7/IE8 look reasonable [16:53:36] robla: page looks ok though, creating one now i end up with OggHandler [16:53:45] i had simiar messsage on another article yesteray (i think it was about a song). i didn't pursue it cuz i was ony fixing a tiny typo [16:53:51] yeah, just saw that Reedy rolled back [16:53:56] john__: https://bugzilla.wikimedia.org/show_bug.cgi?id=41643 [16:54:26] what is that telling me? [16:54:29] ok...so, *could* be unrelated [16:54:47] john__: I've reported it for the relevant developers [16:54:47] john__: that's a bug report for others, but you can help by adding more about what you were doing [16:54:54] weird that it did not show up on test2... [16:55:15] i'm an IP editor. don't think i can edit that page. but thanks for filing a report [16:55:48] i've made tons of edits on other pages with no problems. [16:56:11] i'm IP at http://en.wikipedia.org/w/index.php?title=Special:Contributions/96.232.126.27&offset=&limit=250&target=96.232.126.27 [16:57:31] Reedy: so undeletion fixed in wmf3? Looks like you deployed something [16:57:39] AaronSchulz: should be [16:57:51] https://gerrit.wikimedia.org/r/#/c/31224/ is still outstanding [16:57:55] for what it's worth, the edits were all quite routine. expand/refine categories. add some references. et al. [16:58:48] i'll leave it at that. thanks for submitting the bug report. [17:00:57] Reedy do you suspect https://bugzilla.wikimedia.org/show_bug.cgi?id=41642 is caused by https://gerrit.wikimedia.org/r/#/c/31224/ ? [17:00:59] AaronSchulz: could you take a look at https://bugzilla.wikimedia.org/show_bug.cgi?id=41642 [17:01:23] j^: and mdale as well, if you have ideas about what's happening there [17:01:23] Nope [17:01:30] I was replying to Aarons comment about undeltion [17:02:14] is PageTriage deployed on test2? [17:02:20] * robla looks [17:02:41] Nope [17:03:23] * robla looks up whose window it is now [17:04:28] preilly: jdlrobson: y'all using the deploy window now? [17:04:42] not that i know of robla [17:05:25] Reedy: could you enable PageTriage on test2? [17:05:30] sure [17:05:33] let's just see what happens :) [17:05:40] I think it needs some schema updates [17:05:40] let me check [17:06:01] Yup [17:07:00] Reedy: I just ECed you on Bugzilla >.> [17:07:35] Reedy: the PageTriage tables are probably going to be missing rows, too. Can the contents of the tables be move over from prod? [17:08:07] what sort of rows? [17:08:23] I notice it has some tag rows.. [17:08:26] ('linkcount', 'Number of inbound links'), [17:08:26] ('category_count', 'Category mapping count'), [17:09:03] robla: nope [17:09:04] Reedy: for example: https://gerrit.wikimedia.org/r/#/c/23382/ [17:09:18] this is my great frustration with PageTriage on beta labs right now [17:09:51] https://gerrit.wikimedia.org/r/#/c/23382/3/sql/PageTriageTagsPatch.sql,unified [17:10:00] ^ Those are in the database [17:10:00] with others [17:10:00] robla: your error looks like it could be https://bugzilla.wikimedia.org/show_bug.cgi?id=39875 not sure that was ported back to 1.21wmf2 [17:10:24] !log Created PageTriage tables on test2wiki [17:10:29] Logged the message, Master [17:10:47] Reedy: yay, they were updated [17:11:28] preilly: thanks, looks like we may just spill over into your window if y'all don't mind [17:12:38] Reedy: can you update TMH to HEAD once more, fixed something for frames on test2 that would be good to have deployed to reduce load on imagescalers [17:13:12] robla: what time will en be updated to 1.21wmf3 [17:13:23] next monay [17:13:25] Monday of next week [17:13:26] monday [17:13:36] monday monday monday!!! ;-) [17:14:05] testing on test2 with 1.21wmf3 and using 1.21wmf2 on enwiki might be the reason for the error [17:14:16] yeah, it's a possibility [17:14:36] so we try again monday after the update to 1.21wmf3? [17:14:59] repro'd on test2 [17:15:16] when I created http://test2.wikipedia.org/w/index.php?title=User:RobLa-WMF/Video_testing2 , I got the same error [17:15:21] that's on wmf3 [17:16:38] so...looks like we've probably got a PageTriage/TMH interaction bug to sort out [17:17:30] dont get any error creating http://test2.wikipedia.org/wiki/User:JanGerber/foobar [17:18:18] TMH master out on wmf3 [17:18:23] pagetriage enabled [17:18:31] :) [17:18:45] woo [17:18:45] exciting to try TMH [17:19:56] Reedy: you pushing it out again? [17:20:01] mdale: just on test2 [17:20:02] IT's pushed already [17:20:39] ok [17:21:07] j^: pretty reliable repro for me. just got it again on http://test2.wikipedia.org/w/index.php?title=User:RobLa-WMF/Video_testing3 [17:21:21] try creating a page with the exact text that I used [17:21:27] j^: IE7 is briefly showing "Sorry, your browser either has javascript disabled..." error message while the page loads. Once loaded, the page looks normal. this is on your 'foobar' page on test2 [17:21:55] a possibility for why I'm seeing it might be to do with permissions [17:22:32] robla: do you have special permissions? [17:22:53] same for IE8 [17:22:54] just created http://test2.wikipedia.org/wiki/User:JanGerber/new with same content and dont get an error [17:22:59] j^: another thing to try: new page. looks like foobar already existed [17:23:03] yea creating new pages seems to trigger it [17:23:43] same error on User:JanGerber/new while page loads, then it disappears [17:24:23] so...we should probably make sure that this has to do with TMH being enabled. Should we temporarily disable TMH on test2, and then try to repro? [17:24:25] Reedy: I think that backtrace comes from 9c19cb6e3851c3a7b1f18850e610637f8e18be4a [17:24:38] chrismcmahon, mdale possibly we can replace the message with a better fallback that looks more like the loaded version [17:24:52] * AaronSchulz kept seeing that same line change in the annotations a bunch of times [17:24:55] j^: yes [17:25:06] like an image with link [17:25:12] Reedy: sometimes parse() returns a string, so that doesn't always work [17:25:21] https://gerrit.wikimedia.org/r/#/q/9c19cb6e3851c3a7b1f18850e610637f8e18be4a,n,z [17:25:23] I guess it could just return it if its a string and call getText otherwise [17:25:24] … I mentioned this in the IE email [17:25:56] it of course only shows up for IE users since everyone else supports the video tag. [17:26:20] yes [17:27:18] dont get the error creating a new page :( [17:29:45] Reedy: https://gerrit.wikimedia.org/r/#/c/31276/2 [17:30:33] chrismcmahon, mdale https://github.com/xdissent/ievms looks like a good way to get IE test vms, if only i had enough free space [17:31:44] alright, back now [17:32:13] j^ I once heard a rumor that Microsoft had offered hosted Win VMs to WMF. I'll follow up on that with Erik. [17:32:44] chrismcmahon: MS was giving away old vms for a while as well [17:33:16] the github link uses VMs provided by MS but makes them useable with VirtualBox [17:34:29] so [17:34:36] return of the job queue [17:34:41] AaronSchulz: robla, deployed... [17:34:46] robla: See if you can do it again? [17:35:05] * robla tries [17:35:08] apergos: we should be at a point we can redeploy 1.21wmf3... so it should go away again.. [17:35:12] btw, I could disappear suddenly if Erik decides to put me on the spot in the metrics meeting :) [17:35:19] worksforme [17:35:33] not robla disappearance, redeploy of wmf3 [17:35:41] heh [17:35:47] Reedy: I'd wait [17:37:12] Reedy: w00t, fix0rd [17:37:33] Reedy: so https://gerrit.wikimedia.org/r/#/c/31224/ is the supposed fix? [17:37:35] * AaronSchulz is skeptical [17:37:46] AaronSchulz: to what? [17:38:15] * AaronSchulz reads it through [17:38:56] https://gerrit.wikimedia.org/r/#/c/31276/ is the fix, right? [17:39:03] yup [17:39:11] AaronSchulz: ^ [17:39:29] Reedy: undeletion [17:39:35] oh, that one [17:39:55] multitasking fixes [17:39:55] AaronSchulz: look at the bug [17:39:56] That's move pages [17:40:23] let's reenable TMH on enwiki [17:40:24] AaronSchulz: indelete is https://gerrit.wikimedia.org/r/#/c/31211/ [17:40:27] *undelete [17:41:09] ok that actually makes sense [17:41:45] robla: done [17:42:08] mdale: j^: back on enwiki [17:42:54] Reedy: cool [17:43:40] OK, once again IE7 never loads the video at File:Folgers.ogv on enwiki and shows the "Sorry..." message. [17:44:27] i get Our servers are currently experiencing a technical problem. creating a new page now [17:45:01] Fataling? [17:45:02] Reedy: ok I guess wmf3 is ok [17:45:08] not getting the TMH player [17:45:11] in chrome [17:45:30] ( which would also mean no TMH for IE ) [17:45:44] oh there it goes... [17:45:50] mdale: might need a page purge [17:45:58] probably have to wait for the 5 min manifest expire too [17:46:06] chrismcmahon: how is IE8 looking? [17:46:12] ( for the resource loader ) [17:46:14] checking IE8... [17:46:30] Reedy: yes fatal, you fixed wmf2? [17:46:46] oh [17:46:46] lol [17:47:06] heh...I forgot to test that. good catch j^ [17:47:33] same error for me too [17:47:41] looks OK in IE8, yay [17:47:41] when creating new pages [17:47:44] give me a minute [17:48:12] so, since this is IE7 only, we need to treat it as high priority, but let's leave it enabled [17:48:24] (not a blocker) [17:49:11] IE7 should display as well ( at least when I tested IE8 in IE7 mode per the fixes yesterday ) [17:49:13] Try now [17:49:30] at a guess, the problem with IE7 is the old version of Java that I have locally [17:49:39] Reedy: looks good [17:49:49] chrismcmahon: also possible [17:50:08] yeah, I suppose we don't even know how many IE7 users will actually get hit by this [17:50:21] IE7 already has pretty small marketshare [17:50:33] mdale j^ I am still seeing the "Sorry..." message while the page loads, though [17:50:33] Reedy: your edit has been saved, no more errors [17:51:03] hard refresh [17:51:25] chrismcmahon: that is the no JS message for IE … I guess I could have put it in a

Seem to be getting quite a few OOMs at teh moment [17:51:39] all the other browsers support