[00:01:43] gn8 folks [02:40:44] TimStarling: i'm not sure if you're aware of this, but there are still a handful of tim starling masks scattered about the SF office, and they are eyeless and terrifying [03:25:17] !log on zhwiki: removing 970k country data jobs [03:25:25] Logged the message, Master [04:11:51] !log on zhwiki: removing refreshLinks2 jobs for templates "Side box", "Sister", "Flag/core" [04:11:57] Logged the message, Master [07:08:59] hm, wow [07:09:50] a few minutes ago, i got http://pastebin.com/VZHh438E when trying to access ar.wikipedia.org [07:11:35] DarkoNeko: i can't help, but i'll email our engineering list [07:14:04] that db looks ok according to dbtree [07:15:01] apergos: thanks for looking [07:16:28] yeah well it's a pretty surface check, ganglia doesn't seem out of the ordinary either [07:17:39] what's the output of 'show processlist'? [07:17:54] i basically don't know anything about our infrastructure for managing mysql [07:17:58] yeah I see that error, it's the only one in the db error log, and there's only two errors total [07:18:09] well it's not going to show much now ori-l [07:18:38] do we use anything for monitoring deadlocks like the stuff in the percona toolkit? [07:18:58] I doubt it but ash er is the one to say [07:19:00] also anything like joining tables in alphabetical order etc? [07:19:34] alphabetical order? I can't imagine so [07:20:47] those two errors are from more than a half an hour ago, so at this point it's going to be hard to figure ou t the cause (for me anyways) [07:21:17] yes, my suggestions are from 30,000 feet and most likely useless [07:21:31] well, if things are okay now, they're okay now :) [07:23:35] atop shows nothing in particular form that time period that's interesting [07:24:22] there is a tiny bit of swap in use but that seems to be a constant factor [07:24:52] giving up on it, sorry I couldn't be more helpful [07:25:05] more helpful than i was :) [07:26:03] * apergos needs to wake up and get out of pjs [07:26:05] it's showing normally now [07:26:17] well, what surprised me is that usually the SQL query is hidden [07:26:21] hmm [07:26:32] the [removed] are from me in the pastebin [07:26:40] right [07:26:45] is there a dev or someone on the engineering team that could take a look at a wikitech page on wiki renaming? [07:27:07] at this hour most devs are sleeping I guess [07:27:16] I suppose [07:27:17] (and no I am not going to look at it :-P) [07:27:25] even in west europe, it's not even 9AM [07:27:32] I offer more chocolate :) [07:27:54] it's 9:30 am here but I'm still somewhat groggy, on line working too late lasrt night [07:28:04] aw [07:28:22] but my queue has other things in it, wiki renaming is not something on my plate [07:29:41] oh I am sure [07:30:10] there is a procedure proposed on wikitech and I merely want people to consider if it looks ok, and perhaps run a few tests [07:33:38] page is https://wikitech.wikimedia.org/view/Rename_a_wiki should you change your mind [07:34:55] ToAruShiroiNeko: send it to wikitech if you want people to look at it [07:35:32] p858snake|l I have [07:35:36] its on the mailing list [07:35:38] if memory serves correctly, there are issues with that propsed solution and it needs a bit more work [07:35:39] no one replied [07:35:53] I can bump it [07:36:02] when did you send it? [07:36:57] wikitech-l ? [07:37:15] http://wikimedia.7.n6.nabble.com/wikitech-l-Renaming-wikis-td4987899.html [07:37:56] That was ocrober 23 [07:43:19] october even [07:47:28] "'wikibooks' => 'monobook', // Value for all wikis listed in /home/wikipedia/common/wikibooks.dblist", I beg to differ [09:12:20] hi mark [11:39:12] * ToAruShiroiNeko ponders if anyone is around [11:39:16] * ToAruShiroiNeko drops a pin [11:39:27] pin missle? [12:40:55] !gerrit test [12:41:02] :/ [12:41:03] !g test [12:41:34] !bz is test [12:41:34] Unable to modify db, access denied, link to database isn't valid [12:41:43] aha [12:44:20] petan: testing a new bot / new bot feature ? [12:44:36] no, testing old bot feature on new version of core [12:45:13] MW core ? [12:45:36] Bot core ? [12:45:48] hiya hashar [12:47:08] bot core [12:47:11] !bz is test [12:47:11] Key was added [12:47:14] !bz [12:47:14] test [12:47:17] !bz del [12:47:17] Successfully removed bz [12:47:19] !g test [12:47:19] https://gerrit.wikimedia.org/r/#q,test,n,z [12:47:22] yay [12:47:23] fixed [12:48:11] Cool [✔] [12:49:11] !bz alias bugzilla [12:49:12] Alias is already existing [12:49:17] :o [12:49:20] !bz test [12:49:21] All bugs in MediaWiki should be reported at https://bugzilla.wikimedia.org. Requesting site configuration changes, new features or enhancements to existing features all go here. Bear in mind before making a feature/enhancement request: 1) If the request is specific to a Wikimedia wiki, please discuss it there first. 2) Consider whether a custom extension would be more appropriate [12:49:37] !b 1 [12:49:37] https://bugzilla.wikimedia.org/show_bug.cgi?id=1 [12:49:52] seems to work as desired... [12:50:48] * jubo2 declares yet another "We ♥ MediaWiki devels"-day [12:51:47] Kurt Gödel and Max Weber are the other people that get love days [12:52:25] pro-tip: If run into evil AI tell it to go read up on it's Gödel first thing off. [13:56:10] andre__: sorry for the silly bug ^^' cookie makes me feel stupid [13:56:47] Nemo_bis, heh, trust me, you will see WAY more stupid comments and bugs by me than by you. Promised. ;) [14:26:58] Nemo_bis: do you have access to WMF bots server? [14:27:16] I need coibot restarted [14:29:09] no I don't [14:29:11] ask petan [14:29:21] hi [14:29:42] sDrewth hi, you know who is operator of coibot? [14:29:49] because I do not :D [14:29:56] sure, Beetstra [14:30:07] the instructions to restart the bot are in the root [14:30:09] I don't really like to touch applications of someone else, without their permission or guidance [14:30:16] and it is on instance3 [14:30:27] ok, ok... but why we need to restart it? [14:30:27] permission is at m:user talk:Beetstra [14:30:37] let me check it [14:30:49] b/c the beast has disappeared from #wikimedia-external-links [14:31:36] petan: pretty much Beetstra and I work on the bots output together [14:31:42] ok [14:32:02] so, how do I restart it? you say instruction are in root [14:32:08] and I just have never got off my arse to get my bot access sorted properly :-( [14:32:22] what does it mean? :) in root of beetra's home? or what? [14:32:28] in the user directory, he mentions it on that page [14:32:34] ok [14:32:58] it seems that whole bots-3 server is dead [14:33:04] I will have to reboot it [14:33:12] :-) [14:33:13] For ref: http://bots.wmflabs.org/~hydriz/minimanual.txt [14:34:08] hmm [14:34:16] bots attack! :p [14:34:23] Hydriz: do you have access tehre? [14:34:51] Hydriz you are missing the ArticlesForCreationBot :D [14:34:57] that lives on bots-1 [14:35:11] sDrewth no one has the access there cuz the box is dead :P [14:35:14] working on recovery [14:35:21] I meant in general [14:35:31] it means another person that I can pester :-) [14:35:41] sDrewth: I do, but my adminship was revoked, not sure why [14:35:54] petan: That was just a snapshot :P [14:35:57] * sDrewth grumbles [14:37:29] Hydriz I don't know why you even need that? everyone has root by default on bots-3 [14:37:42] that's why the box is dead all time [14:38:14] yep, but its best to do such tasks as a proper admin instead of a normal user and prying into others' stuff :) [14:38:46] there is no difference between "proper admin" and normal user on bots-3, the configuration of sudo is same there [14:40:32] Its just respecting others' privacy, but anyway lets just not talk about this [14:50:15] okay it's back up [14:50:21] I mean the machine :P now let's fix the bort [14:50:23] * bot [14:53:24] yea! thx petan [14:53:40] does it work? [14:53:53] it doesn't say which application user should be used, so I used beetstra [14:54:17] but other bots that were running there are likely dead now [14:54:33] in fact they were dead all time [14:54:53] sDrewth: No reports in 01 minute 45 seconds. [14:54:54] * xlinkbot - on bots-3 [14:54:55] * unblockbot - on bots-3 [14:55:06] it talks [14:55:12] these likely needs to be started as well [14:55:19] hmm, let me see if I can them to talk [14:56:16] @seen andrewbogott [14:56:21] @seen-on [14:56:21] Seen is now enabled in the channel [14:56:23] @seen andrewbogott [14:56:23] petan: Last time I saw andrewbogott they were talking in the channel, but they are not in the channel now and I don't know why, in #wikimedia-labs at 11/14/2012 1:16:39 AM (13:39:44.1376080 ago) [14:56:39] he doesn't know why :( [14:56:45] that's sad [14:57:45] I think it's because he changed a nick heh :P [15:01:00] @seen closedmouth [15:01:00] closedmouth: I have never seen closedmouth [15:01:07] ;_; [15:01:14] wtf [15:01:16] @seen petan [15:01:16] petan: are you really looking for yourself? [15:01:32] oh the space after [15:01:37] @seen closedmouth [15:01:37] closedmouth: are you really looking for yourself? [15:01:51] @seenrx closed.* [15:01:51] petan: Last time I saw closedmouth they were talking in the channel, they are still in the channel #wikimedia-tech at 11/14/2012 3:01:37 PM (00:00:14.3316290 ago) [15:02:25] btw @seenrx .* is a good way to ping some people... :P [15:39:33] hello [15:39:52] https://upload.wikimedia.org/wikivoyage/en/thumb/4/47/Map_of_Agra_Fort.jpg/763px-Map_of_Agra_Fort.jpg [15:40:07] I get the message: 401 Unauthorized [15:40:07] This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required. [15:40:07] Token may have timed out [15:40:20] the full image displays OK [15:41:01] http://en.wikivoyage.org/wiki/File:Map_of_Agra_Fort.jpg no thumbs [15:45:01] hm I think we had the same bug for that ja.wiki image of a nazi [16:16:11] yannf: either some late wikivoyage migration issue (maybe ask in their channel), or a general thumbnail issue (see https://bugzilla.wikimedia.org/show_bug.cgi?id=41371 ) [16:17:24] andre__, does not work on French WV either [16:17:32] http://fr.wikivoyage.org/wiki/Fichier:Map_of_Agra_Fort.jpg [16:18:52] what's the process for asking for enabling the import tool? http://fr.wikivoyage.org/wiki/Sp%C3%A9cial:Importer [16:20:28] yannf: which one? [16:20:43] https://meta.wikimedia.org/wiki/Importer [16:21:26] is it not a right issue [16:21:40] the message says that the tool is disactivated [16:22:14] impossible [16:22:50] it's not disabled [16:22:57] but you're not importer [16:23:06] and probably no transwiki import sources are defined [16:23:24] I'd really ask on the wikivoyage channel [16:23:29] No transwiki import sources have been defined and direct history uploads are disabled. [16:23:43] I am admin there [16:24:38] admins don't have importupload right [16:24:50] have you read the page I linked you? ;) [16:27:50] yes, it says: "Besides transwiki importers, the 'import' userright is assigned by default to sysops on all Mediawiki projects." [16:28:18] so? [16:28:22] it's what I just said [16:28:35] (it would be weird if it didn't as I wrote it) [16:29:29] what rights do I need to import templates from Commons to WikiVoyage? [16:29:47] who can give these rigths? [16:30:17] yannf: there are no import sources defined [16:30:28] it must be set first [16:30:34] ok, who can define import sources? [16:30:45] sysadmins [16:30:53] this is written on the page as well btw :p [16:31:01] I think we need Commons and French Wikipedia [16:31:03] you don't need import for templates, though [16:31:23] Nemo_bis: why not [16:32:12] Danny_B|backup: https://meta.wikimedia.org/wiki/Terms_of_use#7._Licensing_of_Content – c2 [16:33:40] well manually importing templates from Commons in a pain in the ass [16:33:52] import for templates is only useful if you want to do recursive transwiki import [16:34:03] these have 100s of subtemplates [16:34:16] Nemo_bis: that doesn't say anything about not importing templates [16:34:20] who said manually; you can use pagefromfile.py [16:34:28] Danny_B|backup: I said "don't need" [16:34:58] yannf: if you need it now, try #wikimedia-stewards [16:35:11] ok [16:35:54] but "recursive" export doesn't manage to really export all needed templates, usually [16:36:41] anyway, i filed a request for it in the meantime [16:38:34] yannf: *please* don't import babel templates ;) [16:39:26] ok, we don't need these [16:39:39] :) [16:40:39] they haven't been deleted yet everywhere? [16:43:31] Danny_B|backup: almost nowhere AFAIK [16:43:51] and some wikis didn't even disable them by changing the babel extension MediaWiki: config [16:44:30] i vaporized them on all projects i have rights on [16:45:16] yes, adminbots did the job elsewhere too [16:46:59] babel extension has one major issue - bad default stylesheets. [16:50:43] Danny_B|backup: commit a patch? :p [16:51:07] when it was deployed, Robin almost immediately applied a colour change proposed on talk [16:52:17] it's not only color, but other things as well [16:54:00] it was an example of how things are easy to fix [16:54:26] current color is also not good [17:05:24] sigh [17:05:43] wassup, andre__ ? [17:05:58] I just spotted the regex .*@wikimedia.org in Bugzilla. [17:06:12] and I think this one really does not do what the author meant to express. [17:42:45] Hallo. I get a MWException if I try to go to the preferences at the incubator. [17:42:59] aharoni: Known bug [17:43:12] OK. [17:45:06] do not hang up, please, you're in the queue [17:47:26] Your bug is very important to us [17:48:59] :) [17:49:03] lol [17:50:22] * aharoni realizes that these responses Hebrew are the same, and that they were... localized. [17:50:33] ^in^ Hebrew [18:15:51] anomie: ping :) [18:15:54] anomie: dinner time for me, if you are in SF would you mind talking about merging multi version in mediawiki-config with Aaron ? [18:16:12] anomie: would make all your changes nicer to handle and probably safer to handle in the long time [18:16:17] hashar- In a meeting right now, but I will once I get out [18:16:23] nice :) [18:16:38] will connect again around noon PST to give a shot at your comments [18:17:07] the merge could bedone by creating an orphan branch in mediawiki-config and then merge it , should keep history this way [18:17:11] have a good meeting! [18:47:57] ^demon- ^^ Any idea how to do what hashar is talking about there? [18:48:42] <^demon> No, I'm not aware of the context. [18:52:36] ^demon- The idea is to just merge operations/multiversion into operations/mediawiki-config instead of having it be a submodule, since they're so tied together anyway. Ideally without losing history. [18:54:51] <^demon> http://nuclearsquid.com/writings/subtree-merging-and-you/ it can be done. [18:54:55] <^demon> It's called a substree merge. [18:55:39] <^demon> subtree, even [18:59:12] ^demon- and will Gerrit manage to not get confused by doing that? [19:00:21] <^demon> It should handle it fine...it'd just be a normal commit once you've gone through the steps. [19:02:43] AFK for a little while... I'll get on with the deploy when I get back [19:29:39] ^demon- Tried following the instructions on that page, and then tried the command at [[mw:Git/Workflow#git review doesn't like merge commits]] to send it to Gerrit. It complained at me about other people's email addresses on the commits in the merged history. [19:29:58] Oh, yeah it does that [19:30:08] <^demon> Yeah, we'll need to grant Forge Committer temporarily. [19:30:15] If you need to submit others' commits, you need to get hi to grant you the ... yeah [19:30:44] <^demon> You could grant it too, you know ;-) [19:31:19] * RoanKattouw is lazy [19:33:34] * anomie would be happy to just let someone else commit the changeset to Gerrit [19:37:21] Reedy: you ready to deploy wmf4 now? [19:38:53] robla: 21:02 < Reedy> AFK for a little while... I'll get on with the deploy when I get back [19:39:00] (36 minutes ago) [19:39:03] Yeah [19:39:08] ah. [19:51:05] All looks quiet [19:51:14] yup [19:51:54] Reedy: deploy everywhere else? (that we're supposed to) [19:53:23] meta already done? [19:53:44] yup [19:53:59] Still only image related warnings [19:54:01] I'll do the rest now [19:55:21] Who's handling the wmf4 deployment today? [19:55:36] Me ;) [19:55:38] Just about done, I think [19:56:09] Cool, I was just going to ask you to ping me when it's complete, so I can make sure there are no problems with UploadWizard on Commons [19:56:23] since there are a lot of back-end changes to it [19:56:52] it seems to be working well on test at least :) [19:57:00] heading out to lunch now....I'm assuming things are in good hands [19:59:50] i think the missing next button issue should be gone now as well for UW .... [20:00:24] kaldari: I click on the "upload from Flickr" button has no effect :-( [20:00:47] -I+A [20:01:05] FF17/Win 7 [20:01:37] which button? "Add images from Flickr" or "Upload images from Flickr"? [20:01:55] er sorry "Get images from Flickr" [20:02:00] https://commons.wikimedia.org/w/index.php?title=Special:UploadWizard [20:02:08] sorry, I have the German interface [20:02:23] There are 2 different buttons [20:02:32] The 1st one or the 2nd one? [20:02:41] that mentions Flickr [20:03:06] "Add images from Flickr" [20:03:35] me too [20:03:36] hrm [20:03:39] ack [20:03:45] something is very wrong there [20:04:14] huh ? [20:04:20] that's not supposed to be enabled. [20:04:51] yeah, that button shouldn't be visible yet on Commons. I thought you were talking about test.wikipedia [20:05:16] https://commons.wikimedia.org/w/index.php?debug=1&title=Special:UploadWizard [20:05:22] not visible with debug=1 [20:05:46] how are you deciding whether to load or not? [20:05:49] maybe the CSS hasn't synced yet [20:06:16] one sec.... [20:06:28] need to do an emergency fix real quick... [20:08:50] ok, gone now [20:11:40] the button was also unstyled since the css hadn't syched yet [20:12:18] The problem was the HTML for the button in the PHP didn't have display:none inline, but was rather relying on external CSS [20:12:47] race conditions, fouc, lalala [20:18:50] ori-l: and I believe the PHP gets copied over before the CSS and JS [20:29:12] hey people, i just submitted a change to https://bugzilla.wikimedia.org/show_bug.cgi?id=42097 that sould fix it, and it's marked as high critical [20:29:16] would be nice if somebody looked at it [20:29:28] (a JS patch to $.tablesorter) [20:34:05] anomie: could you take a look at this patch? ^ [20:34:15] Already am, actually ;) [20:34:24] excellent, thanks! [20:35:34] it looks at a minimum that it can't do any harm [20:36:07] ugh, i just noticed that what i wrote above did not make any sense. :P [20:36:34] MatmaRex: how so? [20:38:23] "i just submitted a change to that sould fix it" [20:38:27] how can one submit a change [20:38:28] to a bug [20:38:33] that should fix it [20:38:39] well, nevermind :) [20:38:42] ah...I got what you meant :) [20:40:34] anomie: if you're currently reviewing https://gerrit.wikimedia.org/r/#/c/33417/ , i just fixed a typo in the commit message [20:49:43] (i'm off for now; if there's anything in the patch that needs changing, please do it) [20:50:22] MatmaRex- Code is good, comment and commit summary need work. [20:52:37] AaronSchulz: don't you think queued jobs are dropping? https://gdash.wikimedia.org/dashboards/jobq/deploys [20:52:57] robla: is someone checking that wmf4 didn't break job queue? [20:54:00] Reedy: shoot, we forgot to check the blockers [20:54:00] anomie: i don't have time now [20:54:23] MatmaRex- Per your comment above, should I just make the changes I suggested for you? [20:54:25] anomie: there's a larger comment on the bug [20:54:33] yes, go ahead [20:56:51] robla: nothing that looks to be a new issue in wmf4... Job runners look to be doing a decent mix of everything [20:58:16] bar maybe the tablesorter thing depending on when it actually happened [20:59:58] Reedy: I think Nemo_bis makes a pretty good point by highlighting this: https://gerrit.wikimedia.org/r/#/c/33411/ [21:00:03] er.... [21:00:08] https://gdash.wikimedia.org/dashboards/jobq/deploys [21:00:17] anomie: thanks [21:00:51] job queue did drop off quite a bit coincident with the deployment [21:01:25] I don't recall if there's any good reason for the queue dropping (e.g. did any deduplicaiton code make it in?) [21:01:48] robla: Tim said something about removing jobs today [21:03:11] nighty [21:03:29] gotta make a phone call now. I suppose there's not an emergency around the job queue, so we can just keep an eye on it [21:05:16] paravoid: that was 17 hours ago [21:21:11] AaronSchulz: should we be worried about the state of the job queue? [21:21:17] https://gdash.wikimedia.org/dashboards/jobq/deploys [21:22:32] actually, is that ^ an aggregate count across all wikis, or just enwiki, or something else? [21:22:43] no idea [21:22:46] http://ganglia.wikimedia.org/latest/graph_all_periods.php?c=Miscellaneous%20pmtpa&h=spence.wikimedia.org&v=3190&m=enwiki_JobQueue_length&r=hour&z=small&jr=&js=&st=1352928149&z=large [21:23:04] enwiki looks alright (not wmf4) few thousand jobs [21:23:16] <^demon> It looks fine over the 1wk period. [21:23:54] 3k jobs is nothing [21:24:32] the jobs log on fluorine looks ok [21:24:33] http://ganglia.wikimedia.org/latest/?r=hour&cs=&ce=&s=by+name&c=Jobrunners%2520pmtpa&tab=m&vn= [21:24:37] they look a little lazy [21:25:15] Hmm [21:25:18] frwiki has 0.5M jobs [21:25:27] zhwiki has 163k [21:26:44] yeah, this looks a little suspect: http://ganglia.wikimedia.org/latest/graph.php?r=day&z=xlarge&c=Jobrunners+pmtpa&m=cpu_report&s=by+name&mc=2&g=network_report [21:26:45] ^demon: it's so erratic it's hard to say what normal is :) [21:27:27] Someone should write a script to get a job count for the entire cluster [21:27:29] the dropoff seems to be right around the time we deployed wmf4 a lot of places [21:28:50] between 19:48 and 19:54 [21:30:46] errm [21:36:23] new jobs not getting queued again? or some job types not running? [21:36:25] gah [21:36:39] * apergos slaps self for even thinking of work at this hour [21:36:48] nevermind me I will read about it later in the scrollback [21:37:29] +1 [21:38:19] Hacky script is hacky [21:38:27] apergos: you could also hire someone to slap you every time you read job queue discussions after work hours http://hackthesystem.com/blog/why-i-hired-a-girl-on-craigslist-to-slap-me-in-the-face-and-why-it-quadrupled-my-productivity/ [21:39:07] hahaha [21:40:24] How long do we think it'll take to do 830+ sql queries (one per wiki) [21:43:30] robla: sooo [21:43:32] on the cluster [21:43:34] int(977106) [21:43:40] Just under 1 million jobs, it would seem [21:44:00] real 3m46.802s [21:48:19] Is there an issue with logins right now? [21:48:43] No.. What's up? [21:48:52] and welcome back ;D [21:49:18] Hi. :) Couldn't log in with my regular acct, reset password, reset password still will not allow my log in. Tried in two different browsers. [21:53:02] heya neilk_ ! [21:53:14] hi robla [21:53:16] * robla tries independent test [21:53:37] 40 echo related fatals [21:53:39] * Reedy grumbles [21:53:56] 2 different ones I've fixed today [21:54:06] neilk_: which wiki are you trying to log into? [21:54:12] robla: en [21:54:40] * robla tries same [21:56:55] password reset on test account seems fine for me [21:57:15] yes, same for me too [21:57:23] it's only my regular account NeilK [21:59:39] * robla doesn't see anything obvious [22:00:03] I can't think of anything that could cause this on the server side, like some sort of block on my account, either. [22:00:10] I've got another thing to go to...sorry I couldn't help more. might be an opsen to help somewhere [22:00:19] or the old fashioned otrs route [22:00:31] no worries thanks anyway robla, Reedy [22:01:45] neilk_: you've been locked [22:01:51] Vito: wanna unlock him? [22:02:10] neilk_: Philippe told Daisy to lock (~globally block) you [22:02:34]