[01:13:08] good day [01:13:38] I was wondering if someone can help, there is a user that lost his password and forget which e-mail he used to register, is there a way to retrieve that e-mail adress ? [01:14:15] try all his email addresses, otherwise no [01:14:21] It is technically possible, whether this would be done, I doubt it. [13:52:42] anybody with shell access around? need to check some error in database [13:53:13] * Reedy sighs [13:53:17] Yes? :P [13:56:00] Reedy: pm [17:01:01] BTW whosoever responsible for image scalers, you've got more problem https://commons.wikimedia.org/wiki/Commons:Village_pump#More_image_scaler_problems [17:03:24] Might be another swift 404 returning with a 200 code [17:05:32] hi, can anybody help me lifting user creation restrictions for na IP? [17:05:46] it's kind of urgent... [17:06:20] Is there a bz for it? [17:06:59] nope, they forgot do bz it and the workshop has already started [17:06:59] they tought each machine would have its own ip [17:09:21] Damianz: the ip would be 143.107.79.204 [17:11:46] Probably need someone like Reedy or someone from -ops. I can't find the page for 'I didn't make a ticket, it's important' steps =/ [17:12:58] https://bugzilla.wikimedia.org/show_bug.cgi?id=41355 [17:13:12] has ticket now [17:13:15] :) :P [17:13:23] Damianz: ping [17:14:57] * Damianz pokes paravoid or LeslieCarr [17:15:03] eep [17:15:05] help me! [17:15:07] :o [17:15:11] ok fine :p [17:15:22] solstag_: i'm getting at it [17:15:32] :) [17:15:33] LeslieCarr: :D [17:15:46] Leslie is allways around when you need something [17:15:53] sshhhhhh [17:16:43] Typically I now just found the page that has the 'if filed on short notice' steps. [17:16:57] haha [17:18:10] solstag_: how many peeps [17:18:14] ugh why do people forget these things until the last minute [17:18:40] because they really want to send me nice bottles of whiskey …. hint, hint [17:19:06] eh, i'll set it at 75 [17:19:06] haha [17:19:20] wait, I just realized I forgot an important piece of info [17:19:20] solstag_: which wiki ? [17:19:22] Krenair: So they can jump up and down and yell a lot [17:19:22] its not en.wiki, its pt.wikiversity [17:19:48] LeslieCarr: I don't think there's enough whiskey in the world to cover your floor of the office :D [17:19:53] whiskey tub!!! [17:20:03] we have around 25 people here, more may arrive but 75 is enough [17:20:34] just one more question... [17:20:38] scotch or bourbon? [17:21:15] hrm, I would say bourbon … my scotch collection is pretty full [17:21:15] and i love manhattans [17:21:22] pushing the throttle now [17:21:35] ok, now i need to remember how to do the stupid memcache deleting key thing. .. [17:23:03] Bourbon, you heathen [17:23:17] * Damianz keeps his Scotch collection on the side [17:23:59] ok, looking up the memcache delete key chat logs ... [17:24:26] oh... kay, we're awaiting orders. rs [17:25:10] <^demon> LeslieCarr: Just a single key? [17:25:24] yeah [17:25:28] i know it's the mcc.php script [17:25:38] but the single throttle key [17:25:52] <^demon> Oh, always have to look that up again. [17:25:52] I thought there where 2 keys [17:28:29] Should be ptwikiversity:acctcreate:ip:143.107.79.204 [17:28:51] thanks [17:29:07] solstag_: try it now ? [17:29:26] it takes a (irc based) village to create an account :) [17:29:35] trying.. [17:30:44] it's working! [17:31:08] woot [17:31:21] thanks guys, +42 internets and a bottle of bourbon to the village ;) [17:32:44] :) [17:34:15] ops [17:34:19] LeslieCarr: it seems it worked once [17:34:29] LeslieCarr: but then no more [17:34:33] gah [17:35:36] getting the 6 accounts error again...don't know if this helps, its a similar bug that was solved... [17:35:37] https://gerrit.wikimedia.org/r/#/c/25514/1/wmf-config/throttle.php [17:36:07] that was linked from here: https://bugzilla.wikimedia.org/show_bug.cgi?id=40575 [17:37:04] ok [17:37:18] and double checking, it's all from the 1 IP, on pt wikiversity, yeah ? [17:37:38] yes [17:38:04] this time its all from 143.107.79.204 [17:38:18] ok, trying as an array instead of a single variable ... [17:39:28] ok tried again solstag_ [17:39:43] let's see if this works.. [17:40:41] solstag_: any happier ? [17:41:09] ^demon are there any other keys it could be looking at ? [17:41:44] <^demon> Off the top of my head, no. [17:43:26] <^demon> If any exist, they'd be in SpecialUserLogin. [17:43:37] <^demon> Or whatever it's called [17:43:55] so, apparently its working now! =) [17:45:02] cool [17:52:30] we should probably document that on wikitech [17:52:41] there's a throttle page already [17:56:54] http://wikitech.wikimedia.org/view/Increasing_Account_Creation_threshold [18:02:50] Reedy: updated [18:02:50] thanks [18:02:55] Thanks :) [18:06:10] TBloemink: ping [18:06:13] !ops [18:06:24] lol [18:07:03] He's quite lucky I don't have ops here [18:12:54] Hmmm [18:13:11] robla: there's a load of warnings from wfMerge() appearing.. [18:13:39] * robla reads server admin log backlog [18:14:06] Some from Collection too [18:14:19] Oh [18:14:20] it's one poorly apache [18:15:44] Reedy: scp failure or something? [18:16:17] notpeter: would this be accurate to say in a comment in bug 27699? "Peter's email "[Wikitech-l] All wmf imagescalers now running ubuntu 12.04 precise" clears up that blocker: http://lists.wikimedia.org/pipermail/wikitech-l/2012-October/064019.html We'll watch the imagescalers for issues for the rest of this week before declaring that we can go forward with the TMH deployment." [18:16:36] (bug 27699 is about the TimedMediaHandler deployment) [18:16:40] I would've suspected it to have a full file system, but its ok [18:16:50] robla: writing text to files to diff them etc [18:16:58] sumanah: sure. I'm fine one way or the other. [18:17:04] I just wanted to give as much info as possible [18:17:10] so that people could make the right decisions [18:17:16] yeah, thanks for that [18:17:19] great email [18:17:23] !log Running sync-common on mw40 [18:17:24] for the hell of it [18:17:28] thanks :) [18:17:33] Logged the message, Master [18:17:37] notpeter: thank you for the upgrade & the email! I added a "probably" to my comment and posted it. [18:18:05] heheh, ok :) [18:22:46] one minor correction from notpeter's mail. That was the final *known* issue blocking TMH :) [18:22:56] HA [18:24:11] so we still have to unknown unknowns? [18:24:12] :) [18:24:16] *the [18:27:09] robla: Bar that apache all looks good. Just need to get someone in ops to kick mw40 [18:34:30] Reedy: aside from a restart of apache, does mw40 need anything else? [18:35:24] I'm not quite sure what's up with it [18:35:36] I thought it was going to have a full filesystem or something, but there's room on all partitions [18:36:13] yeah [18:36:31] Have you restarted apache on it? [18:37:03] yeah, but it's depooled, atm [18:37:47] meh, I'l re-pool and see what happens [18:37:53] (I'm the best at science...) [18:37:57] what are you guys working on? Reports of broken "recent changes" and watchlists in #wikipedia-de ... [18:38:08] Broken how? [18:38:39] [1853f2d6] 2012-10-24 18:36:18: Fatal exception of type MWException for example [18:38:52] Right [18:38:53] https://www.dropbox.com/s/8hvkkxhwb6utpu2/wikipedia.png for example [18:39:09] Have they got large watchlists? [18:39:19] I'm suspecting the recentchanges one is link batch related [18:39:48] probably big, yes... mine, for example ;-) [18:40:09] recentchanges seems to not work when you're logged in, but does work when you log out [18:40:40] You mean, it doesn't work if you're a sysop? [18:41:01] https://de.wikipedia.org/wiki/Spezial:Letzte_%C3%84nderungen [18:41:01] WFM logged in [18:41:23] https://bugzilla.wikimedia.org/show_bug.cgi?id=37209 [18:41:24] doesn't work for me, for others neither... normal user, no admin [18:41:48] 2012-10-24 18:41:38 mw47 dewiki: [6f169a9d] /wiki/Spezial:Beobachtungsliste Exception from line 2956 of /usr/local/apache/common-local/php-1.21wmf2/includes/Title.php: LinkCache doesn't currently know about this title: Staatliche_Lehr-_und_Versuchsanstalt_fÃ¼r_Wein-_und_Obstbau_Weinsberg [18:41:52] Yeah [18:41:59] Just logged that on BZ [18:42:02] [4cbb8817] 2012-10-24 18:41:51: Fatal exception of type MWException [18:42:08] There's an UploadStash exception too but it's not giving me a backtrace [18:42:22] exception with no trace? [18:42:37] It's not a fatal [18:42:42] RoanKattouw: it's a dupe [18:43:00] Niklas logged this back in may [18:43:11] all link cache saying no [18:43:42] AaronSchulz: Yeah, probably because it's being caught and rethrown or something. I tried using MWException directly instead of a custom exception class but no luck [18:44:56] AaronSchulz: 2012-10-24 18:44:45 srv300 commonswiki: [2bbf4a48] /w/api.php?format=xml&action=upload Exception from line 112 of /usr/local/apache/common-local/php-1.21wmf2/includes/upload/UploadStash.php: key 'rsion=' is not in a proper format UploadFromChunks::continueChunks/UploadStash::getMetadata/UploadStash::getFile [18:45:09] Hmph [18:45:25] Need to pass a higher limit to wfGetAllCallers() I suppose [18:45:45] I've also logged that bug [18:46:23] OK now I have the full one 2012-10-24 18:46:03 mw70 commonswiki: [2f5ca212] /w/api.php?format=xml&action=upload Exception from line 112 of /usr/local/apache/common-local/php-1.21wmf2/includes/upload/UploadStash.php: key 'rsion=' is not in a proper format require/ApiMain::execute/ApiMain::executeActionWithErrorHandling/ApiMain::executeAction/ApiUpload::execute/ApiUpload::selectUploadModule/Upload [18:46:23] FromChunks::continueChunks/UploadStash::getMetadata/UploadStash::getFile [18:46:24] Reedy: bug #? [18:46:29] LeslieCarr: one of you config commit summaries was just a number, kind of stood out ;) [18:46:38] must have been a bug # of course [18:46:53] hehe [18:47:06] https://bugzilla.wikimedia.org/show_bug.cgi?id=41306 [18:47:08] Logged yesterday [18:48:50] binasher: mc1 is running redis, but it is still up with memcached and ok to use I asssume [18:51:06] AaronSchulz: that redis instance on mc1 is going to be reconfigured, but they are all going to run a tiny redis instance and a large memcached. memcached is configured for prod use though [18:51:56] makes sense [18:52:48] Fscking irc client [18:52:55] RoanKattouw: where did you just ping me? [18:54:56] Does anyone know of a status monitoring system that does not cost very much and is easier to install and configure then say.. Nagios ? I've heard that status.wikimedia.org is bought from an external party. I would like status monitoring, reporting and notification to run on http://status.consumium.org [18:55:38] to stalk one server .. http://develop.consumerium.org/wiki/ [18:56:46] plwiki reports [6753e84d] 2012-10-24 18:48:37: Fatal exception of type MWException [18:57:15] ah ok, seen scrollback [18:57:47] jubo2: google analytics may be useful for that ? [18:58:45] LeslieCarr: They provide status monitoring, reporting and notification ? I did not know. [18:59:10] So I googels for "google analytics" [18:59:10] i don't think notification [18:59:13] but there's some status monitoring [18:59:39] everything else i have used is either non free or non easy [19:01:14] LeslieCarr: tell me about it.. The various apps that offer to configure your Nagios for you ( like wikimafiafoundation does with puppet ) were broken and difficult [19:01:43] P-; [19:02:06] jubo2: If you're just looking for server up/down, and ok with free as in beer, I've been happy with pingdom.com [19:02:46] csteipp: tnx. will check pingdom.com [19:13:40] regarding the watchlist issue; you already aware of the causes? some users are reporting at dewiki village pump that reviewing pending changes via http://de.wikipedia.org/w/index.php?title=Spezial:Seiten_mit_ungesichteten_Versionen&watched=1 solves the issue. [19:15:50] csteipp: I registered the free pingdom.com account with the 20 free beer... SMS'es [19:17:05] great, I hope that works out for you-- I just have it email me whenever checks fail, and it has nice pretty graphs of response times [19:33:00] csteipp: I set it to SMS me when down for 5 or more minutes [19:38:42] I received a worth 50€ of free google adwords ads and I fill the form to receive the benefit [19:40:17] Then a few days later someone from Google phones me up .. I'm like " I saw an ad for free shit and stuff and decided to click it coz I have a website or wiki to promote." [19:41:17] Then we talk with the google representative and come to the conclusion that I should contact them about free stuff and shit when Consumium gets to the implementation stage after 11 yrs of development stage and they will arrange something for me [19:43:02] Guys... It's difficult to sysop a wiki and keep the wiki from going into lockdown if you get inprisoned by the spygiatrists for minorest things.. [19:43:55] We are going to need a Wikidata-like setup to store among other facts the numbers required to form and display the http://develop.consumerium.org/wiki/Consumerium:Metrics [19:44:25] and then teh Translatewiki .. we gonna use that and Universal Language Selector and that's about it [19:49:08] as far as setting up wikis, the #mediawiki channel is more targeted for that jubo2 -- this channel is more like specific techie issues that relate to the wikimedia foundation's installation base [19:50:15] LeslieCarr: I know, that's why I shut up 99,5% of the time to not cause unuseful channel traffic that is perceived distractive [19:50:25] because bz sucks at tracking techie issues :) [19:54:17] Surely someone just doing an endless monologue on subject X non-stop since 2003 and spamming benefit X is just noise that triggers the content notification in irc clients which causes irritation ie general populus of channel devoted to technologically enabling cause Y [20:16:53] something broken for me [20:17:05] Internal error [20:17:05] [6c6470c9] 2012-10-24 20:13:39: Fatal exception of type MWException [20:17:20] I get this when open RecentChanges or Watchlist [20:18:01] if I login with other account (bot) it works good [20:24:44] Eloquence: robla: someone asked about moving future tech meetings to #wikimedia-office (I forget where/who). can we do that? also can we get a public ical/hcal feed like with the other meetings? (or just add the tech meetings to the existing cals) [20:26:14] * Dispenser notifies AaronSchulz about https://commons.wikimedia.org/wiki/Commons:Village_pump#More_image_scaler_problems [20:26:32] jeremyb: I don't see a problem with using #wikimedia-office [20:27:18] * jeremyb will bbl [20:27:54] re: the calendar feed issue... [20:28:29] There is an IRC office hours public Gcal, which I am happy to add anyone too if they are going to add events [20:29:06] https://meta.wikimedia.org/wiki/IRC_office_hours also uses hcal microformats [20:55:08] <^demon> AaronSchulz: Where do wfDebug() calls end up on fluorine? [20:55:29] you mean wfDebugLog? [20:55:38] /a/mw-log [20:55:52] <^demon> Ah, whoops. [20:59:13] heh...whoops indeed :) [20:59:15] ^demon: lol, everytime I refresh https://upload.wikimedia.org/wikipedia/commons/7/76/Albert_Mcintire.gif the white part is different ;) [20:59:41] i think https://gerrit.wikimedia.org/r/#/c/29875/ causes fatals. can somebody look at my comment there? [21:03:58] MatmaRex_: I think aude is looking at that [21:04:19] AaronSchulz: can we get your help on a couple of things? [21:04:53] like that gerrit link? [21:05:00] we probably could use another pair of eyes on that, yup [21:05:34] also, when you're done with that, then probably the patch I made probably needs some cleanup [21:06:25] please check my patches :) [21:09:14] https://gerrit.wikimedia.org/r/#/c/29881/ <- this fixes the fatal [21:09:21] (or at least claims to) [21:10:03] * Damianz claims to be superman [21:10:08] it should but obviously please double check my changes :) [21:10:32] * aude can look at a proper fix for the content handler class tomorrow [21:10:41] this is a bandage fix [21:10:47] Aaron is looking now [21:10:47] * Damianz looks at a var reassignment to use the var once and goes CRAZY [21:10:49] ok :) [21:11:16] heh [21:11:30] Meanwhile, my computer locked again....grargh [21:20:14] error: PHP fatal error in /usr/local/apache/common-local/php-1.21wmf2/includes/content/ContentHandler.php line 69: [21:20:14] Argument 1 passed to ContentHandler::getContentText() must implement interface Content, string given, called in /usr/local/apache/common-local/php-1.21wmf2/includes/EditPage.php on line 1479 and defined [21:20:57] Auric: we know and in process of fixing it [21:21:11] apologies for that error :( [21:21:21] thanks! [21:21:30] can somebody look up and find a stacktrace for https://bugzilla.wikimedia.org/show_bug.cgi?id=41360 , reported earlier today? [21:21:42] 10 bucks says it's the LinkCache issue [21:21:55] senior developers are checking the fix for that and make sure everything is fixed [21:23:53] aude: at least you did something, unlike everybody else :/ [21:23:58] looks like it's merged now [21:24:13] is information about deployment being made public anywhere? [21:24:24] they don't happen straight from gerrit branches, are they? [21:24:38] http://wikitech.wikimedia.org/view/Server_admin_log [21:24:54] no, it's not automatic [21:24:54] ah, right. [21:27:50] Reedy: received the mail for the wikitech. account, thank you. [21:30:15] yeah, that doesn't really merge [21:30:31] a bunch of surrounding things work differently [21:32:46] Does anyone know where magicwords get cached? I keep getting an 'invalid magic word' after adding a new one.. [21:32:58] somebody needs to backport and deploy https://gerrit.wikimedia.org/r/#/c/29881/ . [21:34:13] gah, wrong branch [21:36:34] MatmaRex_: I believe AaronSchulz is working on that [21:37:04] edit to bart the general went through, edit to IP user talk page still stalled [21:38:17] aude: Fatal error: Call to a member function getTextForSummary() on a non-object at /usr/local/apache/common-local/php-1.21wmf2/includes/content/ContentHandler.php on line 859 [21:38:23] is there a bug # for that already? [21:39:01] what? [21:39:17] okay, i can confirm that edit conflicts no longer explode. thanks. [21:39:20] how did you get that one? [21:39:30] fatal.log [21:39:32] ok [21:39:44] i think no bug yet [21:39:44] went through [21:40:42] aude: so yeah, on delete somehow [21:40:57] huh [21:41:16] * aude looking [21:46:05] we're having trouble too deleting articles on fr.wp [21:49:28] fabriceferrer: don't know if it's related but looking [21:49:35] thanks [22:37:55] gn8 folks [23:00:04] AaronSchulz: i can't reproduce the fatal error for deletion nor fix right now [23:00:43] i can poke at the code more in the morning [23:00:48] ok