[00:29:34] gn8 folks [03:07:36] * Jasper_Deng pokes Krenair [04:11:19] Nemo_bis: I'd like to import http://en.wikipedia.org/wiki/Template:Shortcut to wikidata [04:11:36] but right now I can only import from Meta [04:34:13] TimStarling: for grabbing the content of an article from meta, I could either get it via HTTP, or use wfGetDB to get a cursor on meta's db. SpamBlacklist went with HTTP, but only because it was designed to be usable outside the WMF cluster, I'm guessing. [04:34:34] so I'm inclined to go with a DB read. Any reason why I shouldn't? [05:43:53] ori-l: I would use HTTP if I were you [05:44:10] why? [05:44:42] I don't think ExternalStore supports foreign databases, and it's not that simple to interpret the text table without it [05:46:38] TimStarling: loadFromConds( wfGetDB( DB_SLAVE, array(), 'metawiki' ), ... ) ? [05:46:53] i don't know what externalstore is [05:49:36] i'm skimming through ExternalStore.php now. not a part of the codebase i've had the occasion to look through before. [05:50:15] if you use HTTP, then you can take advantage of the revision cache [05:50:36] so it might be even faster on average than direct DB access [05:50:52] although I suppose you will have another cache in front, either way [05:51:09] well, using HTTP will make this easier to develop & test, certainly [05:51:23] i wasn't looking forward to setting up a metawiki db locally [05:57:03] TimStarling: SpamBlacklist resorts to some fancy acrobatics to dodge cache stampedes [05:57:52] "There are two keys, when the warning key expires, a random thread will refresh the real key. This reduces the chance of multiple requests under high traffic conditions." [05:59:13] it adds some complexity, but then again, if SpamBlacklist works well, i can just follow its example, i guess. [06:01:20] how many requests per second are you expecting? [06:05:16] a lot. it'll be listed as a dependency by a javascript module that is loaded on a lot of pages [06:05:56] enough to warrant carefulness handling cache expiry [06:06:00] maybe you should use an ArticleSave hook [06:06:09] yeah, that's a good idea [06:07:15] write the text to the database, in a special table, and to memcached [06:07:24] with an infinite memcached expiry [06:07:28] why write it to a special table? [06:07:46] then in the unlikely event that the key goes missing from memcached, you can just read it from a slave [06:08:21] why? because by "unlikely" I mean it will only happen once every few months [06:08:34] right, but still needs to be handled gracefully [06:08:37] and you probably don't want the feature to fail completely so regularly [06:14:32] ok, makes sense, thanks. [08:12:39] poor dbbot-wm [08:16:44] hi. Anybody awake? [08:17:26] just reporting a strange error message I just reveiced when trying to see a diff: [08:17:37] [6409cba8] 2012-11-12 08:15:53: Fatal exception of type MWException [08:17:51] on https://pt.wikipedia.org/w/index.php?title=Auguste_Rodin&curid=58688&action=history [08:20:45] history worked for me. logged in user [08:21:12] Teles: what diff? [08:22:39] not a diff; sorry. That link above to the page history [08:22:56] it is working now btw [11:44:33] andre__: https://bugzilla.wikimedia.org/show_bug.cgi?id=41938 parser test or unit test? [11:44:43] (I don't see how parser is related) [11:45:48] Nemo_bis: I assumed dropping unknown tags requires parsing tags, but please feel free to correct me in that report :) [12:06:02] hey all, anyone online that understand gerrit? [12:13:19] dan-nl: what's your problem? [12:20:04] paravoid: i'm trying to amend an initial commit, https://gerrit.wikimedia.org/r/#/c/31423/. i want to improve the change, but i'm not sure exactly how to checkout the change ... do i start with git review -d Id8603271 or git review -d I883af73e? [12:20:39] oh I don't know git review, sorry [12:20:46] generally, you can do git commit --amend to amend a commit [12:22:01] ja, tried that earlier and got a message with several outstanding commits asking me if that's really what i wanted to do, submit all of the outstanding commits, but i only want to submit the collection of them as a patch set to that initial commit ... [12:22:37] so i find it unclear if git review will submit them as a patch set or as individual commits ... [15:16:10] QUESTION: How can I track whether a change that got merged in gerrit has been deployed or when it is scheduled to be deployed? [15:16:54] ragesoss, basically https://www.mediawiki.org/wiki/MediaWiki_1.21/Roadmap [15:16:58] Well, no [15:17:23] Well, kind of? [15:17:30] sort of [15:17:38] If you click on the link next to Change-Id, you can find out (if for core) if it's been merged into a deployment branch [15:17:39] Agreed. ;) [15:17:52] if clicking ont the link gives you more than one entry, chances are, it has [15:17:57] And it'll tell you what branches [15:18:15] 99.9% of the time merges to deployment branches are pushed and deployed within a short period afterwards [15:19:20] Extensions are updated out of order too.. You'd have to checkout the branch, the submodules and then view the extension log [15:19:37] The roadmap can give you some idea - if it's merged to master (core or extension) before the monday deployment date (mid afternoon UTC), it should be then be set to be deployed over the next 2 weeks [15:20:56] I'm not exactly sure what I'm looking for, after clicking the change-ID link... [15:20:57] https://gerrit.wikimedia.org/r/#/c/28816/ [15:21:27] The same thing [15:21:31] it's in master, but I don't see any deployment branches that I recognize. [15:21:35] Indeed [15:21:41] Most extensions don't have deployment branches [15:22:07] They're taken from the trunk version at the time of branching, unless specifically up date out of band [15:22:12] so that means, it'll get deployed within roughly two weeks, at the next wmf# rollout? [15:22:21] Unless you poke someone to deal with it [15:22:38] That'll be in 1.21wmf4 which I'm branching now.. So on enwiki (at the current rate) next monday [15:22:51] I can however just update the deployed version of the extension now-ish for you [15:23:07] please do. :) [15:23:20] and thanks much for the explanation! [15:24:23] grrr, gerrit die [15:24:24] d [15:24:41] gerrit died, or you're telling it to do so? [15:24:52] died. [15:24:53] I see. [15:24:58] :( [15:28:55] So in general, the date of the Phase 1 deployment of a wmf# branch is also the time that it branches? [15:29:32] or rather, you branch it just before and then start deploying it to the Phase 1 set? [15:29:59] yeah [15:30:06] usually done a couple of hours before [16:40:31] hey akoopal [17:09:30] ^demon: akoopal: any hints about recent deploys that may have broken something related to search? it looks like (from reading these reports) not a server side error but rather some issue with the form never submitting at all. (maybe JS?) most mention trying to click the magnifying glass [17:09:40] here they are: 2012110910007595 2012111010007351 2012111110002273 2012111210003378 2012111210002851 [17:11:24] <^demon> jeremyb: No clue. I know we've been doing some minor tweaks to support search in Wikidata, but that's about it. [17:11:35] hrmmm [17:12:03] so, now what? i was thinking about poking robla but i guess it's too early there [17:13:19] first report was 2012-11-09 15:48:31 UTC [17:15:07] oh, huh, that's who that is... [17:15:15] but he's not here. (ironholds) [17:15:28] he's in other channels [17:17:19] Reedy: i know, i did once get him to join here for a short bit but he doesn't stick around [17:17:44] fuck yeah, outreach [17:18:45] wondering who to poke to get access to OTRS [17:19:19] An OTRS admin? [17:19:39] I think Philippe is probably the best bet [17:19:42] Would email him though [17:21:11] andre__: oh, whoops, i confused you with akoopal [17:21:22] which is bad since i've even met him once [17:21:27] :D [17:21:44] andre__: why the __ ? [17:22:01] andre__: anyway, first step is /j #wikimedia-otrs ;-P [17:22:09] jeremyb, because my name is obviously too popular and I've been too lazy to change to something like aklapper. maybe I should :) [17:23:43] https://meta.wikimedia.org/wiki/OTRS#OTRS_administrators [17:24:04] ah [17:24:05] thanks [17:24:39] probably not a complete list [17:24:56] let's see if it's complete... [17:26:33] analytics1001 - analytics1010 192 GB RAM O_o [17:27:29] Nemo_bis: there's no philippe or maggie on that list [17:28:07] basile is missing and RD is on the list. [17:28:14] * jeremyb tries to remember the other recent changes [17:29:07] it was supposedly updated in Sep [17:29:21] I don't remember if admins have the list of admins :) [17:34:46] hrmmm, quick poll: what's worse between a) having 10 dbbots with varying numbers of _ or b) having only one variant but frequent join+quit ? ;-P [17:35:28] @replag all [17:35:29] jeremyb: [s1] db63: 0s, db38: 0s, db32: 0s, db59: 0s, db60: 0s, db36: 0s; [s2] db54: 0s, db52: 0s, db53: 0s, db57: 0s; [s3] db34: 0s, db39: 0s, db64: 0s, db11: 0s [17:35:30] jeremyb: [s4] db31: 0s, db33: 0s, db51: 0s, db65: 0s; [s5] db45: 0s, db35: 0s, db44: 0s, db55: 0s; [s6] db47: 0s, db43: 0s, db46: 0s, db50: 0s; [s7] db37: 0s, db56: 0s, db58: 0s, db26: 5s [17:35:39] well at least it's functioning [17:41:50] * apergos votes for one of the other choices [17:42:05] you want one bot [17:42:12] it shouldn't need to join/quit multiple times [17:42:26] I'm guessing it keeps dying and someone on TS keeps re-starting it [17:42:52] i e-mailed krinkle about it, apparently they're aware [17:42:58] oh. this is the 'toolserer is having problems with some nfs server which is impacting the bot" problem (or so I heard iit from hydriz ealier [17:42:59] earlier [17:43:18] *toolserver [17:43:35] hmm /me is not in that channel, woops [17:44:20] status dashboard shows all happy, meh [17:44:57] Reedy: i was assuming it was SGE? [17:45:05] * Reedy shrugs [17:45:13] don't know, don't really care ;) [17:45:25] well the point is it's not human [17:45:41] I meant something [17:45:43] not someone [17:45:49] not that it really makes much of a difference [17:48:17] what sort of strategies do we have in place for preventing cache stampedes when memcached gets restarted? [17:48:43] err, huh? [17:48:49] memcache doesn't get restarted! [17:48:54] ori-l: seen poolcounter? [17:48:57] well, in the event that it does [17:49:00] nope [17:49:03] what is it? [17:49:19] prevents all the apaches trying to parse/render the same page at the same time [17:49:22] i don't remember exactly [17:49:28] Also known as the "Michael Jackson effect" [17:49:39] MJ died. Wikipedia soon followed [17:49:41] also, some is now in redis and some (parser cache) is in mysql [17:49:52] Reedy: right, i'll go look at that [17:49:53] only session cache is in redis [17:50:00] http://www.mediawiki.org/wiki/Extension:PoolCounter [17:51:19] i'm implementing an RL module that is going to fetch an article from an metawiki via http and stuff it in memcached. just thinking about how to handle expiration gracefully. [17:51:31] tim suggested writing to some local table [17:52:07] why is it doing that? [17:52:19] which part? [17:52:28] fetching from meta [17:52:47] long story [17:53:36] you've seen extract2.php ? i guess maybe not actually relevant but I was thinking about it [17:57:09] reedy, I think you need to roll back that education program deployment. [17:57:18] Why? [17:57:28] it looks like it overwrote the default admin usergroup management. [17:57:38] now the ONLY thing admins can change are the EP permissions. [17:57:59] not file mover or reviewer, etc. [17:58:39] lol [17:58:59] :) [17:59:09] 353 +$wgAddGroups['sysop'] = array( 'eponline', 'epcampus', 'epinstructor', 'epcoordinator' ); [17:59:09] 354 +$wgRemoveGroups['sysop'] = array( 'eponline', 'epcampus', 'epinstructor', 'epcoordinator' ); [17:59:09] 352 355 [17:59:12] Very likely [17:59:14] * Reedy facepalms [18:00:32] interesting approach [18:04:12] ragesoss: it'll be fixed on the site in the next minute or so [18:06:18] cool. now I'll have a scarlet letter for the rest of my days, as my first +1 on gerrit was terrible. :) [18:06:54] alright, looks good! [18:53:17] hrmmm, is a WMF holiday? [18:54:04] jeremyb: still looking for robla? [18:54:25] yes wmf has a holliday today [18:54:34] "veterans day" I think [18:58:24] yes [18:58:42] apergos: Veterans' Day was yesterday, but we celebrate weekend-holidays by rounding [18:59:21] right, nearest monday [18:59:36] anyone have a sec to +2 i18n change? https://gerrit.wikimedia.org/r/#/c/33105/ [19:00:38] apergos: Actually, apparently Saturdays round down to Friday instead of up to Monday. [19:01:02] ori-l: That seems like something that will need design review [19:01:30] marktraceur: the change was already reviewed / merged / deployed for personal tools in general [19:01:37] there are some extensions that need to be brought to the fold [19:01:45] see mediawiki.org now for example [19:02:21] ori-l: Right, sorry, I went back to see the wikitech thread now, I had forgotten that there was consensus. [19:03:07] Nemo_bis: i just didn't know if WMF was observing [19:03:17] ok [19:04:30] Reedy: Forgot this earlier, but I need https://gerrit.wikimedia.org/r/#/c/31949/ to be deployed for VE on MW.org to work again [19:07:41] marktraceur: we usually don't round down for holidays. veterans day seems to be an exception. [[Uniform Monday Holiday Act]] and the end of [[Veterans Day]] [19:15:29] * aude don't take holidays :) [19:16:13] it'd be nice to take weekends ;) [19:16:41] Reedy, i was going to say, you must be shooting for some sort of commit record [19:17:11] all of the commits! [21:33:05] pagecounts-20121112-210000.gz, size 161M (http://dumps.wikimedia.org/other/pagecounts-raw/2012/2012-11/) [21:33:11] why so bigger than usual? [21:53:14] jeremyb: Philippe / Maggie are not OTRS admins, so that's why they're not on the list ;-) Also, guillom resigned a couple months ago [21:53:22] So yes, the list is up-to-date and current [21:53:26] Nemo_bis: ^ [21:57:31] thanks RD [21:57:38] can you add andre__ then? ;) [21:57:46] unless someone else did so in the meanwhile [21:59:56] darrn, I sent a message to Maggie already :) [22:00:23] RD: errrr, yes, basile was then a valid benchmark for whether it's up to date [22:01:20] i just can't remember how close steward resignation was to OTRS admin resignation [22:06:31] Add andre why? [22:07:04] * RD just stares; Philippe/Maggie don't have all the rights either....I'm an OTRS admin if you have questions I might be able to answer [22:07:25] RD: he's the WMF bugmeister. bugs sometimes get reported to OTRS. [22:07:31] ah [22:07:31] Hi RD. I'm after sometimes asking followup questions for important issues in order to narrow them down - same as in the forums. [22:07:36] 12 17:09:40 < jeremyb> here they are: 2012110910007595 2012111010007351 2012111110002273 2012111210003378 2012111210002851 [22:07:52] RD: I don't plan to be very active as bugzilla.wikimedia.org is my natural habitat, but OTRS would be another interesting source of feedback, apart from Bugzilla itself and all the Village Pump etc. forums that I try to look at [22:07:57] I imagine Maggie [22:08:04] *Maggie'll forward the note [22:44:38] andre__: what would be the best bugzilla query to list all the bugs "touched" by a user? [22:44:51] (with a comment or change to any field) [22:46:19] !log deleted refreshLinks jobs corresponding to my recent edit to enwiki's Template:Citation/core [22:46:26] Logged the message, Master [22:48:53] Nemo_bis, could you elaborate? [22:52:01] I love «Nobody ever held a straw poll asking the community "can we please make article parsing 27% slower in order to support a rarely-used metadata feature?"» [22:53:02] andre__: I'd like to list e.g. all the bugs I did something to [22:53:23] which I'm not necessarily a reporter, cc'ed, voter or commenter of [22:54:27] a clearer use case perhaps: some ad-hoc accounts run big mass-changes, typically not adding themselves to cc; how to list them all? [22:54:39] Nemo_bis, "Custom Search" on query.cgi offers "foo | changed by | username", however you cannot set "foo" to something generic [22:54:55] so I think what you want is not possible, no [22:56:00] ..because not all fields are available there, right? [23:03:23] are query groups declared somewhere? or can i simply decide on one (the name of the extension, in this case) and pass it to wfGetDB? [23:03:54] (are there conventions regarding their use? name of the extension responsible for the query seems sensible..) [23:06:33] why do you want to use a query group? [23:07:44] to make it traceable to my code if it ends up being responsible for mischief and mayham [23:08:00] that's what the function comment is for [23:08:33] brb, baby [23:08:36] you know how most Database methods have a $fname parameter? [23:09:38] no [23:09:39] * ori-l looks [23:14:00] Revision::loadFromTitle doesn't, and that's what i'm using. I suppose it's common enough that it doesn't merit having every single call uniquely identified. I was going to use it here because it's pulling a revision from across databases. (I ended up deciding to go that route rather than grabbing via HTTP because grabbing via HTTP would need to be backed by a DB copy anyway.) [23:16:06] you can use Revision::newFromRow() [23:16:28] or "new Revision($row)", that appears to be a synonym [23:17:11] but if it's a query with a title in it, your extension will be traceable from that [23:17:32] Nemo_bis, most fields are available, but you'd have to "call" each one separately. Creating that query will be cumbersome and lots of clicking. [23:17:54] heh, that's a good point [23:40:00] http://dpaste.de/p3AC2/ [23:40:11] TimStarling: if you have a minute, can you glance at this? ^^^ i'm not done with it, but want to make sure i'm heading in a sane direction. [23:40:38] ^demon: andre__: add 2012111210000648 to the list of tickets i mentioned earlier [23:40:51] configuration should be in a global variable, not a constant [23:41:09] right, i'll fix that [23:42:59] hmm, there are some other bits of code using 2592000 as expiry. i'll go for 29 days or something. [23:45:52] creating Revision objects for content in foreign databases is not what the $db parameter was intended for [23:46:41] Revision::loadText() obviously won't work [23:47:02] it gets the text from wfGetDB( DB_SLAVE ) [23:48:07] looking at ExternalStore now, it seems it does actually have facilities for accessing foreign databases [23:48:57] but Revision doesn't use that facility [23:49:23] so in terms of the general direction, no it is broken for all the reasons that made me recommend the use of HTTP yesterday [23:50:00] well, not _all_ the reasons. you were worried about caching too. [23:50:20] caching is broken [23:50:40] $textId = $this->getTextId(); [23:50:41] $key = wfMemcKey( 'revisiontext', 'textid', $textId ); [23:50:59] see, wfMemcKey(), not wfForeignMemcKey() [23:51:23] calling Revision in this way will pollute the revision cache of the local wiki of the caller [23:51:45] aw, fuck. didn't think about that at all. [23:52:04] well, no obvious spelling mistakes, maybe i get half a point for that [23:55:42] I'm sure you can find a way which is both extremely simple (<50 lines) and works at 1000 req/s [23:56:16] ok not really [23:56:29] I think you should just use one of the methods I said before [23:56:35] yeah, i think so too [23:56:51] ExternalStore would work, but i'd have to write off the facilities provided by Revision, so no point [23:57:14] so i think 1) look up key in memcached [23:57:25] a PageContentSaveComplete hook is pretty easy [23:57:39] (that is what ArticleSaveComplete is called now) [23:57:41] 2) if key is not there, try to get a lock for re-grabbing via http [23:58:00] and works at an arbitrarily large scale [23:58:07] 3) if unable to get lock, return empty response and just deal with it, event-logging is non-essential [23:58:28] TimStarling: but you said once in a blue moon the key will be missing from memcached for whatever eason [23:59:26] yeah, the solution for that is pretty easy though