[00:00:58] !log preilly synchronized php-1.20wmf3/extensions/MobileFrontend/ 'try again' [00:01:02] Logged the message, Master [00:04:50] "Due to high database server lag, changes newer than 64 seconds may not appear in this list. " - ENWP looking at my contribs. Time keeps going up every time I go back to it [00:05:05] Not sure if it's an issue, but figured I'd give a heads up [00:06:22] !log preilly synchronized php-1.20wmf2/extensions/MobileFrontend/ 'try again' [00:06:26] Logged the message, Master [00:11:36] !log preilly synchronized php-1.20wmf3/extensions/MobileFrontend/ 'try again' [00:11:39] Logged the message, Master [00:11:42] just db12 [00:13:19] !log preilly synchronized php-1.20wmf3/extensions/MobileFrontend/ 'try again' [00:13:22] Logged the message, Master [00:14:05] preilly: what did you change? [00:18:32] AndrewN: how is it now? [00:18:50] * AndrewN looks [00:19:08] Box is gone. [00:19:27] looks fixed [01:10:08] !log synchronized payments cluster to DonationInterface 4be175e43f [01:11:27] ...hm. No log? [01:13:01] Logged the message, Master [02:23:33] !log LocalisationUpdate completed (1.20wmf3) at Wed May 23 02:23:33 UTC 2012 [02:23:37] Logged the message, Master [02:45:44] !log LocalisationUpdate completed (1.20wmf2) at Wed May 23 02:45:44 UTC 2012 [02:45:47] Logged the message, Master [06:35:17] is there any special link for diffs like there is for Special:PermanentLink? [06:35:51] would make it easier to add diff information to edit summaries and log summaries (e.g., blocking, unblocking) [06:36:41] obnoxiously, permanentlink doesn't appear to be documented anywhere, even in Special:SpecialPages [06:44:24] I don't think there is [07:45:43] hi [16:10:08] is bits ok? i'm not getting .css stuff [16:11:02] well, seems to be only on en.wp [16:14:55] nop, now i'm having problems on br.wikimedia.org [16:24:31] Hi everyone. Wikimedia Localisation team office hours in #wikimedia-office in 6 minutes. [16:51:11] robla: I'll be about for the todays deploy [16:51:16] Not sure if I said that before or not [16:51:25] cool [16:53:35] robla: I'll help check pages when that happens [17:00:05] !log reedy synchronized wmf-config/ 'Tidying unpushed changes' [17:00:08] Logged the message, Master [17:12:17] !log preilly synchronized php-1.20wmf2/extensions/MobileFrontend/ 'RL hack' [17:12:21] Logged the message, Master [17:13:24] !log ben synchronized wmf-config/CommonSettings.php 'changing the URL for the mobile feedback page to the Project namespace' [17:13:27] Logged the message, Master [17:24:09] Reedy: https://gerrit.wikimedia.org/r/#/c/8622/ [18:05:58] Reedy: Aaron|home: ready to deploy? [18:06:43] yes [18:07:04] Sure [18:07:07] How do we want to do this? [18:07:35] Do the big ones first? [18:07:37] Just don't break it. :P [18:07:54] * MatmaRex_|away awaits 1.20wmf3 in pl.wiki [18:08:06] Reedy: let's just do the whole batch at once [18:08:15] * MatmaRex_|away needs to do some expansion depth debugging. :) [18:08:26] we can always revert if things get crazy [18:08:56] since we have so many wikis on 1.20wmf3 already, this seems pretty low risk (famous last words) [18:09:47] hokay [18:10:02] Also, yay for wikiversions.dat being in version control ;) [18:10:11] srsly [18:10:40] 286 wikis [18:11:31] !log reedy rebuilt wikiversions.cdb and synchronized wikiversions files: Last 286 wikis over to 1.20wmf3 [18:11:34] Logged the message, Master [18:12:05] Reedy: what ever happened with wikiboos? [18:12:07] *wikibooks? [18:12:13] What about them? [18:12:14] were they put back to wmf3? [18:12:18] yeah [18:12:22] I only did it temporarily [18:13:03] big wikis look fine [18:13:08] Nothing new/unexpected in the error logs [18:13:17] * Reedy waits for php-1.20wmf2 OOMS to go away [18:13:20] * robla starts on the 100,000+ list here: http://www.wikipedia.org/ [18:15:06] now for the 10k wikis [18:15:11] Only wmf3 errors now showing [18:15:16] * AndrewN waits for an influx of people from the updated wikis to come in and scream OMGWTFUBROKEIT!!! [18:15:21] I should hope :) [18:17:06] 10k wikis look fine. 1k wikis now [18:17:09] Call to a member function getRawText() on a non-object in /usr/local/apache/common-local/php-1.20wmf3/extensions/AbuseFilter/AbuseFilter.hooks.php on line 33 [18:17:14] * Aaron|home wonders what's with that [18:17:22] I've got a fix in gerrit for that [18:17:51] werdna seemed to think it was somewhat simplistic, because "that shouldn't happen" (IIRC), but it obviously does ;) [18:18:16] Fatal error: Call to a member function getDBkey() on a non-object in /usr/local/apache/common-local/php-1.20wmf3/extensions/Collection/Collection.body.php on l [18:18:16] ine 678 [18:18:30] * Aaron|home is of the opinion that fatals should never happen [18:18:42] indeed [18:18:50] * Aaron|home has high standards [18:18:54] well, OOM fatals are a bit of an exception [18:19:09] Asher has said we can put the memory limit up when memcached is on it's own cluster [18:19:20] eta one that? [18:19:39] binasher: ^ [18:19:48] I think the machines have been ordered [18:20:21] maybe a month from now [18:20:30] cool [18:20:58] i'm not sure if the order has actually gone out, the original quote from dell had the wrong type of nics (we're putting 10GB nics on these) [18:21:05] 1k wiki home pages all look fine [18:21:07] but if it isn't, it's supposed to by friday [18:22:30] robla: nothing worth worrying about in the error logs [18:26:53] Reedy: xlnt....looks like all of the wikis' homepages load (even the locked ones) [18:27:19] Nice [18:30:15] Reedy: do we have a fix in master for https://bugzilla.wikimedia.org/show_bug.cgi?id=36950 ? [18:31:09] Yeah [18:31:13] Reverted the revision out [18:31:17] Aaron approved it [18:31:20] ah, ok [19:16:44] I would like Ops help with a Gerrit issue. I mistakenly removed myself from the list of members of a Gerrit group before I made it world-visible. Can one of you add me back? The group is "extension-Translate" [19:17:21] I am not sure whether it was created at all, except that I get an internal server error if I try to create a new Gerrit group with the same name. [19:18:29] Ryan_Lane: maybe you can help? I would ask Chad but he's not around, and Antoine is busy with a deployment and Sam is in transit [19:19:17] done [19:19:45] Thanks [19:19:56] yw [19:31:24] mutante: ssmollett: maybe you can help me with this Labs question - I am running add-labs-user (see process # 11638 on formey) and it's hung. Should I just control-C and try again, or should I wait it out....? Don't want to cause weird inconsistent issues in LDAP [19:31:44] you won't [19:31:49] it's a single LDAP call [19:32:09] you can run it and kill it as many times as you want [19:32:29] ok, thanks Ryan_Lane - http://p.defau.lt/?Y58eERmcMnhosdx2d2LG1g in case you want to see the traceback that resulted [19:32:52] it was timing out on mediawiki [19:33:28] * sumanah runs it again, kills it again, gets another timeout at the same place [19:33:58] again, same issue. hm, will wait 5 min and try again [19:34:09] you're running this from formey? [19:34:43] Yes. [19:35:58] hm [19:38:47] now *this* is a bug in OATHAuth :) [19:40:01] sumanah: ok. now run it [19:40:32] Ha! [19:40:42] It works! Thanks Ryan_Lane [19:40:54] * sumanah accidentally had her tester hat on? [19:41:44] not sure how this didn't happen every time [21:31:11] Having some trouble with de12 it appears. (Reported by Earwig) http://en.wikipedia.org/w/api.php?action=query&meta=siteinfo&siprop=dbrepllag&sishowalldb= [21:31:16] *db12 [21:34:53] "Due to high database server lag, changes newer than 317 seconds may not appear in this list. " - My watchlist on ENWP [21:35:03] 317 second lag on en.wp watchlist [21:35:59] It's db12 again >.> [21:38:17] * AndrewN looks to see if Ryan_Lane or Reedy are around. [21:39:16] binasher is probably the gent your after [21:39:40] I just know those 2 were active in here earlier. Haven't seen binasher around. [21:45:25] !log asher synchronized wmf-config/db.php 'pulling db12 from enwiki, moving watchlist / special queries to db60' [21:45:28] Logged the message, Master [21:45:55] thank you [21:46:01] Good :) That's the third time that db12 has done this. [21:46:06] Thanks :) [21:49:09] db12 is going to be replaced in about a month [21:49:21] binasher: Bad hardware? [21:49:54] just old and underpowered for enwiki these days [21:50:29] Makes sense [21:50:54] Yeah, it does. [21:58:34] poor box [21:59:36] I'm wondering if doing API queries with a maxlag of 5 is too careful - perhaps 10 or 15 would be better? [22:41:50] Hey all. At what point (in terms of number of edits) does a change of username start to become a pain in the ****? Specifically, how much difficulty is involved in renaming an en.wp user with 29,146 edits? [22:45:43] Jarry1250: It's not a problem to rename you [22:45:52] Jarry1250: it is possible to rename, there's no limit of usernames now [22:45:56] Just request it in the proper location, and if you need help, head to #wikipedia-en-help [22:46:00] no max edit number* [22:46:14] No [22:46:31] wait, yes, shit. [22:46:43] Sure, thanks. It's not me (I'm on a mere 20,911) but rather my bot. [22:46:48] I'll go request it anyway. [22:46:49] AndrewN: tim changed it recently, apparently it won't break the servers [22:46:52] AndrewN: he's a well established member of the community and Wikipedia admin, I'm sure he can find a crat if he needs one ;) [22:47:05] but not in #wikipedia-en-help :P [22:47:11] Evening guys - just checking, are you aware of the massive lag on DB1003? Replication delay of about 2.5million seconds [22:47:11] * AndrewN didn't know [22:47:17] Jarry1250: out of curiosity, when you do API queries, what do you use as your maxlag? [22:47:17] AndrewN: no problem :) [22:47:34] Thehelpfulone... always so helpful. :P [22:47:47] * Thehelpfulone bows [22:47:48] Earwig: Good question. I use the Peachy default, one sec, I'll check. [22:48:58] it is dropping, it's gone down by about 40,000 seconds in the last 25 minutes or so, but still wise to let you know about it, even if you're already aware :) [22:49:15] Earwig: 10, apparently. [22:49:33] thanks - maybe I should be using that myself instead of 5 [22:50:18] Earwig: Oh, actually I use 0 if I use my normal (non-bot) credentials [22:50:47] is zero... not following maxlag at all, or ultra-restrictive? [22:57:12] Jarry1250: additionally, any knowledge of how peachy throttles queries? [22:58:38] Earwig: I forget, I assume it just fires $maxlag at the server and continues to do and then retries every now and again thereafter. [22:59:15] Jarry1250: thanks, but I was thinking more of... er... barring any lag issues, will it do another query right after the last one with no wait time? [22:59:31] Oh. sorry, my bad. [23:00:42] I think it just continues to fire queries. I believe that's not too bad for the server (parallel threads, I'm told, is what really starts to drain resources) [23:01:44] Jarry1250: thanks again [23:02:27] Earwig: No worries, it's been ages since I looked over its code (and indeed contributed some) so I could be wrong still. [23:03:46] Jarry1250: I'm asking because I have a few of things in my bot designed to be nice to the servers (low maxlag value, not doing multiple API queries within three seconds of each other, etc), and I'm wondering if it's a bit unnecessary (as things would go faster if I toned them down a bit) [23:05:06] Earwig: As I say, AFAIK (server config isn't m speciality, maybe someone else could chip in?), a good low maxlag and avoiding multiple simultaneous connections is about right. [23:05:43] been trying to get someone else to talk, but no one seems to be alive :/ [23:10:48] It might help, it might not, Earwig - but usually when I set a value of 0 to something, it either disables it or sets an absolute minimum to whatever I'm running - i'd assume a maxlag of zero may very well make it ignore maxlag and carry on running regardless [23:11:05] that's what I thought, yeah [23:11:29] which makes sense considering it's when he's using normal non-bot credentials [23:11:50] by giving it a !0 value, I'd assume it follows the setting, a zero value would probably disable it [23:41:07] gn8 folks