[01:51:07] <blur>	 Hi all, I was looking to port over some pages from Meta-Wiki to nyc.wikimedia.org and received the response "direct history uploads are disabled." Another friend has told me that from his recollection is it's disabled for "the whole cluster".
[01:51:27] <blur>	 if anyone knows what to do or is willing to help, it'd be much appreciated
[01:53:40] <jeremyb>	 oh, meta. you should have said that ;)
[01:54:36] <blur>	 the other friend lol
[01:54:37] <jeremyb>	 blur: and enwiki too i assume?
[01:55:21] <^demon>	 File a bug in BZ for metawiki to be added to wikimedianyc's import sources.
[01:56:01] <jeremyb>	 ^demon: i was going to just do a gerrit
[01:56:09] <^demon>	 Or that too.
[01:56:15] <^demon>	 Keep forgetting that's all gerritized.
[01:56:17] <blur>	 yes, enwiki too eventually perhaps
[01:56:27] <jeremyb>	 enwiki now ;)
[01:56:37] <blur>	 sure, why not? lol
[01:56:47] <blur>	 i defer to both your experience
[01:56:48] <jeremyb>	 ^demon: hard for me to forget!
[01:58:01] <jeremyb>	 ^demon: is there supposed to be any consistency in import sources?
[01:58:12] <^demon>	 Not that I know of.
[01:58:17] <jeremyb>	 some are 'meta', some are 'm'
[01:58:17] <^demon>	 Ask Reedy I guess.
[01:58:34] <^demon>	 Either probably works, but meh.
[01:58:40] <^demon>	 It's late. Don't care now. Goodnight :)
[01:58:46] <jeremyb>	 nacht ;)
[02:01:29] <blur>	 so jeremyb is this a one time thing with gerrit or would it be forward-thinking to open the BZ ticket as well?
[02:01:56] <jeremyb>	 i think BZ is unnecessary
[02:02:24] <jeremyb>	 !g I39d1ca7cccb6 | blur
[02:02:25] <wm-bot>	 blur: https://gerrit.wikimedia.org/r/#q,I39d1ca7cccb6,n,z
[02:03:34] <blur>	 im supposing thats a good thing :)
[02:03:47] <jeremyb>	 if you like
[02:03:49] <jeremyb>	 ;)
[02:05:56] <blur>	 if it gets us any closer to importing some page histories, then yes. i very much appreciate it :)
[02:07:46] <jeremyb>	 blur: you can feel free to babysit it to get it deployed ;)
[02:08:32] <blur>	 awesome!
[02:31:05] <logmsgbot>	 !log LocalisationUpdate completed (1.20wmf8) at Fri Jul 27 02:31:05 UTC 2012
[02:31:16] <morebots>	 Logged the message, Master
[02:53:30] <logmsgbot>	 !log LocalisationUpdate completed (1.20wmf7) at Fri Jul 27 02:53:30 UTC 2012
[02:53:39] <morebots>	 Logged the message, Master
[12:51:22] <Reedy>	 !log Created tgs_lang indexes on wikis that have translate installed
[12:51:31] <morebots>	 Logged the message, Master
[13:31:56] * jeremyb  pokes reedy

[13:32:17] <jeremyb>	 2 questions:
[13:32:46] <jeremyb>	 1) do we attempt any kind of consistency in import sources conf? e.g. some are 'meta' and some are 'm'
[13:32:54] <jeremyb>	 2) can you deploy for me? ;)
[13:32:57] <jeremyb>	 !g I39d1ca7cccb6
[13:32:57] <wm-bot>	 https://gerrit.wikimedia.org/r/#q,I39d1ca7cccb6,n,z
[13:33:15] <jeremyb>	 (^demon said you're the guy to ask about consistency)
[13:33:39] <^demon>	 Only cuz he does most shell requests these days :)
[13:34:11] <Krenair>	 Consistency in mediawiki-config?
[13:34:42] <jeremyb>	 Krenair: yes!
[13:34:49] <Krenair>	 Does not compute
[13:35:00] <jeremyb>	 what architecture?
[13:40:23] <logmsgbot>	 !log demon synchronized wmf-config/InitialiseSettings.php  'Deploying I39d1ca7c: import sources for nycwikimedia'
[13:40:31] <morebots>	 Logged the message, Master
[13:42:02] <jeremyb>	 ^demon: oooh, i think the string i put in the array goes straight into the drop down box. so if it's 'meta' or 'm' it's the same in the dropdown. not really tested though
[13:42:15] * jeremyb  had assumed there would be some sort of normalization

[13:42:23] <^demon>	 Na, it just lists the interwiki id.
[13:42:29] <^demon>	 so "meta" is probably better.
[13:43:06] <jeremyb>	 oh, damnit
[13:44:00] <jeremyb>	 ^demon: i think I'm hitting https://bugzilla.wikimedia.org/31042 ; followup on the way
[13:44:03] <jeremyb>	 ;-(
[13:45:31] <jeremyb>	 yeah, works for meta
[13:45:32] <jeremyb>	 https://nyc.wikimedia.org/wiki/Special:Log/import
[13:47:35] <hashar>	 ahhh 
[13:47:40] <hashar>	 people are writing tests!
[13:47:43] <hashar>	 !g 15690
[13:47:43] <wm-bot>	 https://gerrit.wikimedia.org/r/#q,15690,n,z
[13:47:50] <hashar>	 lovely SQPRobin
[13:47:58] <jeremyb>	 say it in dutch
[13:48:03] <jeremyb>	 err, flemish. sorry
[13:54:40] <jeremyb>	 hashar: or ^demon? want to take the followup?
[13:55:03] <hashar>	 jeremyb: let me review some Uzbek test suite first :-D
[13:55:15] <jeremyb>	 heh
[13:55:17] <jeremyb>	 !g Ibd6f1895e68c4
[13:55:17] <wm-bot>	 https://gerrit.wikimedia.org/r/#q,Ibd6f1895e68c4,n,z
[14:01:15] <hashar>	 so we now have a Uzbek latin-cyrillic converter thanks to Robin  yeah!!!
[14:01:20] <hashar>	 jeremyb: looking
[14:02:18] <hashar>	 I hate our InitialiseSettings.php wayyyy tooooo loooonnnng
[14:02:18] * jeremyb  is strictly latin (and a small bit of hebrew). never cyrillic

[14:02:28] <jeremyb>	 hashar: i made it longer recently!
[14:02:39] <hashar>	 I think cyrillic is most a one to one relationship to latin
[14:03:06] <jeremyb>	 well it's greek to me!
[14:03:10] <jeremyb>	 !g I442bd7d6b5b | hashar
[14:03:10] <wm-bot>	 hashar: https://gerrit.wikimedia.org/r/#q,I442bd7d6b5b,n,z
[14:03:27] <jeremyb>	 +110, -46
[14:03:49] <hashar>	 so the nycwikimedia addition
[14:05:27] <hashar>	 jeremyb: merged it
[14:05:37] * jeremyb  watches for the !log

[14:05:38] <hashar>	 Reedy: is it safe to deploy some init setting on cluster or are you working on something ? :-)
[14:07:06] <Reedy>	 I'm not touching anything
[14:07:21] <hashar>	 Reedy: thanks!
[14:07:28] <logmsgbot>	 !log hashar synchronized wmf-config/InitialiseSettings.php  '{{gerrit|16817}} - fix nycwikimedia wgImportSources'
[14:07:28] <hashar>	 grmblblblbl srv281: rsync: write failed on "/apache/common-local/wmf-config/InitialiseSettings.php": No space left on device (28)
[14:07:31] <hashar>	 !log srv281: rsync: write failed on "/apache/common-local/wmf-config/InitialiseSettings.php": No space left on device (28)
[14:07:37] <morebots>	 Logged the message, Master
[14:07:45] <morebots>	 Logged the message, Master
[14:07:56] <Reedy>	 Ignore it
[14:07:57] <hashar>	 jeremyb: the nycwikimedia import source should be live
[14:07:59] <Reedy>	 It's not in production
[14:08:02] <hashar>	 jeremyb: no idea how to verify it
[14:08:06] <jeremyb>	 hashar: i will
[14:08:12] <hashar>	 Reedy: can't we get it out of the dsh group?
[14:08:20] <hashar>	 also got srv206
[14:09:28] <jeremyb>	 hrmmmm
[14:10:06] <jeremyb>	 different error this time:
[14:10:10] <jeremyb>	 > Import failed: No pages to import. 
[14:10:36] <jeremyb>	 trying to import [[Wikipedia:Sandbox]]
[14:10:49] <jeremyb>	 >     Page "wikipedia:Sandbox" is not imported because its name is reserved for external linking (interwiki).
[14:12:08] <jeremyb>	 hashar: https://nyc.wikimedia.org/wiki/Special:Log/import
[14:12:19] <jeremyb>	 works for some source namespaces (mainspace) but not others
[14:12:32] <jeremyb>	 i assume this needs a filing
[14:12:33] <hashar>	 ohhh
[14:12:34] <Danny_B|backup>	 try to import to different namespace
[14:12:57] <hashar>	 I am not familiar at all with the import system
[14:13:03] <hashar>	 I just know how to export Barrack Obama 
[14:13:11] <hashar>	 and how to apply the $wgImport setting :(
[14:13:31] <jeremyb>	 Danny_B|backup: huh?
[14:13:38] <jeremyb>	 Danny_B|backup: the problem is source NS not target
[14:13:45] <jeremyb>	 Danny_B|backup: and did you see the log?
[14:15:42] <Danny_B|backup>	 in import you can set different target namespace so far i remember
[14:15:49] <jeremyb>	 sure. but why?
[14:16:01] <jeremyb>	 ohhhh
[14:16:12] * jeremyb  thinks maybe he understands now...

[14:19:04] <jeremyb>	 Danny_B|backup: check the log ;)
[14:19:31] <jeremyb>	 thanks hashar ^demon Danny_B|backup ;)
[14:19:39] <jeremyb>	 blur: give it a whirl
[14:19:53] <Danny_B|backup>	 see? ;-)
[14:20:35] <jeremyb>	 i suppose
[14:54:59] <MikeOscar>	 Please msg me if you're familiar with Google Apps - I need help!
[15:24:26] <jeremyb>	 MikeOscar: GAFYD (google apps for your domain)? or which kind of google apps?
[17:11:51] <simonshek>	 hello, i have a problem with wm2013 wiki
[17:12:54] <simonshek>	 about the mobile view, http://wikimania2013.m.wikimedia.org has nothing… how to set-up mobile view?
[18:44:40] <Frakir>	 Some connections issues :(
[18:44:57] <jeremyb>	 Frakir: could you be more vague?
[19:07:18] <jeremyb>	 ^demon: august 9, NYLUG at google nyc: the drupal project talks about their migration from cvs to git
[19:07:30] <^demon>	 I'll be in SF then.
[19:07:38] <jeremyb>	 oh
[19:07:50] <jeremyb>	 i assumed you wouldn't come anyway. at least we could get you the video
[19:09:02] <^demon>	 Video is cool :)
[19:09:06] <^demon>	 I could watch it on my plane ride home
[19:09:30] <jeremyb>	 heh. idk if they are *that* fast. and i might forget
[19:09:35] <jeremyb>	 when do you come back?
[19:10:33] <^demon>	 Fly back on the 12th.
[19:14:00] <jeremyb>	 k
[19:24:45] <logmsgbot>	 !log aaron synchronized php-1.20wmf8/includes/DefaultSettings.php  'deployed 718e305c2213125b1df41323f22b7db400a77139'
[19:24:53] <morebots>	 Logged the message, Master
[19:25:36] <logmsgbot>	 !log aaron synchronized php-1.20wmf8/includes/filerepo/FileRepo.php  'deployed 718e305c2213125b1df41323f22b7db400a77139'
[19:25:43] <morebots>	 Logged the message, Master
[19:26:05] <logmsgbot>	 !log aaron synchronized php-1.20wmf8/thumb.php  'deployed 718e305c2213125b1df41323f22b7db400a77139'
[19:26:12] <morebots>	 Logged the message, Master
[19:31:31] <logmsgbot>	 !log aaron synchronized php-1.20wmf7/includes/filerepo/FileRepo.php  'deployed 3f79e142beade8135514a3ac2a1af2cdd8a30901'
[19:31:39] <morebots>	 Logged the message, Master
[19:31:51] <logmsgbot>	 !log aaron synchronized php-1.20wmf7/includes/DefaultSettings.php  'deployed 3f79e142beade8135514a3ac2a1af2cdd8a30901'
[19:31:58] <morebots>	 Logged the message, Master
[19:32:10] <logmsgbot>	 !log aaron synchronized php-1.20wmf7/thumb.php  'deployed 3f79e142beade8135514a3ac2a1af2cdd8a30901'
[19:32:17] <morebots>	 Logged the message, Master
[20:25:59] <Nemo_bis>	 ^demon: this says fixed, right? can we increase the query limit on our install? https://code.google.com/p/gerrit/issues/detail?id=934
[20:29:03] <Reedy>	 Do you really need more?
[20:35:29] <Nemo_bis>	 Reedy: yes, for instance sumanah had to use three queries to get her stats on monthly committers
[20:35:49] <Reedy>	 Right, ok
[20:35:50] <Reedy>	 I was just checking their was a valid use case
[20:35:50] <Reedy>	 rathe rthan "cause"
[20:36:06] <Nemo_bis>	 and be forced to create scripts for everything is not nice
[20:38:18] <Reedy>	 https://gerrit-review.googlesource.com/#/c/23053/
[20:38:20] <Reedy>	 It was merged a while back
[20:38:36] <Reedy>	 "This limit applies not only to the `gerrit query` command, but also to the web UI results pagination size."
[20:38:36] <Reedy>	 yay
[20:39:17] <Reedy>	 Nemo_bis: log a bugzilla bug under gerrit asking for the config change
[20:39:45] <Nemo_bis>	 Reedy: oki thanks
[20:41:10] <Reedy>	 I'd make the change in the puppet repo myself, but it seems we don't have a project.config file..
[20:45:14] <Nemo_bis>	 hmpf https://bugzilla.wikimedia.org/show_bug.cgi?id=38768
[20:45:52] <Elfix>	 it appears that it hasn't been reported yet that many users from france are having trouble connecting to Wikimedia projects
[20:46:09] <Elfix>	 it times out 2 times out of 3
[20:48:33] <RoanKattouw>	 Elfix: Of course this has to happen on our network person's day off :S
[20:50:20] <Elfix>	 RoanKattouw: well, it may not be a problem on WMF side anyway
[20:50:29] <Elfix>	 3 people having that issue use the same ISP
[20:50:51] <Elfix>	 faulty DNS I would assume, it's not the first time it happens
[21:04:51] <Krenair>	 https://bugzilla.wikimedia.org/show_bug.cgi?id=38766
[21:05:08] <Krenair>	 To fix this a shell user would have to run maintenance/updateSpecialPages.php wouldn't they?
[21:06:34] <Reedy>	 It's the same for enwiki
[21:06:53] <Reedy>	 (and every other wiki?)
[21:10:52] <Elfix>	 it's the same for all big wikis
[21:11:15] <Elfix>	 (at least)
[21:12:43] <Reedy>	 Bug closed
[21:18:05] <Nemo_bis>	 Krenair: I was just reading https://bugzilla.wikimedia.org/show_bug.cgi?id=14786
[21:28:00] <jeremyb>	 Reedy: where did you get the name project.config from?
[21:28:30] <Reedy>	 "Administrators can add a global block to All-Projects project.config file with group(s) that should have different limits:"
[21:28:56] <jeremyb>	 where is that?
[21:29:01] <Reedy>	 https://gerrit-review.googlesource.com/#/c/23053/
[21:29:29] <jeremyb>	 that's not working... it wants me to log in?
[21:29:39] * jeremyb  doesn't understand why

[21:29:49] <Reedy>	 chrome dev by any chance?
[21:30:37] * jeremyb  ? no

[21:30:52] <Reedy>	 current chrome dev build has a bug where it doesn't work with gerrit
[21:30:54] <Reedy>	 fixed in canary
[21:31:56] <jeremyb>	 no, it really wants me to log in
[21:32:00] <jeremyb>	 i don't get it
[21:32:27] <Reedy>	 WFM on canary
[21:32:57] <jeremyb>	 Reedy: The page you requested was not found, or you do not have permission to view this page.
[21:33:05] <jeremyb>	 even now that i managed to log in
[21:33:23] <Reedy>	 Google have blacklisted you
[21:33:48] <jeremyb>	 can you pastebin?
[21:33:57] <Reedy>	 +807, -212
[21:34:12] <Reedy>	 http://p.defau.lt/?Po9F8FVn0S688lnuZizvpA
[21:48:38] <Reedy>	 commonswiki:  Getting list of files to clean up...
[21:48:38] <Reedy>	 commonswiki:  Removing 124980 file(s)...
[21:48:38] <Reedy>	 lol.
[21:50:18] <jeremyb>	 ?
[21:53:37] <Nemo_bis>	 Reedy: who could be interested in working to wikidiff2 fixes?
[21:53:48] <Nemo_bis>	 basically only Tim worked on it
[21:54:15] <Reedy>	 Probably your anser
[21:54:18] <Reedy>	 *answer
[21:54:45] <Nemo_bis>	 aww
[21:59:47] <jeremyb>	 no demon
[22:00:02] <jeremyb>	 !bug 38768 | Reedy: Nemo_bis
[22:00:02] <wm-bot>	 Reedy: Nemo_bis: https://bugzilla.wikimedia.org/show_bug.cgi?id=38768
[22:00:24] <Reedy>	 pipe hacks
[22:00:46] <Reedy>	 It might be it's just not been added to git..
[22:01:14] <jeremyb>	 Reedy: did you read what I wrote? ;)
[22:01:48] <Reedy>	 Not completely :p
[22:02:03] <jeremyb>	 speak of the devil!
[22:02:04] <Nemo_bis>	 jeremyb: looks ok then? :)
[22:02:17] <Reedy>	 No
[22:02:23] <Reedy>	 The config is there on the labs dev instance
[22:02:28] <Reedy>	 it's apparently completely MIA on production
[22:02:31] <jeremyb>	 !bug 38768 | ^demon
[22:02:31] <wm-bot>	 ^demon: https://bugzilla.wikimedia.org/show_bug.cgi?id=38768
[22:02:54] <^demon>	 I just left a comment.
[22:02:59] <^demon>	 It's there on production too.
[22:03:10] <^demon>	 But it's unrelated to the discussion at hand.
[22:03:12] * jeremyb  waits for email to catch up

[22:03:16] <jeremyb>	 how so?
[22:03:27] <Reedy>	 open the bug ;)
[22:03:35] <Nemo_bis>	 Reedy: now I'm looking for users ranting about the diff engine to see if someone might be able to commit a patch against it instead...
[22:03:36] <^demon>	 What does it being hidden have to do with increasing the query limit?
[22:03:57] <jeremyb>	 damnit, why didn't the mail come in yet?
[22:04:09] <jeremyb>	 oh, because it already did!
[22:04:58] <jeremyb>	 ^demon: because Reedy said he filed only because it wasn't in gerrit "yet"
[22:05:05] <jeremyb>	 ^demon: (i guess puppet)
[22:05:08] <^demon>	 It is.
[22:05:12] <^demon>	 Doesn't need puppet change.
[22:05:13] <jeremyb>	 ^demon: so i was pointing to where it lives
[22:05:23] <^demon>	 I could change it in 20 seconds.
[22:05:37] <MaxSem>	 Nemo_bis, what did you want with wikidiff2?
[22:05:38] <jeremyb>	 now why can't i clone the dev copy?
[22:05:49] <Nemo_bis>	 MaxSem: a couple bugs fixed
[22:05:52] <jeremyb>	 fatal: http://gerrit-dev.wmflabs.org/p/All-Projects.git/info/refs not found: did you run git update-server-info on the server?
[22:05:56] <MaxSem>	 ha
[22:06:07] <^demon>	 jeremyb: dev instance is down.
[22:06:13] <jeremyb>	 ^demon: i started it
[22:06:19] <Nemo_bis>	 MaxSem: there are only 3 open against it :)
[22:06:22] <^demon>	 Oh. Well blargh
[22:06:28] <jeremyb>	 seems up to me
[22:06:29] <^demon>	 It's being replaced :p
[22:06:48] <jeremyb>	 wow it's late
[22:06:58] <Nemo_bis>	 MaxSem: https://bugzilla.wikimedia.org/show_bug.cgi?id=13462 https://bugzilla.wikimedia.org/show_bug.cgi?id=13466
[22:07:50] <Nemo_bis>	 jeremyb: is gerrit's DB replicated in Labs?
[22:07:59] <jeremyb>	 no
[22:08:08] <jeremyb>	 what diff does it make though?
[22:08:12] <Nemo_bis>	 and could it possibly ever be
[22:08:22] <MaxSem>	 13462 is unaccpetable for perf reasons /me afraid
[22:08:23] <jeremyb>	 idk...
[22:08:30] <Nemo_bis>	 jeremyb: that people can't run their own queries
[22:08:42] <jeremyb>	 Nemo_bis: ohhhh
[22:08:55] <jeremyb>	 Nemo_bis: isn't that just being fixed by removing stuff from the DB?
[22:09:05] <Nemo_bis>	 hmmm
[22:09:22] <Reedy>	 MaxSem: CLOSEITCLOSEIT
[22:09:29] <jeremyb>	 Nemo_bis: every or nearly every gerrit release removes something from the DB (destructively)
[22:09:32] * Nemo_bis  slaps Reedy 

[22:09:36] <jeremyb>	 i think
[22:09:49] <^demon>	 Pretty much. Especially if it's project-related metadata.
[22:09:57] <Nemo_bis>	 MaxSem: how can you know? there's a clear solution and implementation of it and tests of the implementation? :)
[22:10:05] <Nemo_bis>	 MaxSem: the situation looked more unclear to me
[22:10:06] <^demon>	 I'm unsure how they plan to move user data out of the DB, but that remains to be seen.
[22:10:58] <MaxSem>	 Nemo_bis, the solution is trivial, as well as its computational complexity analysis
[22:11:41] <Nemo_bis>	 MaxSem: I see. Can you comment?
[22:12:09] <MaxSem>	 something like (number of lines)^2 * (avg number of characters in line) ^ 2
[22:12:22] <Nemo_bis>	 :/
[22:12:42] <MaxSem>	 the first part can be shortened a bit to ~number of changed lines
[22:12:50] <Nemo_bis>	 MaxSem: are dwdiff and such so slow?
[22:13:16] <MaxSem>	 but Levenshtein distance is still too slow for our scale
[22:14:14] <Nemo_bis>	 maybe we could make it a configuration variable and enable it only on mediawiki.org where it harms the productivity of devs? :p
[22:14:52] <MaxSem>	 nah
[22:15:31] * jeremyb  runs away

[22:15:40] <Nemo_bis>	 in layman terms, what would that computational cost mean? application servers going from 10 to 20 % load?
[22:15:56] <Nemo_bis>	 :)
[22:18:34] <Nemo_bis>	 aka is there a way to evaluate it (besides turning it on and looking at the results)
[22:21:48] <MaxSem>	 let's say so: there's a bug about some diffs taking  40s to render, smart diffs might square this (aka 1600 seconds). of course, in reality it will be a bit faster, but still these smart diffs look scary
[22:24:35] <Nemo_bis>	 MaxSem: ah
[22:34:17] <Nemo_bis>	 of course I can't find it
[22:34:33] <Nemo_bis>	 MaxSem: are you sure of that figure? 
[22:35:02] <Nemo_bis>	 Aryeh https://bugzilla.wikimedia.org/show_bug.cgi?id=8981#c3 "diffs were very slow at one point, now they're a few ms"
[22:35:25] <Nemo_bis>	 brion https://bugzilla.wikimedia.org/show_bug.cgi?id=932#c1 "The PHP version (roughly the same code as in 
[22:35:28] <Nemo_bis>	 1.3) takes a whopping *41.5* seconds
[22:36:17] <Nemo_bis>	 and finally https://bugzilla.wikimedia.org/show_bug.cgi?id=7601 is about standard diff engine
[22:36:21] <MaxSem>	 22559 is explicitly against wikidiff2
[22:37:48] <Nemo_bis>	 sigh, I already knew that bug so I ignored it, hmpf
[22:46:56] <Nemo_bis>	 uh, found the diff domas mentioned: eg https://en.wikipedia.org/w/index.php?title=User:JamesR/AdminStats&diff=342751956&oldid=342740711
[22:47:00] <Nemo_bis>	 took 2 min for me
[22:48:49] <Nemo_bis>	 now it would be interesting to know how many pages have that many rows
[22:51:40] <Nemo_bis>	 MaxSem: it's istantaneous with dwdiff though
[22:55:11] <Nemo_bis>	 goes from 320 to 440 ms in the two testcases, so surely not square
[22:55:44] <Nemo_bis>	 while diff from 4 to 20