[00:07:04] hi sumanah , this is me commuting :) [00:07:55] ha! [00:07:58] sumanah, may I ask you a favor? Please let me know if I start disconnecting / connecting in an annoying way all the time, thanks to the Bay Area 3G coverage [00:08:05] oh you left the office when I wasn't looking? you sneaky sneak [00:08:11] okay, will do qgil_ [00:08:18] sumanah, :) [00:50:53] sumanah, ok approaching Mountain View. Not bad, seems to work. [00:51:00] good, good [00:51:17] :) see you tomorrow! [00:51:27] bye! [09:30:38] New patchset: Hashar; "enable triggering of the 'cldr' extension" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36717 [09:30:52] Change merged: Hashar; [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36717 [09:34:45] New patchset: Hashar; "cldr does not have any unit test yet, disable them" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36719 [09:35:02] Change merged: Hashar; [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36719 [09:42:43] New patchset: Hashar; "enable reporting for Translate extension" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36721 [09:43:00] Change merged: Hashar; [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36721 [09:50:36] New patchset: Stefan.petrea; "Refactoring bash scripts for generating reports" [analytics/wikistats] (master) - https://gerrit.wikimedia.org/r/36722 [09:58:51] New patchset: Hashar; "enable reporting for TranslationNotifications ext" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36725 [09:59:41] Change merged: Hashar; [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36725 [10:03:43] New patchset: Hashar; "enable reporting for UniversalLanguageSelector extension" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36726 [10:03:53] Change merged: Hashar; [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36726 [10:09:01] New patchset: Hashar; "enable reporting for MobileFrontend extension" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36727 [10:09:22] Change merged: Hashar; [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36727 [10:48:25] New patchset: Hashar; "job for mw/ext/cldr" [integration/jenkins-job-builder-config] (master) - https://gerrit.wikimedia.org/r/36729 [10:48:38] Change merged: Hashar; [integration/jenkins-job-builder-config] (master) - https://gerrit.wikimedia.org/r/36729 [10:57:23] New patchset: Hashar; "linter did not work properly on extensions" [integration/jenkins-job-builder-config] (master) - https://gerrit.wikimedia.org/r/36732 [10:57:35] Change merged: Hashar; [integration/jenkins-job-builder-config] (master) - https://gerrit.wikimedia.org/r/36732 [11:23:35] New patchset: Hashar; "job for mw/ext/VisualEditor" [integration/jenkins-job-builder-config] (master) - https://gerrit.wikimedia.org/r/36737 [11:23:52] Change merged: Hashar; [integration/jenkins-job-builder-config] (master) - https://gerrit.wikimedia.org/r/36737 [11:26:27] New patchset: Hashar; "enable reporting for VisualEditor" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36738 [11:26:44] Change merged: Hashar; [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36738 [11:57:06] New patchset: Hashar; "jslint jobs (non voting for now)" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36744 [11:57:20] Change merged: Hashar; [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36744 [11:58:43] Reedy: hi! i want to do a double check on the wb_terms schema. can you give me the result of SHOW CREATE TABLE wb_terms on wikidatawiki? [11:59:16] i need to know in particular the charset/collation used for each field [12:01:09] New patchset: Hashar; "jslint jobs for extensions" [integration/jenkins-job-builder-config] (master) - https://gerrit.wikimedia.org/r/36745 [12:01:21] Change merged: Hashar; [integration/jenkins-job-builder-config] (master) - https://gerrit.wikimedia.org/r/36745 [12:04:16] DanielK_WMDE_: Yeah, give me a few minutes [12:06:37] DanielK_WMDE_: http://p.defau.lt/?CkWJIJAIobguqYFe1L2ciw [12:06:45] Reedy: i dug up the schema the toolserver uses. that *should* be the same, but would be great if I could confirm with your info. [12:06:57] Reedy: tanks! [12:07:58] Reedy: errr. wb_terms, not wb_changes, please :) [12:08:10] bleugh [12:08:31] Amusingly, I did actually scroll back to a sql query I did against wb_terms last night [12:08:48] http://p.defau.lt/?2hzRY_1QRZxl6UKN7N9www [12:08:58] hm... are varchar columns are automatically (manually?) converted to varbinary? [12:09:58] Reedy: because locally, term_text is a varchar(255), and but on the cluster it's a varbinary, [12:10:06] is there a way to avoid this kind of conversion? [12:12:42] I note in core things are varbinary.. [12:13:26] but you've got both in wikidata, nvn [12:15:02] Reedy: well... in the actual database, the fields are all varbinary. even though wikibase.sql sais varchar [12:15:06] i wonder how that happens. [12:15:12] yeah, i noticd [12:15:16] lets see [12:15:20] or did we originally declare them as varbinary? [12:15:45] Nope [12:15:52] I just tested it creating the table on testwiki [12:15:58] copy paste current schema from master [12:16:08] varchar becomes varbinary [12:16:13] so it's something mysql side that's doing it [12:16:49] o_O [12:17:03] ah... varbinary is the same as varchar() binary. [12:17:24] so if you have DEFAULT CHARSET BINARY, i suppose that is what'll happen [12:17:57] hm, right. I guess that explains it. [12:18:05] we should be careful about this. [13:15:24] New patchset: Hashar; "job for mw/ext/SVGEdit" [integration/jenkins-job-builder-config] (master) - https://gerrit.wikimedia.org/r/36752 [13:15:53] Change merged: Hashar; [integration/jenkins-job-builder-config] (master) - https://gerrit.wikimedia.org/r/36752 [13:17:36] New patchset: Hashar; "enable reporting for SVGEdit" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36753 [13:18:07] Change merged: Hashar; [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36753 [13:24:02] New patchset: Hashar; "actually make jslint jobs non voting" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36755 [13:24:22] Change merged: Hashar; [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/36755 [15:52:50] Change merged: Stefan.petrea; [analytics/wikistats] (master) - https://gerrit.wikimedia.org/r/36722 [18:27:32] thx for the reply kaldari - gosh this issue is annoying [18:28:04] ? [18:28:26] sumanah: definitely [18:28:43] I'd like to avoid repeating yesterday x 100. [18:30:37] andre__: BTW, in WMF contexts 'Ryan' usually means Ryan Lane, while Faulker and myself go by last names to avoid confusion :) [18:31:09] er Faulkner [18:31:53] <^demon> And Rob is usually RobH, whereas RobLa is robla. [18:32:14] robla's name is rob? weird :) [18:32:14] <^demon> I, however, am the only Chad. Bwahaha. [18:32:19] * RobH eyeballs chad [18:32:26] gratuitous ping [18:32:35] ^demon: ping [18:32:36] ^demon: ping [18:32:36] ^demon: ping [18:32:39] revenge! [18:32:54] <^demon> kaldari: I really wish RobH would stop pinging me. [18:32:59] Hey All, it's Chip. Does anyone have a moment to whitelist me for the staff channel? [18:33:01] <^demon> RobH needs to learn when it's appropriate to not ping. [18:33:06] kaldari: I was afraid of that :) [18:33:07] hehe [18:33:24] James_F: ^ [18:33:37] cndiv: Hey. [18:33:38] cndiv: James usually does that i think [18:33:55] * sumanah will not always be the only Sumana! [18:33:57] or i just keep sending folks to him and he will snap and stab me when he sees me next. [18:34:07] cndiv: Done. [18:34:16] woohoo, thank you sir [18:34:21] * James_F bows [18:34:33] sumanah: if you claim sumanah@ email then you are the only one [18:34:44] and future sumanahs will be sumanah+last initial at best. [18:34:55] or sumana [18:34:59] but my first name is Sumana, so that's what I'd need to claim [18:34:59] <^demon> RobH: sumanah is sumana + last initial ;-) [18:35:13] * sumanah feels weird seeing her name over and over [18:35:15] well, last multiple initials [18:35:34] there are now three or four robs. [18:35:47] unfortunately [18:35:48] ;) [18:35:49] whenever a new one is hired i get an email destined for them to me. [18:35:57] * sumanah will know she has succeeded at recruiting women & the GLOBAL SOUTH when we have multiple Sumanas [18:36:27] you can only fit 3 Ryans in the Ryangle [18:41:22] AaronSchulz: is this — the correct change-set? [18:41:53] you mean https://gerrit.wikimedia.org/r/#/c/16696/ [18:42:04] AaronSchulz: argh [18:42:18] AaronSchulz: yes, I copied and pasted the wrong tab [18:44:12] AaronSchulz: Have you ever looked at: https://bitbucket.org/wez/couchshare/src/bcbf02e1a70d/OmniTI/Util/UUID.php [18:45:24] I think I looked at that months ago, or at least some uid thing from omniTI [18:45:51] AaronSchulz: Have you read RFC 4122 [18:46:05] AaronSchulz: e.g., http://www.ietf.org/rfc/rfc4122.txt [18:46:51] that page doesn't load [18:46:57] if it's the uuid spec, then yes [18:49:16] preilly: I could probably split the uid stuff as a separate commit even [18:49:38] AaronSchulz: I'd like that [18:49:56] AaronSchulz: I was actually just thinking about doing that myself [18:50:09] AaronSchulz: But you can go ahead and do it as well [18:50:19] go ahead, as long as the author and stuff is correct ;) [18:58:44] AaronSchulz: okay smarty pants [19:00:34] AaronSchulz: see https://gerrit.wikimedia.org/r/#/c/36801/1 [19:02:26] that was fast [19:03:13] did you amend this out of the old patch now? [19:36:48] New review: Hashar; "Thanks for your work on Maps :-] I have changed the system to use a python script that generate job..." [integration/jenkins] (master); V: 0 C: 0; - https://gerrit.wikimedia.org/r/32834 [19:44:26] kaldari, oh the irony of this [19:44:42] :) [19:44:49] You want to be able to give out edit rights [19:45:08] But people who don't have edit rights can't participate [19:45:18] yes this is a problem :) [19:45:35] all the people who I could most easily bribe can't help me! [19:46:20] your WikiLove cannot save you now? [19:47:25] Ah, looks like Eloquence just accepted it [19:47:56] looks like I greased the right palms (or something like that) [19:49:23] lol chad... [20:18:40] csteipp: got a sec? [20:18:51] robla: Sure [20:19:25] csteipp: how are wikivoyage pseudousers implemented? [20:19:45] (the pre-merge users, that is) [20:20:09] So at the old wikivoyage, they existed like IP addresses-- not real users. [20:20:24] When we transfered the data, Hans made them into real users [20:20:38] (found them all, and then added a row in the user table for each one) [20:20:55] the accounts are just locked then? [20:20:59] Each was named with the (WT-xx) prefix [20:21:01] Yep [20:21:09] Then we went in and locked all of the accounts [20:21:18] And created userpages with the template on it [20:21:57] At least, they should be locked... [20:22:09] They don't have anything in the password field in the database [20:24:30] ok....just poking at http://it.wikivoyage.org/wiki/Utente:(WV-it)_Luke_Westwalker [20:24:53] which, you can see the text of here: http://it.wikivoyage.org/wiki/Utente:(WV-it)_Luke_Westwalker?action=edit [20:25:35] Hmm.. that's not good [20:25:53] it looks like MediaWiki is getting confused as to whether or not "gen" interwiki prefix is a valid interwiki prefix [20:26:02] backtrace: https://bugzilla.wikimedia.org/show_bug.cgi?id=42649 [20:26:59] Ah. So that was added in at some point after the conversion-- we only added the "{{Wmf_import_prefix}}" when we imported.. [20:27:35] Ah, so that was on the userpage before we imported. [20:39:53] preilly: https://gerrit.wikimedia.org/r/#/c/36803/ [20:40:48] AaronSchulz: Change merged: preilly; [mediawiki/core] (master) - https://gerrit.wikimedia.org/r/36803 [20:41:03] also https://gerrit.wikimedia.org/r/#/c/36816/ [20:45:56] preilly: if I can make everyone's life just a little more miserable... [20:46:37] AaronSchulz: fixed [20:46:52] AaronSchulz: see https://gerrit.wikimedia.org/r/#/c/16696 now [20:46:54] * AaronSchulz looks for patchset 5 million [20:47:05] heh [20:48:00] AaronSchulz: can you please review patch-set -1.8959581529998104E-4 [20:52:31] * preilly — realizes that it's 7:52am Wednesday (EST) [20:52:37] Tim-away: ping [21:14:09] andre__,^demon: BTW, really loving the upgraded Bugzilla's HTML e-mails. Thanks. :-) [21:15:50] urgh. [21:15:55] we send HTML by default? [21:15:55] <^demon> James_F: I didn't do it :p [21:15:56] I hope not. [21:16:07] oh crap. [21:16:09] I hate that. [21:16:14] <^demon> James_F: I don't want to be blamed for those ;-) [21:16:18] andre__: Yeah, the new BZ has switched over. It's really helpful for the tables they used to send. [21:16:29] ^demon: OK, then I only have a complaint for you. ;-) [21:17:03] * andre__ switches his mail client to prefer Plain Text over HTML [21:17:28] ^demon: Namely, I'm not entirely convinced that TWB should get self-+2; it's trivial work for humans to merge, and VE alone has had two breaks from it doing so in the past year. [21:18:02] <^demon> Two breakages doesn't outweigh the thousands of commits that occur without problem. [21:19:38] ^demon: Two breakages in one repository of 300? [21:20:23] <^demon> It's not trivial work at all--it was a huge timesink to manually merge them before we had auto-review working. It's part of the expected workflow for all developers (as it was in SVN). [21:20:25] ^demon: Even when humans, from whom we expect more comprehension, do mostly-trivial commits we ask for it to be +2'ed by others. :-) [21:20:39] <^demon> I trust L10n-bot more than I trust most of our users. [21:20:48] * James_F sighs. [21:21:14] <^demon> In theory, a repo could override the permissions if they really want to do it manually for their repo. But I'm not going to *help* anyone shove beans up their nose. [21:21:21] * James_F nods. [21:22:11] VE gets a TWN bot commit about once a day right now, so I suppose we don't see how big it is for core. [21:23:28] <^demon> All repos get at most 1 commit per day, but that's times ~450. [21:23:38] <^demon> 450 extra commits per day to approve does not scale :( [21:23:54] <^demon> Granted that's the upper limit, but still. [21:29:45] ^demon: Oh, I thought it was hourly? [21:30:09] <^demon> No, daily. [21:30:28] ^demon: Ideally every repo should get more than 1 commit a day from non-robots anyway. So, "ideally", it would be noise. But I accept your experience with this. :-) [21:34:25] James_F: it's triggered and checked manually, so it can't be hourly anyway [21:34:31] (not that it would be desirable) [21:35:05] * James_F nods. [22:12:38] AaronSchulz/robla/preilly: you know what I think of multi-author patchsets, right? [22:13:42] at least they have a common parent, so it might be possible to find out what changed [22:15:44] you know, this is what topic branches are meant to be used for [22:15:54] they are meant to be actual branches, not just names we give to a single change [22:33:32] TimStarling: What's the issue? [22:34:09] just reviewing this RDBStore thing that robla asked me to look at today [22:34:54] I guess that means no time to look at 34062 again [22:35:43] MaxSem: did you see my comments on https://gerrit.wikimedia.org/r/#/c/36892/ ? [22:35:57] I'm not sure what the relative priority is [22:36:12] TimStarling: AFTv5 is depending on it [22:36:28] TimStarling: That is the reason it's getting so much attention right now [22:36:36] TimStarling: RE: RDBStore [22:36:39] AaronSchulz, no - I noticed it independently:P [22:36:56] how will it be configured? [22:37:14] are there actually multiple servers lined up to be used in this way, or is it just fantasy? [22:37:35] * AaronSchulz thought it would be fantasy mariadb servers [22:37:51] ;) [22:37:55] TimStarling: right now it's just fantasy [22:38:10] TimStarling: binasher is going to chime in [22:38:31] TimStarling: but once you review it — it won't be [22:38:42] AaronSchulz, that is you think just PoolCounter is enough? [22:38:44] TimStarling: i opened a provisioning ticket about two months ago, i think servers available but i'm not sure of the status [22:38:55] MaxSem: job + PC [22:39:17] that would make stronger guarantees [22:39:31] I think I asked this before, but can you remind me, what is the data set size? [22:39:39] MaxSem: and having a PoolCounterWork class in a SolrUpdateJob.php file seems confusing [22:40:19] AaronSchulz, I just don;t like the Java custom of creating separate files for every 3-line class:P [22:40:39] it's not about java, it's about not being misleading [22:40:43] order of magnitude? [22:41:41] AaronSchulz, so I do use PC+job queue as the maint script does everything via PC [22:42:14] hmm, I see [22:42:37] wait [22:42:49] MaxSem: if you push the jobs the normal runners with do them right? [22:42:57] and I don't see any PC code in run() [22:43:48] * AaronSchulz also isn't super familiar with this ext [22:44:17] it calls $maint->execute() which calls $work = new SolrUpdateWork( $this ); and !$work->execute() [22:44:53] ah, that works [22:44:56] class SolrUpdate extends Maintenance [22:45:18] it seems odd to have a job call execute() for a Maintenance class [22:45:58] thats probably why I wasn't seeing that before [22:46:41] I tried to factor out chunks of the maintenance job - this looked even uglier with all those adaptor abstracting output(), error() and getDB() [22:47:38] TimStarling: I've not made any real changes to https://gerrit.wikimedia.org/r/#/c/16696 [22:47:47] TimStarling: I just moved one file out of it [22:48:16] yeah, for most of those patchsets I haven't been able to get git to give me any sort of diff [22:49:15] MaxSem: what happens if these jobs are run in sapi mode? [22:49:17] so aft_article_answer seems to be the biggest aft_* table, and it has 1.1M rows [22:49:24] for third parties [22:50:51] c.f. pagelinks 651M rows [22:51:01] AaronSchulz, if there's a lot of data to be pushed, the job might me a bit slow (but not terribly slow - initial import for enwiki takes ~5mins). but job updates are disabled by default anyway [22:51:39] I assume the docs tell people do set this stuff up with runners anyway [22:52:27] for small wikis cron updates would be optimal performance wise [22:52:43] yeah [22:55:39] ehhh Job vs. Work [22:56:00] AaronSchulz: have you looked at http://code.google.com/p/shard-key-mapper/ [22:56:18] no [22:56:57] AaronSchulz: okay [22:57:47] so, I'm torn between saying "whatever, just merge it, I don't care" and "you don't need 1000 lines of code and 8 servers to store 100MB of data" [22:58:07] TimStarling: I'm feeling just about the same way [22:58:28] the second followed by "just patch it out of AFTv5" [22:58:33] * preilly that is if you're talking about Change-Id: Ic1e38db3d325d52ded6d2596af2b6bd3e9b870fe [22:58:37] one could just vertical partition yes...make a new aft5 server and call it a day [22:58:47] yes, that's what I'm talking about [22:59:15] AaronSchulz: should we just Abandon Change-Id: Ic1e38db3d325d52ded6d2596af2b6bd3e9b870fe [23:00:14] my concern is that ideally, we would design this class to work in a case where it's actually needed (like pagelinks) [23:00:28] TimStarling: Yeah, that's a great point [23:01:26] or revision, it's 473M rows now [23:01:50] TimStarling: another good candidate for sure [23:03:31] AaronSchulz: I'm voting for just patch RDBStore out of AFTv5 [23:04:03] preilly: and instead make the DB used configurable so a different server can be used [23:04:04] i'm all for that too [23:04:08] I'd agree with that [23:04:21] AaronSchulz: exactly [23:04:29] binasher: awesome [23:04:53] AaronSchulz: don't get me wrong I totally want something like RDBStore at some point for sure [23:04:58] preilly: leaving the patch in gerrit is fine though [23:05:11] providing sharding for a big existing table like pagelinks is going to be more difficult than using it for new non-essential data [23:05:31] but that might mean different interfaces [23:05:40] AaronSchulz: I vote to abandon it right now [23:05:43] the ability to dump aftv5 to its own db would be nice, but rdbstore not needed [23:05:44] it doesn't necessarily make AFTv5 a good model for pagelinks [23:05:59] binasher: right [23:06:05] AaronSchulz: a big existing table while drive the design in a different direction [23:06:11] AFTv5 is a sink for unlimited garbage [23:06:27] s/while/will [23:06:28] preilly: flaggedtemplates/flaggedimages is kind of big on dewiki [23:06:31] you can just do wfGetDB(DB_MASTER,array(),'aft')->insert('aft', array( 'aft_wiki' => wfWikiId() ) ); [23:06:35] binasher: true that [23:06:42] a text box that saves to a blob open to any anon or bot.. fun times [23:06:49] TimStarling: or use the factory() function [23:07:14] binasher: it's a design pattern [23:07:21] yes [23:07:37] maybe we can have a special janky cluster for data that follows that design pattern [23:07:47] binasher: now your thinking [23:07:47] * AaronSchulz thought of making a job queue option for that a while back ;) [23:08:03] job-priority: meh [23:08:05] <^demon|busy> We've had this problem with leaving open text boxes to anons forever. It's called action=edit ;-) [23:08:20] binasher: you just call wfGetDevNull [23:08:34] ^demon|busy: ha ha ha [23:08:55] ^demon|busy: see, that's why revisions are stored in their own janky cluster(s) [23:09:28] ugh, I hate that code [23:09:36] let's bring back mysql on apaches for aft [23:09:57] hehe [23:10:10] I accidentally my AFT5 [23:10:22] it was a nice solution for zero-cost terabyte-scale storage [23:10:52] whether you think that was a good thing to aim for is another matter ;) [23:12:03] the storage requirements of AFT should be pretty modest compared to regular text, before recompression [23:12:25] because with regular text, we duplicate the article every time someone adds "poop" to the end of a random line [23:13:26] TimStarling: ha ha ha [23:14:02] that's another thing, someone should probably run that recompression script [23:15:17] then we can fit the whole site on jimmy's laptop, like he always wanted [23:16:19] AaronSchulz: I'll look at async upload concatenation now [23:23:51] AaronSchulz: this is what I was talking about: https://www.mediawiki.org/wiki/Gerrit/personal_sandbox [23:32:05] AaronSchulz: just wait 10 minutes while I try to work out what you changed between patchset 8 and 11 [23:33:30] hey, I wonder if a later patchset can be cherry-picked into a branch containing an older patchset [23:34:34] nope [23:44:05] 11 minutes [23:44:12] TimStarling: was there an automatic rebase in between? [23:45:15] patchsets 8, 9, 10 and 11 all had different parents so I guess git-review is rebasing it for you [23:45:30] it does, unless you have -R [23:45:36] and I always forget to do that [23:45:58] I think it is configurable [23:46:57] - 'async', 'Make potentially large file operations asyncronous when possible', [23:46:57] + 'async', 'Make potentially large file operations asyncronously when possible', [23:47:20] is that some re-regression or something [23:48:09] and missing an H both times [23:50:43] it can have +2 [23:51:08] but read my latest comment on https://gerrit.wikimedia.org/r/#/c/34062/8/includes/upload/UploadBase.php,unified