[10:10:03] New review: Siebrand; "Antoine, what's the status of this?" [integration/jenkins-job-builder-config] (master); V: 0 C: 0; - https://gerrit.wikimedia.org/r/38555 [11:44:17] New patchset: Siebrand; "Enable CodeSniffer for Translate" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/45744 [11:44:44] New review: Siebrand; "Not sure if this is correct. Otherwise, please point me in the right direction." [integration/zuul-config] (master); V: 0 C: 0; - https://gerrit.wikimedia.org/r/45744 [14:24:16] <^demon> qchris: Oh, one tool I forgot to tell you about: we do nightlies of gerrit & plugins we care about via jenkins: https://integration.mediawiki.org/ci/view/Java/ [14:24:22] <^demon> Good for helping spot regressions :) [14:25:01] Ah Jenkins :-) [14:25:07] Yes I encountered it before. [14:25:20] <^demon> I need to get its and bugzilla in there too. [14:25:37] <^demon> They'll fail probably since they rely on 2.6-SNAPSHOT of core right now. [14:25:41] But first the commits need to get merged :-) [14:25:53] Yes, think so too. [14:26:00] Should we backport to 2.5? [14:27:00] geee. I just saw WMF is at 2.4.2 [14:27:00] <^demon> We can try. I know there's plans for a 2.5.2 release soon. [14:27:21] Should we backport to 2.4.2 as well? [14:27:25] <^demon> Yeah, we've been waiting on a bunch of group-related stuff to be merged. 2.5.x is totally broken for WMF. [14:27:39] <^demon> 2.4.x isn't possible--plugin interface didn't exist til 2.5. [14:27:47] Ok :-)) [14:27:54] <^demon> I'm planning to upgrade us to something close to 2.6/master very very soon. [14:28:01] <^demon> One last stacktrace I'm trying to debug. [14:28:10] Ok. I see. [14:28:28] So I'll give some TLC to hooks-bugzilla documentation? [14:28:47] Or is there anything to improve in hooks-bugzilla? [14:29:28] <^demon> Documentation is good. How do you feel about the commenting/workflow parts? [14:30:09] hooks-bugzilla is small enough that I would not see too many issues there [14:30:22] but documentation has a faulty sample config [14:30:30] <^demon> Yeah, it was mostly copy+pasted from jira. [14:30:37] <^demon> I wouldn't be surprised if it's not totally correct. [14:30:41] and the sample REs produce non-working links etc [14:31:02] I thought so. Yes. That's why I want to fix them [14:31:09] <^demon> Go for it :) [14:31:16] Fine :-) [14:31:29] <^demon> saper: Have you ever hit a OrmConcurrencyException? [14:31:36] no [14:31:41] What's on the agenda for me afterwards? (As I'll probably spend some hours on the weekend working for WMF) [14:31:43] but sounds nice to debug [14:32:00] I remember glancing at the gwtorm source code [14:32:12] <^demon> http://p.defau.lt/?rUL1Ahn1JlI7fv6hErQ4bA - this is the only blocker to a 2.6 upgrade on us. [14:32:47] is it always when you delete or only occassionally? [14:33:00] <^demon> Always when deleting, only on LDAP backend. [14:33:08] <^demon> (ie: not on DEVELOPMENT_BECOME_ANY_ACCOUNT) [14:33:11] k [14:33:58] <^demon> Doesn't matter if group only has actual members, or internal includes, or ldap includes, or any combination of the 3. [14:34:48] <^demon> gerrit-dev.wmflabs.org is running 45a3621e4321054db00956ab7c647a793e38f1c5 right now. [14:35:04] probbaly a stupid question, but by "delete" you mean "deleting a group"? [14:35:11] <^demon> Deleting members from groups. [14:35:16] Ok. [14:35:17] <^demon> Or includes from groups. [14:35:44] shouldn't have anything with ldap really [14:36:06] <^demon> Probably not, but DEVELOPMENT_BECOME_ANY_ACCOUNT is a huge hack anyway so it wouldn't surprise me if that was Just Working. [14:36:12] <^demon> I haven't played with openid in awhile. [14:36:21] ok [14:36:25] I have openid here [14:36:28] postgresql and mysql [14:36:40] <^demon> I've been using mysql, since that's what we've got in prod. [14:36:48] yeah I know [14:36:51] <^demon> I haven't tried pg. Would suck if it was a mysqlism. [14:38:26] why does it try to update first, really [14:38:55] <^demon> Oh, bug possibly goes back to 2.2.x: https://code.google.com/p/gerrit/issues/detail?id=1165 [14:39:12] ah, that's INSERT IGNORE kindof stuff [14:44:59] <^demon> qchris: So, did you ever get your BZ install working enough to where the plugin would post a comment when a change was committed mentioning the bug? [14:45:14] <^demon> (I never got that far either) [14:45:22] Haven't tried that [14:45:24] are we getting a dev bz in labs? [14:45:43] <^demon> We should probably do that. [14:45:50] Yes. I will. [14:46:23] good, since I have this little tiny thing https://gerrit.wikimedia.org/r/#/c/44393/ [15:29:12] <^demon> saper: Remember when our gerrit install had screwed up utf-8? Fun times. [15:29:42] I've already wiped that out of my mind and am wondering how [15:29:59] 0.7 and 0.8 xml schemas managed to flip the order of sha1 and text elements with no on enoticing [15:30:00] yeah but that was only JDBC URL fix? [15:30:04] until me >_< [15:30:22] <^demon> saper: JDBC URL + changing default table/database encoding for future tables. [15:30:29] <^demon> And converted the tables to utf8. [15:31:00] uh [15:31:01] eh? [15:31:07] waitwait [15:31:07] sometimes I think MySQL should be renamed to BagOStuff [15:31:23] * saper waits. [15:31:25] whiiich tables? [15:31:36] <^demon> Gerrit. This was months and months ago. [15:31:40] oh whew nm [15:31:51] sorry my head is deep in mw tables [15:31:58] just /ignore me [15:31:58] <^demon> When Ryan installed gerrit, he made the mistake of following the instructions and installing with latin1. [15:32:00] <^demon> :) [15:32:04] ahahahaha [15:32:09] yeah *gong* and thanks for playing [15:32:24] hope they changed their docs [15:32:32] <^demon> Nope, docs are still totally wrong. [15:33:36] hahaha even better [15:33:46] <^demon> That we could probably easily fix. [15:33:48] now others will be left to suffer as we did [15:34:59] <^demon> Not as many people use utf8 in their comments & commit summaries :p [15:35:24] <^demon> There's some ancient patch to clean this all up, but we never really got around to it. [15:35:51] <^demon> (Obviously became less important since we fixed it for ourselves) [15:42:45] mm guesso [15:43:24] dangit now I am sucked into the gong show on you tube time machine and will never get out [15:43:26] example... [15:43:44] Gong Show - Feelings : The infamous show where every act sang "Feelings"; aired November 1976. [15:48:45] * saper does not check this out not to get sucked in [15:49:41] I'll just leave this here then.. https://www.youtube.com/watch?v=tzXMPDqICFE [16:13:12] <^demon> apergos: http://www.youtube.com/watch?v=GI6CfKcMhjY [16:13:28] and so it begins... [16:13:31] ^demon: can you log an actual query that fails on http://p.defau.lt/?rUL1Ahn1JlI7fv6hErQ4bA for me? [16:13:54] <^demon> I was about to enable the query log on gerrit-db and try that exact same thing. [16:13:56] I think I have some idea how to fix it [16:13:57] :) [16:15:35] <^demon> apergos: I think it's probably one of their best. [16:15:55] <^demon> And being old enough to know who Michael Bolton is makes is doubly funny. [16:24:24] <^demon> saper: http://p.defau.lt/?lMFOLpiU2l_1ty3_U_7aCg [16:24:52] <^demon> This was trying to delete Krinkle from https://gerrit-dev.wmflabs.org/r/#/admin/groups/10,members [16:24:57] thanks! that perfect [16:25:29] that's penalty for not using primary keys btw. [16:26:27] ^demon: how do you talk to mysql on gerrit-dev? [16:26:52] <^demon> I ssh'd to gerrit-db and connected directly. [16:27:01] ah ok [16:27:12] <^demon> I don't know if I have mysql-client installed on gerrit-dev. [16:27:19] do you mind if I install jython on gerrit-dev? [16:27:25] there is none that's why I'm asing [16:27:26] <^demon> That's fine. [16:27:42] strange [16:27:48] <^demon> It's well puppetized. I can spin up a new gerrit-dev install in < 30m if we ever break it badly. [16:28:01] apt-get wants to install x11 stuff, don't we have java there already? [16:28:10] <^demon> Should have openjdk-6-jre [16:28:35] ah it wants openjdk-6-jdk [16:28:44] <^demon> Ah, that's fine. [16:28:49] which tlots of crap [16:28:57] wit lots of crap [16:29:48] <^demon> I think it'll be fine. The data's all on project storage or gerrit-db. [16:30:16] <^demon> So even if we decide ewww and want to rebuild the vm, we'll live. [16:33:22] hm, access denied on -db using the right pwd for gerrit [16:34:22] <^demon> The mysql root password is stored in a file in /root :p [16:34:38] yeah it got changed [16:35:05] <^demon> Oh yeah I had to rebuild gerrit-db a couple weeks ago. Instance was messed up. [16:35:41] uff got it [16:39:38] and no maven? [16:39:47] or are you building locally [16:39:58] <^demon> I've been using builds from jenkins. [16:40:19] <^demon> (Occasionally a local build, but mostly builds from jenkins) [16:40:29] ok to install? [16:40:31] <^demon> They're all in /var/lib/gerrit2. [16:40:44] <^demon> Yeah, might need to force-run puppet after doing init. [17:08:52] <^demon> saper: Where on gerrit-dev did you build? Curious to see your patch. [17:12:31] <^demon> Oh, looking at wrong server. Still running jenkins #64. [17:36:15] ^demon: no, didn't change yet [17:36:36] need to play a bit with ORM, actually might be motivated to bring my jython thing back [17:36:51] but cooking first :) [17:38:29] <^demon> I wonder if the jython thing could be a plugin. [17:39:16] maybe, but the overhead is very little. It could check if jython is installed and then launch the shell [17:39:29] or we can just put jython.jar in [17:39:51] <^demon> We could shade jython into the plugin :p [17:48:21] $HOME is shared, so no need for dev cruft :( [17:52:38] <^demon> I wonder how much space is wasted in $HOME/.m2/ [17:53:51] heh [17:54:05] a shared something could be useful [17:55:43] do we have postgresql on -db? [17:57:36] ^demon: please see new comment on: https://gerrit.wikimedia.org/r/#/c/45777/1/LICENSE [17:57:49] <^demon> Sounds fine. [17:58:05] <^demon> saper: No, I never installed it. Feel free though. [17:59:55] need for running automated test [18:03:31] ^demon: poke [18:03:36] <^demon> Ouch! [18:03:44] :) [18:03:54] you may get an email shortly about wikidc.org [18:04:00] forward it to me, please :) [18:04:10] * aude renewing domains, etc. [18:06:44] hah, the phone number for wikidc.org is invalid now :o [18:15:41] ^demon: thanks :) [18:16:02] now the epp code [18:16:31] <^demon> *twiddles thumbs* [18:22:07] think they never sent it to us last time :( [18:24:09] I'm using php importDump.php, started around 4pm yesterday and still going. It's putting out stuff like this: [18:24:22] 11900 (0.18 pages/sec 0.17 revs/sec) [18:24:23] 12000 (0.18 pages/sec 0.17 revs/sec) [18:24:23] 12100 (0.18 pages/sec 0.17 revs/sec) [18:25:04] The data files is the simpleWiktionary dump around 6MB compressed. [18:25:50] Anybody know if there is a way to translate these messages to what percent remains to be done ? [18:26:11] <^demon> xyzram: http://simple.wiktionary.org/wiki/Special:Statistics shows 25,880 pages and 263,048 edits. [18:26:48] Thanks, looks like I'll have to kill it and try some other way. [18:27:45] <^demon> You may want to look into mwdumper. importDump isn't really suited for any dumps of significant size. [18:36:59] mwimport as well [18:37:18] search on meta for that (I think it's on meta?) [18:38:05] xyzram: if you are going to use the xml dumps more than a couple times you might want to get on the xmldatadumps mailing list [18:38:18] because annoucements about tools for import and other things happen there [18:38:45] Ok. [18:39:09] (and you can ask folks for help too) [18:39:22] Looking at the mwdumper page, it only talks about mysql; does it support sqlite (that's what I'm using) [18:39:56] ohh [18:39:58] uh [18:40:05] most conversion tools convert to mysql [18:40:09] what dump are you importing? [18:40:24] simplewiktionary-20130113-pages-meta-current.xml.bz2 [18:40:41] any notion of how may revs? [18:41:22] <^demon> Something in the realm of ~200k. [18:41:26] <^demon> We looked on special:stats. [18:41:26] Earlier in this thread, I saw ^demon: xyzram: http://simple.wiktionary.org/wiki/Special:Statistics shows 25,880 pages and 263,048 edits. [18:41:34] not awful, yeah sqlite could probably handle it [18:42:19] http://www.sqlite.org/cvstrac/wiki?p=ConverterTools well there are converter tools [18:42:23] no idea how good any of them are [18:42:59] <^demon> I really wish we could make importDump work a little better. [18:43:05] <^demon> There's gotta be some low-hanging fruit. [18:43:13] importDump.php seems to work without any conversion tools but is just so slow. [18:43:31] Is this due to parsing in php ? [18:43:36] there's no substitute for being able to jsut shovel in a pile of sql tables [18:43:42] part of it is that it's ph [18:43:43] p [18:43:44] <^demon> One thing that helps is disabling things like extensions for the duration of import. [18:43:54] <^demon> Things like parser functions make the import slower since it has more things to parse. [18:44:03] <^demon> (You can re-enable later, you'd just have to flush your parser cache) [18:44:28] how do I do that (sorry, I just came on board) [18:44:44] <^demon> In your LocalSettings.php [18:46:02] I don't have any extension in my setup except MWSearch. [18:46:19] <^demon> Hmm, that shouldn't affect importing much. [18:46:50] (I gotta get going but I'll be around later) [18:47:37] pgehres|away: mwalker: Your monday deploy overlaps the standard mediawiki deploy. It will probably need to be moved a couple hours (not now, because it will be overwritten) [18:48:22] tewwy: ah; foo; I must have forgotten to check that :p [18:48:27] move us whereever is needed [18:48:49] mwalker: It'll be pasted back to wikitech in about 15 minutes. So don't touch it until after that ;-) [18:48:59] kk [18:50:10] I'll move it to 1PM the same day. If it feels wrong, you can move it when it's pasted back :-) [19:10:47] hey Reedy, when I try to edit Schema:EventCapsule on meta I get "[2655b0e3] 2013-01-25 19:09:37: Fatal exception of type MWException". Where can I see the detailed exception log? I'm poking around fluorine but not sure where to look. [19:13:39] ori-l: /a/mw-log [19:13:56] fatal.log and exception.log [19:14:05] Should be able to search in the files for that hash [19:14:27] http://wikitech.wikimedia.org/view/Logs says it :p [19:14:28] or grep and include a few lines before and 10-15 after [19:14:49] aha, thanks! [19:15:10] to both of you [19:15:36] a thanks o' both your houses [19:17:03] Wow, a bit high-brow for us, isn't it? [19:20:39] nah [19:33:11] !log temp taking down mw1001 [19:46:59] notpeter: wrong channel :) [19:47:15] heh [19:47:16] eh [19:47:19] it's back up now [19:51:31] I think a syntax and a feature to mark exactly what article text is sourced from what reference is much needed, if it doesn't already exist. [19:52:02] moreover, mousemover on reference number should probably highlight this text [19:57:41] it's kind of ambiguous to put a bunch of text and a bunch of references without marking exactly what text maps to which reference [20:01:10] RandIter: I've no idea what you're talking about, dunno the others. [20:05:00] Nemo_bis: i will try to be clearer. My concern is with "text-source integrity" http://en.wikipedia.org/wiki/Wikipedia:Citing_sources#Text-source_integrity [20:06:17] Nemo_bis: Basically, an editor must be able to document exactly what text is covered by what reference, and a reader must be able to easily determine this [20:06:45] and there can be nestings and breaks among such mappings [20:09:00] For example, consider this example: The sun is pretty big, but the moon is not so big. The sun is also quite hot.[1][2][3] [20:09:26] looking above, the reader cannot tell exactly which claims [1] [2] [3] reference [20:12:21] Nemo_bis: Consider the simple example above rewritten as follows: The sun is pretty big, but the moon is not so big. The sun is also quite hot. [20:13:20] This allows a thorough reference-to-text mapping, and can allow for auto-formatting and display of things like "[1][2][3]" at the end of the sentence or paragraph [20:13:42] All this is off-topic here. [20:13:52] why? it's a development concern [20:13:58] Anyway, you Just want: The sun is pretty big,[1] but the moon is not so big.[2] The sun is also quite hot.[3] [20:14:09] no that's still too vague [20:14:13] it's not formal at all [20:14:17] ridiculous [20:14:37] You aren't thinking much at all. Consider a slightly more complicated and ambiguous example [20:14:37] this is not a development problem at all, I'm afraid, it could be a JS like used on some wikis like Wikisource [20:14:44] --> #wikipedia [20:15:04] No, I'd like for this to be a core thing [20:15:17] references are relevant everywhere [20:15:35] though I agree that js can be used [20:17:06] Consider the rewritten example 2: The sun is pretty big, but the moon is not so big. The sun is also quite hot. [20:17:26] You said I want "The sun is pretty big,[1] but the moon is not so big.[2] " but this is vague [20:18:21] if I do "The sun is pretty big, but the moon is not so big.[1][2] " above, i don't really know exactly which claims are covered by [1] and [2] [20:19:28] I am not editing articles with proper scientific references until this core issue is addressed [20:21:08] I did try #wikipedia and got no response [20:24:19] You may think the vagueness doesn't matter, but it does, for verifying specific scientific claims [20:26:12] It's unlikely to happen quicky [20:26:14] *quickly [20:28:28] I understand, as long as the issue is acknowledged and maybe some ideas are proposed [20:28:55] I do realize that attributing each piece of article text is burdensome [20:29:20] and it also balloons the text [20:29:30] but nevertheless it's warranted IMO [20:39:06] You'd be best both opening a bug in bugzilla for this issue, and also probably bringing it up on enwiki or similar [20:41:10] ok thanks