[03:28:35] hi [03:29:01] does anyone here have any insights into working as a software developer for wikimedia ? [12:34:35] OrenBochman: hey, how's the clone on labs [12:35:26] last I cheked it is not fully imported [12:35:30] in case you wanted me to install some extensions let me know [12:35:43] it's still in 40% [12:36:53] I think it will be done in 30 hours [12:37:08] but you can already use the site [12:37:18] it's just little bit slow [12:38:29] yep it is slow [12:38:39] it probably needs tidy as well [12:38:54] since there are broken tags [12:39:26] I can access tomcat too but jenkins does not seem to work [12:39:51] I put the war it in the web apps dir but nothing happened [12:40:21] huh [12:40:31] what [12:40:45] also the search shows only a few docs [12:40:53] but that may improve [12:41:10] aha, you mean on search [12:41:20] yes, that's because it's not finished yet [12:41:25] I agree [12:41:34] we need to let it finish [12:41:43] is there anyone else working with you [12:41:51] nope [12:42:03] I thought jeremyb \ [12:42:05] but I tried to install jenkins CI [12:42:07] he doesn't? [12:42:15] jeremyb: help out [12:42:31] helps out I mean [12:42:42] sure but only in ops [12:42:54] right, want me to help with jenkins set up? [12:43:01] he does not know much about search [12:43:03] sure [12:43:07] btw do you need a full db or latest revision only? [12:43:19] in can you wanted full you need to talk to Ryan and ask for more space [12:43:22] in case [12:43:49] ok, let me ssh [12:43:50] this instance should be just the latest revision [12:44:08] yes [12:44:11] also I you can install SOLR on tomcat [12:44:41] later I will need a much bigger instance like the real search setup [12:45:38] probaly with 4 machines in a cluseter [12:45:54] probably with 4 machines in a cluster [12:46:59] ok [12:47:04] how big db? [12:47:19] because Ryan would probably set up a separate machine for database rather than vm [12:47:28] it would be faster [12:47:44] where did you install jenkins [12:49:25] solr is installed and jenkins seems to be working [12:50:01] I think [12:50:12] I didn't set up a tunnel so I can't conect to its port [12:51:14] for full database you need at least 200 gb if you want to keep backups [12:51:56] but I think I could extract it to 80gb storage too, but probably with no recovery [12:52:11] it would not be easy though [12:53:43] btw load of test is 0.5 so I don't think it should be so slow now [14:14:07] Reedy: I am wondering how to fix that version compare stuff, is it even needed when extensions doesn't have a special page [14:14:32] 1 of 2 ways [14:14:39] You just set them unconditionally to "other2 [14:14:42] *"other" [14:14:51] Or you add the message to trunk, and then >= 1.19 you can use it [14:14:51] hm [14:15:09] it seems a reasonably enough sane group to exist [14:16:11] but it wouldn't be useable in 1.18 [14:16:22] I can't insert message to trunk anyway [14:16:26] because I don't have core access [14:16:27] indeed, not without backport [14:16:42] but I could create a ticket for that so someone would [14:38:25] @add #mediawiki-i18n [14:46:27] petan: thanks for SOLR , if you succedded with jenkins what is it's url? [14:46:54] no idea I need to properly set up proxy here [14:58:51] OrenBochman: hello :) [14:59:08] OrenBochman: I am reading a mail you sent to Sumana and mentioning Jenkins / Continuous integration [14:59:27] OrenBochman: I am the one in charge of Jenkins, so if you need anything there let me know :D [15:03:31] hashar: I think he talked about local installation of jenkins he has on labs [15:03:50] most probably [15:04:10] I don't have a big knowledge of that I hope you help him better than me [15:04:11] :D [15:04:40] or we can learn together! [15:10:07] or that :) [15:10:14] I have no idea what is actually jenkins for [15:10:26] !wiki [15:10:26] --elephant-- I don't know anything about "wiki". You might try: !1.17wmf1 !b !class !e !hook !mwbot !pad !rev [15:10:53] !wiki is http://en.wikipedia.org/wiki/`e1 [15:10:53] --elephant-- You don't have permission to do that. [15:10:58] :( [15:15:09] hi hashar [15:15:22] :) [15:15:32] I want to put search into jenkins [15:15:53] we can add a job to the existing one on http://integration.mediawiki.org/ci/ [15:16:03] I'd since it in java and has an ant build script it should be easy [15:16:27] hasar that would be great for the long term [15:16:30] that machine has both openjdk and the sunjdk installed [15:16:44] ant is 1.8 (1.7 can not be used) [15:17:51] I'd like to simply my development cycles so just bulding it is not enough - it would need to be able to access live wikis and dumps testing and benchmarking [15:18:45] OrenBochman: let me know about your plans on bigger wiki [15:18:55] we will need to talk to Ryan when he's here [15:19:29] petan I am thinking to replicate the search machines in labs [15:19:42] you wanted a full db of simple wiki? [15:20:00] no latest version is fine [15:20:05] ah ok... [15:20:12] I thought you talked about some cluster or what [15:20:18] yes [15:20:28] I'm trying to explain [15:21:49] what I will soon need is setup similar to production with machines indexing the real wikipedias with the index split on servral machines [15:22:01] ok [15:22:28] that would be possible but I think we should set up an sql server first Ryan has some plans but he didn't make a proposal yet [15:22:30] int that setup I don't need local db [15:22:49] I'll be glad to discuss with evryone [15:23:00] ok [15:23:05] I think it shouldn't be problem [15:23:26] biggest problem is storage apart of that it's ok [15:23:41] or that's what he said [15:23:45] now regarding a full db that indexer is a feature request and needs to be specified and developed (dev is easy) [15:24:07] but it is about a month away [15:24:32] it shouldn't be problem to create a full db import too, but let me know because it can take even few days to set up everything [15:24:44] ok [15:24:59] petan I'll give you a head start with that [15:25:09] import of latest rev is running since yesterday and still running but it's partially because I started it wrong [15:25:23] I should have wait for apergos [15:27:29] I am wondering if it would be possible to make a instance with lots of dbs (one per each language/script) with a about 1000 pages + the template namespace??? [15:28:00] yes [15:28:16] petan: when I imported wiktionary on a local machine and it took about 3 weeks [15:28:42] it shouldn't be problem but I want to discuss this with others before I mess it up again :) [15:28:52] no problem [15:29:27] sometimes I start working on stuff and after few hours someone tell me that if I started it in a different way it could be way more easy [15:29:32] hashar: could you help with the local jenkins instalation - on search ? [15:30:00] petan: it is the same with developing too [15:30:13] I am talking about developing :) [15:30:36] OrenBochman: maybe ;-) [15:30:54] hashar can I accss your jenkins ? [15:31:05] OrenBochman: yes [15:31:13] or I see you there in user list [15:31:13] :D [15:31:24] gues I should be [15:31:28] !jenkins [15:31:28] --elephant-- I don't know anything about "jenkins". [15:32:10] oh wait, no I don't see you [15:32:16] weird [15:32:33] probably someone need to insert you there, I guess hashar is right person for that :) [15:32:35] wo I;m in there already [15:32:41] ah [15:33:07] OrenBochman: https://integration.mediawiki.org/ci/ [15:33:16] maybe I see only people in same project then [15:33:16] yep I just logged in [15:33:19] if you have a lab account you should be able to identify yourself there [15:33:30] but you probably can't do that much there :D [15:34:28] hashar you need to install mave there [15:34:39] maven? [15:34:41] hashar you need to install maven there [15:34:43] yes [15:35:23] the current set up of search is in ant but I plan to move to maven [15:35:48] oh noooo [15:35:50] :-D [15:36:06] I have antlr grammers and other components that are more easy to do with maven [15:36:49] I can get maven installed, would need a list of libs to install too [15:37:03] ok [15:37:44] based on libmaven* package list for Ubuntu 10.04 (lucid) [15:38:19] if you have an email I'll drop you a note - I'll probably set things up on my local dev machine > then the labs > then your CI [15:38:39] I am wondering about one thing [15:39:11] mail: amusso at wikimedia dot org [15:51:48] OrenBochman: I have send basic changes to install Apache Maven in production [15:51:49] https://gerrit.wikimedia.org/r/#change,1768 [15:51:50] https://gerrit.wikimedia.org/r/#change,1769 [15:52:19] still need to add maven plugins though [16:22:34] *jeremyb looks up [16:25:59] OrenBochman: petan: i know a little (not a lot about how the search infra works. i know more about lucene itself (from another place where I did some hacking on an app that used lucene with a local index) but not so much about theoretical search stuff (e.g. lucene internals) [16:26:15] *jeremyb is mildly confused about why the convo is in this channel now? [19:01:50] hi all - features team - let's get started [19:02:02] Meeting Agenda: [19:02:02] 1. Brief (2-3 minutes) status update by each team for project teams listed below: [19:02:02] a. Editing tools: Visual Editor [Neil], Parser [Gabriel] [19:02:02] b. Editor Retention: Feedback Dashboard [Brandon, Benny, RobM], AFT v5 [Dario] [19:02:02] c. I18n: Narayam, WebFonts, Translate [Siebrand, Amir, Santhosh, Niklas, Gerard] [19:02:02] d. Multimedia: Upload Wizard, TMH [Ian] [19:02:02] e. Resource Loader / Gadget Manager [Timo] [19:02:03] f. Global Education [Jeroen] [19:02:03] g. Features Support (Deployments, Testing) [Roan, Timo] [19:02:04] h. Other projects [19:02:16] thanks for updating the etherpad [19:02:40] visual editor - neil - updates [19:03:00] you want me to reiterate what I wrote in the etherpad? [19:03:14] sure - highlights [19:04:19] * Refactored parser execution framework (was a bit intertwined with testing) [19:04:19] * Wrote parse.js "standalone" program. Wikitext stdin, Wikidom stdout. [19:04:19] * Wrote patch to grey out undo/redo when nothing to undo/redo (based on community patch) [19:04:38] ok, I'm trying to get to a state where the VisualEditor can request a wikidom parse of some article on the wiki [19:05:04] this will involve an API in MediaWiki, and a standalone parse service (you'll have to run node.js locally) [19:05:11] k [19:05:11] the standalone parse thing is done [19:05:33] conferring with Gabriel about redesigning the parse to be evented, I didn't manage to do it but he apparently just landed it this morning. [19:05:46] and, then I did a simple patch for VisualEditor gui related to undo/redo. [19:06:23] great! thanks [19:06:33] *werdna-holiday sneaks in late. [19:06:34] gabriel? [19:06:37] andrew! [19:06:39] hi werdna! [19:06:47] didn't work much last week [19:06:56] but got the refactoring finally checked in today [19:06:59] understandably :-) [19:07:03] andrew, were you the one who wrote the etherpad patch that fixed the padding and such? [19:07:08] so now we have async token transformations [19:07:18] gwicke: yay [19:07:19] Not I, said the little red hen [19:07:20] hopefully a good base for template expansion [19:07:33] will work with Neil on that [19:07:49] gwicke: thanks; looking forward to seeing you in sf :-) [19:07:52] there are still a few loose ends design-wise [19:08:06] but those should be tied up when it is used for templates etc [19:08:15] that's it from me for now [19:08:20] ok - thanks! [19:08:37] b. Editor Retention: Feedback Dashboard [Brandon, Benny, RobM], AFT v5 [Dario] [19:08:42] jorm: any updates [19:08:56] feedback on MAH? [19:09:02] Yeah, how IS the feedback dashboard? :) [19:09:15] well, we had a deployment. [19:09:22] it's pretty damned awesome, if i do say so myself. [19:09:23] werdna: we deployed last week on wednesday [19:09:29] Oooo [19:09:48] this latest deployment included a feature "Mark as Helpful", which allows people to say whther or not the responses they got were useful. [19:09:50] jorm: any interesting community feedback you've seen [19:10:01] it's been dead. everyone's been partying. [19:10:01] That's awesome. [19:10:10] so the volume has been down. [19:10:10] but [19:10:17] the feature is well receieved. [19:10:29] that could just mean that the trolls haven't seen it yet. [19:10:31] evening [19:10:43] right - the trolls haven't arrived yet [19:10:55] we have an office hours for features tomorrow [19:10:56] Hi Nikerabbit [19:11:04] on irc [19:11:17] so all of you are welcome to join [19:11:22] What time? [19:11:38] 3p-4p PST [19:11:53] Oh ,right [19:11:57] The midnight office hours :) [19:12:01] bsitu: anything else to add [19:12:09] Added clicktrackng to feedback response email [19:12:12] tomorrow? [19:12:15] *werdna-holiday is lucky enough to be in PST at the moment, but I'll probably be away from wifi then [19:12:18] Roan:the midnight office hours for you [19:12:24] jorm: yes [19:12:33] k. [19:12:43] i have an interview just before then but that'll be okay. [19:12:44] bsitu: did legal clear this? [19:12:50] we will be doing a features office hours once every month [19:13:06] howief: dario did talk with legal [19:13:08] howief: I got greenlight from dario so I assumed it gets cleared [19:13:15] awesome! [19:13:20] howief: it is cleared [19:13:31] What're the next steps for FBD? [19:13:32] last week, I also added page display rule to 'mark as helpful' [19:13:38] Do you want help? [19:13:47] i'm curious to hear how you implemented this. I'll follow up with you guys separately. [19:14:14] howief: yup - we should be meeting on this today at 430-5p PST [19:14:24] robm: anything else to add? [19:14:56] Not much, just that we came across some things to add in this week to MAH in regards to page rules and whatnot [19:15:10] MAH? [19:15:15] alolita, legal cleared the use of clicktracking as we planned to implement it now, for more granular data collection we will need to have a full review with them [19:15:22] werdna-holiday: mah = mark as helpful [19:15:30] Aha, thx [19:15:32] werdna-holiday: MarkAsHelpful, as opposed to Mark A. Hershberger [19:15:47] Yeah, that's what confused me. I thought we were adding features to mark [19:16:01] Like, extra beards or something [19:16:18] werdna: :-) [19:16:47] hexmode does not need more beard. [19:16:48] howief, dartar: anything else on aftv5 [19:17:08] I have just committed a fix for the clicktracking data format [19:17:08] someone is giving away beard? [19:17:17] so aftv5 [19:17:21] And I'll get Dario some dumps of that today [19:17:29] RoanKattouw: thanks! [19:17:33] the big thing is to get the links to the feedback widget more prominently displayed [19:17:33] yeah, as noted in the etherpad: we expanded the sample of cherry-picked articles and this week will work on doubling the random sample [19:17:37] RoanKattouw: awesome [19:17:46] RoanKattouw: Mark A Hershberger is marked as helpful :P [19:17:53] haha [19:18:00] *hexmode goes radio silent again [19:18:03] I got a few compliments on AFT from random strangers while I was away, by the way. Nice work! [19:18:18] werdna-holiday: that's cool [19:18:20] what did they say? [19:18:24] howief: nice work indeed [19:18:25] on aft? [19:18:30] Just that they thought it was a fantastic idea [19:18:37] cool [19:18:43] RoanKattouw: I'd like to test the accuracy of the AFT5 random bucketing on a sample of users as we discussed last week if you can help with this [19:18:45] werdna: good to hear that [19:18:56] DarTar: Right, that too [19:19:12] DarTar: Tomorrow, if that's OK with you [19:19:17] sounds good [19:19:26] DarTar: anything else to cover? [19:19:37] c. I18n: Narayam, WebFonts, Translate [Siebrand, Amir, Santhosh, Niklas, Gerard] [19:19:51] I worked further on the toolserver dashboard/feedback stream for AFT [19:20:05] as noted in the etherpad ;) [19:20:14] DarTar: great thanks! aharoni, nikerabbit: updates? [19:20:27] Hallo [19:20:34] hi amir [19:20:57] Translate extension: [19:21:02] Translate workflow states are now displayed in language and message groups statistics pages. [19:21:27] *werdna-holiday has to disappear. Back hopefully in 15 mins or so. [19:21:39] Next week i plan to complete the current "workflow states", er, roadmap, by making the state names discoverable and localizable. [19:21:40] bye werdna [19:22:04] aharoni: great, thanks [19:22:12] The extension's documentation is being improved. [19:22:28] "Review recent translations" feature was added. [19:22:33] (by Niklas) [19:22:46] nikerabbit: good stuff! [19:22:55] deployed yet? [19:23:38] deployed only on translatewiki.net, and not completely reviewed yet (i hope to finish the review today, it's quite big) [19:24:00] aharoni: are you covering for santhosh also [19:24:11] i'm trying :) [19:24:17] :-) [19:24:50] so Santhosh has been working mostly on integrating and testing gender support in JavaScript, but nikerabbit can update about this better. [19:25:15] ok, nikerabbit: ? [19:25:54] Santhosh has also been trying to make mediawiki.feedback.js useful for sending feedback about WebFonts. [19:26:23] neilk: did you have a chance to look at the ongoing gender support discussion on bugzilla [19:26:56] hmm [19:26:59] nikerabbit: ? [19:27:14] alolita: Krinkle, RoanKattouw and I are going deep into the weeds there :) [19:27:27] well our JS plural and gender work is a bit open now that RoanKattouw said he wants to have the main work done serverside [19:27:45] neilk: that's normal :D i'll catchup on that thread [19:27:56] I talked to Tim about that earlier today [19:28:00] alolita, Nikerabbit: that's what I have always wanted too, but our parser is difficult to instrument. He demonstrated a way to do it that *might* work. [19:28:11] RoanKattouw: ok, what did Tim say [19:28:27] RoanKattouw: is it likely that we can make happen together this week? [19:28:31] or should we take that offline [19:28:33] Well, probably not [19:28:51] We should probably just go forward with the existing client-side parser, it works fine [19:28:54] the question to me is, does the i18n team need to care how it's implemented [19:28:55] yeah [19:28:59] we can refactor it later [19:29:09] that sounds like a plan [19:29:10] RoanKattouw: i agree with that strategy [19:29:14] the client side cost is minimal, although the whole thing is inelegant [19:29:34] Doing things server-side would be Better (TM) (although Brion seems to disagree on wikitech-l just now), but not required [19:29:44] neilk: understood [19:29:47] oh wait, there's a thread?! [19:30:03] Not very substantial [19:30:09] [Wikitech-l] Plural, Gender parsing support for interface messages at client side [19:30:11] It was next in my queue I see it now. [19:30:15] I'm writing a reply to Brion now [19:30:20] reading it now [19:30:26] I'll also include my plan for server-side processing [19:30:36] Which is entirely different from what I showed you last night [19:30:59] alolita: we might have a larger feature deployment next week, depending on how stable we get things [19:31:16] nikerabbit: that makes sense [19:31:29] and not this week [19:31:49] yeah this week is definitely too early for Translate at least [19:31:57] *brion waves [19:32:08] server-side processing is easier in some ways yes :D [19:32:12] nikerabbit: no worries; lets do it next week during our standard window [19:32:16] brion: hi! [19:32:51] nikerabbit: anything else to add [19:33:21] else we move on to Multimedia: Upload Wizard, TMH [Ian] [19:33:26] alolita: I think we covered the most important things [19:33:34] raindrift is sick/WFH today [19:33:39] raindrift = Ian [19:33:45] nikerabbit: that works - thanks! [19:34:09] neilk: ok i guess he is not on irc - he is wfh [19:34:12] The last thing I heard was that we finally had a working chunked upload protocol with tests and stuff. [19:34:26] neilk: thanks - appreciate the update [19:34:32] We requested a review from Tim [19:34:43] great [19:35:09] there are a number of fixes piling up with UW over December, so I'm going to deploy right after Pacific time lunch, instead of 4:00pm friday, which is when Erik wanted me to deploy :) [19:35:19] (4:00pm last friday I mean) [19:35:49] neilk: no worries .... let's push these fixes this week ... today [19:35:56] we can deploy it without turning on Chunked uploading. THere's a config flag. [19:36:06] yup [19:36:07] as for TMH, I'm not sure what the status is. [19:36:24] I think that's fully reviewed by now. [19:36:38] yes - that's what ian mentioned [19:36:58] so let's move on to the resource loader - RoanKattouw, Krinkle [19:37:08] progress: ? :-) [19:37:10] neilk_: You guys got TMH reviewed before the end of the world as predicted by the Mayans? I'm impressed [19:37:14] So, ResourceLoader [19:37:27] I was poking at the server-side message loading thing, which I'm writing a wikitech-l post about right nwo [19:37:42] RoanKattouw: which means there is hope for the world? [19:37:47] My plan touches ResourceLoader code [19:38:08] RoanKattouw: what are we talking about here, server side message parsing to wikidom? [19:38:13] RoanKattouw: or something else [19:38:15] Sort of, yeah [19:38:17] I'll write a post about it [19:38:32] yeah, I noticed you used a wikidomish format which perhaps makes sense [19:38:32] Aside from that, I should pick up where I left off with my RL2 work [19:38:36] how does it affect RL code [19:38:40] Which is writing unit tests, mostly [19:38:50] RL does message delivery to the client [19:39:20] My plan would add a processing phase on the server, then the processed message would be delivered to JS, where expansion would happen [19:39:36] Changing the nature of what's being delivered means I have to change the delivery code [19:39:44] Not very substantially, but not trivially either [19:39:57] RoanKattouw, sounds good. (just sent a reply on-list -- we should be able to work with that for the mobile app) [19:39:58] right and you would be storing JSON not strings [19:40:12] caching that is [19:40:15] Right [19:40:32] siebrand: Online already? [19:40:47] So yeah, there's RL2 work to be done [19:41:01] I need to get back into that game and see what the State of the Project is there [19:41:17] RoanKattouw: yup and update / add more detail to your plan on mw.org [19:41:43] And I think Krinkle (not here?) is working on a RLv1 issue locally [19:42:00] RoanKattouw: can't see him online [19:42:03] But he's had close to no time off from his internship for the holidays, so he's busy as ever [19:42:12] wow [19:42:27] this internship is making him work hard [19:42:51] If he'd been in classes, he would've had last week and this week off [19:42:55] RoanKattouw: are you done moving [19:42:58] I was hoping to capitalize on that this week [19:43:00] Yes [19:43:03] cool [19:43:17] I'm working normally starting today [19:43:21] for as long as that lasts [19:43:28]