[13:29:47] <^demon> sumanah: Good morning. Hope you had a nice weekend. [13:30:18] hi ^demon! Good morning, and yes, I did! I saw a few very very good episodes of The Muppet Show. :-) And you? [13:31:31] <^demon> Good. Watched some Voyager reruns while working on a project for class. [13:32:31] Huh, these Muppet episodes did not include any Pigs In Space sketches or else there would have been a possible crossover between our weekends! :) [13:35:30] ^demon: Mind if I ask about the project? [13:36:06] <^demon> Yeah. It's just a senior development project. They're having us build an application backed by a database. [13:36:19] Oh right. [13:36:23] You're the one using LAMP. [13:36:28] <^demon> And since I'm in the business school, it has to do various accounting functions. [13:36:29] <^demon> Yep. [13:36:30] Everyone else is, what, doing Microsoft stuff? [13:36:38] <^demon> Yeah :) [13:37:07] You may have heard, ^demon, that the project that was the first guinea pig for what turned into Agile -- was an accounting project. [13:37:14] At Ford or something like that. [13:39:22] ^demon: so, there's a bunch of raw text in that talk page now. Can we turn it into a blog post today? [13:39:46] <^demon> Yep. [13:40:16] ^demon: Can you get the draft going or do you need me? [13:40:43] <^demon> If you could get me something started, I can fill in missing bits and expand on it. [13:50:55] <^demon> johnduhart: Glad we all agree :) And when we say "will revisit it in 3 months" we really meant it. [13:50:59] ^demon: ok. Will do that in a few minutes. At some point you should probably look at the stuff people have been asking on wikitech-l on that thread [13:51:12] <^demon> It's worth investigating seriously, and by then we'll have some solid usage data on gerrit. [13:53:28] ^demon: Great :) [14:24:21] *werdna waves [14:24:32] wassup ^demon [14:25:04] hi werdna [14:25:38] hi sumanah [14:25:51] werdna: would you say Nikola should apply for commit access, or not yet? [14:28:45] sure [14:28:50] I would recommend it [14:29:59] ok, will tell Nikola [14:54:05] Nikerabbit: for your 2 live hacks to Translate... [14:54:20] Reedy: crap [14:54:24] Do you want them both putting into 1.19wmf1, or just hte parser->setFunctionHook ? [14:54:44] let me see [14:55:19] Reedy: the version compare should be fine [14:55:32] oh, it's only for older versions [14:55:32] heh [14:55:34] it's just bugged with 1.18wmf1 > 1.18 [14:55:57] right [14:56:18] Reedy: and the translationdialog should be fine too, I had some problems that it couldn't find the magic word, so I just commented it out and forgot [14:56:34] heh, right, cheers :) [14:56:36] I had no idea how to update the l10n cache [16:55:38] ^demon: am working on http://etherpad.wikimedia.org/Git-Feb13 [17:38:58] au: ping [17:40:53] gwicke: pong [17:41:08] hi Audrey, here is Gabriel [17:41:16] hi, nice to virtually meet you :) [17:41:26] same here ;) [17:41:36] did you have a chance to look at the todo list yet? [17:41:47] which TZ are you in? I'm in CET for the next 3 weeks -- just arrived to Paris [17:41:58] CET here too [17:42:01] Germany [17:42:06] great! [17:42:10] that simplifies matters [17:42:11] the Parsoid TODOs? yes [17:42:33] did you see anything that sounded interesting to you? [17:42:33] hi au, are you going to be working on Parsoid? I'm Sumana, nice to meet you [17:42:47] hi Sumana, nice to meet you too :) [17:42:57] the parser function stuff might have a few nice bits in it [17:43:10] a safe expression evaluator for example [17:43:53] hmm, where in https://www.mediawiki.org/wiki/Parsoid#Todo is that listed? (just making sure we're on the same page) [17:44:10] in the token stream transform section [17:44:14] the first item [17:44:44] most of the work is really in token massaging [17:45:44] if you have any questions about the general architecture or anything- just shoot! [17:45:58] ok. is that all in the trunk/extensions/VisualEditor directory [17:46:08] or are additional codebases I should check out too? [17:46:08] yes, in modules/parser [17:46:30] no, trunk/extensions/VisualEditor/{modules,test}/parser [17:46:41] ok [17:47:18] the phase3 checkout in 'getting started' is only needed to get the parser tests file [17:48:18] ok. still waiting for the parsers/graveyard to checkout... hotel wifi not exactly fast here [17:48:49] if you'd like to save time, then don't check out phase3 [17:49:41] k. I'll wait it out since I'll start working for real around Friday [17:49:45] instead, just get trunk/phase3/tests/parser/parserTests.txt [17:50:00] ok [17:50:32] would you briefly explain how the current evaluation in unsafe? [17:50:40] not yet seeing it on Token_stream_transformations page [17:51:03] there are some parser functions that evaluate arithmetic expressions [17:51:10] in the stub I simply used eval for those [17:51:57] ok. which file should I look at for the stub? [17:52:18] modules/parser/ext.core.ParserFunctions.js [17:52:41] k, danke sehr [17:52:56] there might be expression interpreters we could adapt, I have not yet checked that [17:53:38] ok. is the code exclusively running in browser, or also in node context? [17:53:41] (except in tests) [17:53:48] right now, it is only running in node [17:53:59] we don't care about the browser yet [17:54:18] ah ok [17:54:23] the idea is to build an HTML DOM, and ship that to the editor [17:54:48] we add some annotations to aid round-tripping, but those are very much work in progress [17:55:05] so maybe something as simple as a vm.runInThisContext with timeout guard could work? [17:55:27] would that provide a sandboxed namespace? [17:55:50] yes, runInThisContext has no local scope access [17:55:51] unlike eval [17:56:08] we can swap out global objects too with runInNewContext [17:56:12] but the idea is the same [17:56:23] if that is safe, then that would be a very elegant solution [17:56:29] # http://nodejs.org/docs/latest/api/vm.html#vm.runInNewContext [17:56:47] the only thing it doesn't guard is while(;;){} [17:56:53] which is why a timeout would be needed [17:57:03] ah- so it is node-specific? [17:57:05] but if that doesn't arise in practice then even the guard is optional [17:57:14] it is, hence the running-context question [17:57:26] ok [17:57:45] we have no concrete plans to run this in the browser, so that should be fine [17:58:05] ok. should the need arise, we can always reimplement it in client side with something like Caja or other JS sandboxers [17:58:33] it is just arithmetic, so I guess a small parser and interpreter would work too [17:58:41] ok [17:59:25] there many other areas you could work on though [17:59:26] it's been a while since I used svk/svn... should I send patches somewhere? [18:00:01] yeah, you could send patches [18:00:17] but plain svn should not be too hard [18:00:40] that is more a question about repository policy [18:00:58] yeah, I'll try to get you an account [18:01:10] i.e. do I get a commit bit? if yes, do you adopt a more "trunk must always be reviewed and green" stance and changes go to a branch first, or should I commit directly to trunk [18:01:14] will need a public key [18:01:26] ssh-dss AAAAB3NzaC1kc3MAAACBAOOY9pVJ3VvHQsrLtXrGxkEwFuuFfg9w17OemA4JH8rWp8nRPgH7M/PncuLJU2b7jsOfQvpuZjTp7ftClfUnjxjJZlZIJo9qtLLoUL1bLYTa3t7MlKUl+y1Axtl1ey9Q8UiRF6RxMB2R2/MuwqMZzdPZXfImbARByPE31UdmxbItAAAAFQCpgi7k03H7sLWfAp+JWeIXiADr7QAAAIEAtREZlZWxpdRMRf7D7VFElZWdsHc+VgsaYxbydL6l/qtHbiP1sgzDqTkWCKc/TH6optgd6GC24zLLSHBjcH+u+r4kgKaWt/ulkvxlZEIPF3Ge+KDEZl2hlU7FMRFbf2ls2ZB0ZtYPqzlMxNQ8K9H3hY8kINl1YSI/FFdst4RpmkkAAACAHlq9bz+xRx3hjRx4uzYOblvo0nEN [18:01:31] right now we commit directly to trunk [18:01:33] (sorry for the flood) [18:01:36] np [18:01:57] there are plans to complicate things a bit further, with gated trunk, git etc [18:02:06] not there yet though [18:02:55] ok. is there a commit-mailing-list for code review I should subscribe to? [18:03:27] http://www.mediawiki.org/wiki/Commit_access_requests#Requesting_commit_access [18:03:33] sumanah: thanks! [18:03:49] aha [18:03:51] sumanah++ thx [18:04:43] *mutante notices a list called "MediaWiki-CVS -- MediaWiki CVS commits" .. CVS :p [18:04:43] au: there is no mailing list, but there is Special:Code for code review [18:04:58] well- there is a list, but you don't need to follow it [18:05:18] <^demon> mutante: We've just never renamed it :p [18:05:36] ^demon: i guess i know why, you cant in mailman :p [18:05:55] au, https://www.mediawiki.org/wiki/How_to_become_a_MediaWiki_hacker is useful as well for HOWTO stuff [18:06:04] ^demon: so that will also receive git commits_ [18:06:10] <^demon> :p [18:06:22] <^demon> We can just claim nostalgia. [18:06:56] au: could you send the request directly as described in http://www.mediawiki.org/wiki/Commit_access_requests#Requesting_commit_access ? [18:07:48] and add me as cc [18:08:27] you could use patches in meantime though [18:09:25] au: https://www.mediawiki.org/wiki/How_to_become_a_MediaWiki_hacker#Submit_your_changes for patch procedure [18:09:33] (sorry for redundancies) [18:10:52] mail sent to commit-access-requests [18:11:10] au: thanks! [18:11:16] Fwd'd to you [18:11:26] thanks, au. gwicke are you vouching for au? for extensions & branches, right now, or core as well? [18:11:42] sumanah: extensions & branches [18:11:48] core is not needed [18:12:20] and I am vouching for Audrey [18:12:23] ok, thanks gwicke, good to know [18:12:32] gwicke: I should run through the commit queue today and again tomorrow [18:13:04] sumanah: ok, thanks! [18:13:15] thank YOU for recruiting new devs :D [18:13:25] thank you gwicke and sumanah both -- quite happy to join the fun :) [18:13:32] :) [18:14:14] au: http://www.mediawiki.org/wiki/Help:Extension:ParserFunctions [18:15:01] just to set expectations -- as I mentioned to Erik, I can only work for avg. 4hr per week for the next few months, preferably in bite-sized tasks first [18:15:23] au: oh, remind me, are you a volunteer, a contractor, or other? [18:15:27] a contractor [18:15:31] ohoooooo [18:15:36] has an email about you gone out yet? [18:15:37] au: I think parser functions are quite good for that [18:15:53] because you can tackle them individually [18:16:00] yeah, the "Stub section: Pick any of these and actually implement them" line is very encouraging [18:16:14] ;) [18:16:29] (quite sincere -- no sarcasm intended) [18:17:37] sumanah: re email gone out, Erik Cc'd gwicke disokawa and asharma, not sure if it's... [18:17:41] ...oops, missed :) [18:19:25] au: I really hope to find some interesting bits for you, so that I can draw you into parser stuff [18:20:16] right now I am the lone hacker on the parser, and there is plenty of work to be done.. [18:20:47] k. just three months ago I co-designed and implemented a two-way wiki/html parser/converted: https://github.com/socialtext/open/tree/master/nlw/share/node_modules [18:21:09] (admittedly more of coffeescript inclination) [18:21:28] nice! [18:21:51] (it's open source too (MIT license)) [18:22:02] ohai au, I just realized who you are :) [18:22:04] would be happy to learn from the Parsoid codebase [18:22:21] so you went html -> wiki for editing, and then back? [18:22:23] oh wow. we meet agian :) [18:22:49] gwicke: that's right, full lossless roundtripping on top of CKEditor [18:22:56] au it's a small wiki world I guess :-) really good to see you working for WMF as well. [18:23:39] au: our case is a bit harder, as we go from wiki to html, and then back [18:24:02] so we'll have to preserve whitespace variations to avoid dirty diffs and so on [18:24:21] yeah, that happens when wikitext is canonical storage, not html [18:24:32] yep [18:24:36] the old Socialtext parser (Wikiwyg) was like that [18:25:19] preserving everything is quite cumbersome, so we have some ideas to only serialize out actually modified parts of the dom [18:25:38] and splice in the original source for unmodified dom parts [18:26:28] we add some round-trip data as JSON in a data-mw-rt attribute [18:27:14] don't know yet if we can push that far enough to avoid dirty diffs even if we serialize everything out again [18:27:28] au: the serializer also needs to be written btw [18:28:40] as in ve/dm/serializers/ve.dm.HtmlSerializer.js ? [18:28:53] that is a simple version we can take and adapt [18:29:05] no support for whitespace round-tripping yet [18:29:32] it would also be nice to have per-tag handlers [18:29:37] ok. where would the modified-part-detection take place? [18:29:46] so that we can use the same handlers to serialize tokens, or the dom [18:30:01] the idea is to track the modifications in the editor [18:30:14] and ship it back as attributes to the node server [18:30:23] ah, flagging as part of editing, instead of taking a hash digest of the DOM [18:30:26] got it [18:31:00] internally the editor builds transactions, so tracking should not be that hard [18:31:22] so, a naive question -- is the eventual goal to make this real-time collaborative like etherpadlite? [18:31:37] or would it be batch-transactional like the current editor [18:31:40] yes, that is a longer-term idea [18:31:48] ok, so txns would help [18:32:04] the transaction model is already geared towards operational transformations [18:32:29] it converts the dom into a linear model [18:32:39] excellent, since bolting OT on is... usually a nontrivial exercise [18:32:53] which would be mirrored on the server for multi-user OT [18:33:22] ok. that'd be an area I'm interested in as well, having worked with EtherPad(+Lite) and EtherCalc codebases [18:33:47] right now that all lives in the client code [18:33:57] but will move towards the server as we get further [18:34:10] got it [18:34:46] templates and parser functions were and still are quite time-consuming [18:35:22] we also have some plans to use microdata to represent template expansions in the dom [18:35:37] http://www.mediawiki.org/wiki/Future/HTML5_DOM_with_microdata [18:36:17] nice use of microdata [18:36:26] that is quite high on the todo list, as the editor will need it to identify content coming from templates [18:36:30] node ./parserTests.js: 179 total passed tests, 498 total failures [18:36:34] is that... normal? [18:36:45] node v0.6.7 OSX [18:37:03] latest versions of the npm modules listed in README plus "request" [18:37:04] I get 238 currently, but there might be a bug in the html5 parser [18:37:21] ..which would be a good thing to track down as well [18:37:36] right now I have patched one or two lines in there [18:37:41] k [18:38:02] *au tries to work out which ones are test-driven todo tests and which ones are regression [18:38:45] looks like normalized-rendered e.g. "__TOC__" is currently unexpanded [18:39:27] while another class of failures are in Users/audreyt/i/trunk/extensions/VisualEditor/tests/parser/parserTests.js with the following error: TypeError: Object #