[00:00:00] so I'm thinking, we could do all those things, plus change the colour scheme slightly [00:00:31] If it had a new parser, people would notice [00:00:47] OMG YOU'VE BROKEN STUFF!!!!oneoneone1111!111! [00:01:26] oh, I think we could bring in a new parser with very little disruption [00:01:36] I've been talking to robla a bit about this [00:01:43] Yeah, version it [00:01:47] *robla looks up [00:02:14] I think you first talked to me about versioning the parser at the first Berlin conference in '09 [00:02:26] I have a new idea for UI though [00:03:16] imagine if somewhere on the edit page, it said "Parser version: 1.12 [[change]]" [00:03:26] and if you just edit the page, it leaves it at the same version [00:03:45] if you click "change", it takes you to a page which lets you change the parser version, persistently for that article [00:04:10] it would let you view the two parser outputs side by side in a kind of preview [00:04:40] you could change what needed changing for migration purposes, then when everything looks good, click save [00:05:04] Only problem though is tranclusion [00:05:23] Yeah I think you'd also want a diff between preprocessor outputs, if the pp changed [00:05:26] Like ParserDiffTest [00:05:44] features like rollback and editing of old revisions could revert the parser version [00:06:06] that way you could let anyone change the parser version, and it would be trivially revertable if it broke things [00:07:00] I haven't quite gotten up to templates yet, this is where the idea is at [00:07:09] so it'd be in the text table rather than revision table ? (if rollback should affect it) - would make sense. [00:07:26] probably vice versa [00:07:34] oh wait [00:07:42] so that they're asociated. [00:07:43] it can work either way [00:08:10] text table is smaller and revision also contains null revisions. [00:08:16] but sure either works. [00:09:40] would it be a version, or perhaps something a little more akin to a mime-type? [00:10:03] *robla doesn't think it should be strictly linear [00:10:28] sure, we have to have wikicreole support don't we? ;) [00:10:39] maybe not for wikipedia... [00:10:51] but certainly for third-party installations, we could have wikicreole, bbcode, etc. [00:10:59] exactly [00:11:38] ....and if someone manages to come up with something truly wonderful, we could consider it for Wikipedia, but that decision is probably years from now [00:12:02] HTML? [00:12:19] HTML5 so we can put the shield in the dropdown [00:12:44] right, and HTML5 is still cool [00:12:54] HTML is old and festering [00:13:44] "Now with HTML 3.2 tables!" [00:14:09] "Best viewed with Netscape" [00:15:49] So, right now 1.16 is default and almost(?) all wikis are in 1.17.dblist, perhaps swap this so it's clear which aren't running 1.17 (if any) [00:16:03] We'll do that next week or so [00:16:18] I also noticed monobook is still the default fallback for bogus ?useskin= entries, wheareas in 1.17/trunk/plain install it's vector. [00:16:26] Hmm [00:19:55] all are on 1.17 [00:20:14] fixing these scripts and removing 1.16 is high on my list of things to do [00:20:25] Except testwiki [00:20:41] is there a reason for that? [00:20:43] better schedule a window for testwiki ;-) [00:20:48] And a few private wikis iirc [00:20:50] Guess not [00:20:56] Well php-1.17 != wmf-deployment [00:21:01] Krinkle: Shouldn't be [00:21:01] RoanKattouw: That wiki you fixed a few hours/minutes ago was on 1.16 [00:21:10] No, it was on 1.17wmf1 according to Special:Version [00:21:14] in this channel, wikimaniateam or something [00:21:18] Yes [00:21:20] Hm.. [00:21:21] Aude said it was on 1.17 [00:21:28] Might just be a broken MWVersion.php [00:21:30] [00:21:39] This page was last modified on 13 July 2010, at 17:19. [00:21:41] hehe [00:21:55] ah, purged it. [00:22:00] indeed 1.17 :D [00:22:24] *robla was wondering about that [00:22:46] anyway, maybe we should think about transitioning from here to a more permanent multi-version system [00:22:56] +1 [00:22:57] instead of from here to wmf-deployment again [00:23:16] btw has anyone checked if the job runners are on 1.17? [00:23:33] Good point, probably not [00:23:49] LocalisationUpdate also broke last night. It'll probably log a failure again, I didn't have time to investigate [00:23:51] I'll poke at it tomorrow [00:24:07] I'm doing it now [00:24:12] OK [00:24:22] assuming cawiki apostrophe link trails can wait [00:24:29] Sure [00:24:55] I'll change the maintenance scripts to use /home/wikipedia/common/php for now [00:25:22] that's a symlink that I've been maintaining to point to the current version since not long after the 1.5 switch [00:25:43] and it worked before then too [00:34:45] I have to repackage wikimedia-job-runner to fix it [00:35:00] I'll set up something a bit more permanent before I do that [00:35:20] *robla makes a couple of notes in http://wikitech.wikimedia.org/view/Heterogeneous_deployment [00:35:52] I'll make a script in /h/w/c which will give you the $IP for a given DB [00:36:10] I'll split getMediaWiki() to support it, we don't want the chdir() in there [00:49:00] ok this is getting complicated, I'd better do something temporary [00:49:34] I accidentally created /trunk/tools/multiversion and was just looking up the manual for CDB [00:49:40] time for a temp fix [01:32:44] nothing is ever simple when dpkg is involved [01:33:00] how long did that take? 40 mins? [09:11:10] anyone know how i'm supposed to force loading of jquery ui with RL ? [09:12:20] is it load.using( 'jquery.ui' ) ? [12:27:58] don't anyone edit JavaScriptDistiller until my next commit [12:28:18] I'll put a lock on it [12:31:00] There doesn't look to be anyone about who'll be likely to [12:33:44] I don't think there's a way to do this [12:33:52] so maybe it doesn't matter anyway [12:42:30] Can someone check if LocalisationUpdate is correctly set up (or whatever else that can cause this); http://www.mediawiki.org/wiki/MediaWiki:Codereview-email-body3/fr (other languages too) is still at the 1.16 version and it's annoying to receive this in notification emails [12:43:42] how often does it normally run? [12:44:14] LU? iirc once a day [12:51:03] I checked it this morning my time, and it had already been fixed [12:51:07] but probably less than 24 hours ago [12:51:26] can you have multi-line regex literals in JS? [12:51:39] and before you answer, yes, it would make me cry [12:53:38] I mean, if you want to parse the whole javascript language with a single massive regex, you're going to have problems with edge cases, right? [12:56:28] Indeed [12:56:43] You're increasing the complexity, as well as decreasing readability [13:02:48] hi RoanKattouw [13:02:58] Hi Tim [13:03:01] I was just saying, I'm working on JavaScriptDistiller, I have a lock on it [13:03:05] OK [13:03:12] but I'm not sure if the problem is tractable [13:03:19] Which problem? [13:03:32] parsing javascript with a single regex [13:04:04] without segfaulting [13:04:27] Oh [13:04:28] so more to the point: parsing potentially malicious user input as javascript with a single regex [13:05:06] The single regex is not necessarily required [13:05:21] If you would, say, significantly rewrite ParseMaster [13:05:28] It would have to use armoring [13:06:09] it's simple enough to do with basic string functions [13:06:12] I did it already with JSMin [13:06:23] it's very complicated to do it with regexes [13:07:04] How are you going to, e.g., strip /* */ comments while preserving them in quoted strings? [13:08:11] strcspn [13:09:05] Right [13:10:12] We do have a lot of [...] and [^...] in those regexes, so I guess it could work [13:10:51] Although armoring of quoted strings, if you want to do that, will be slightly more complex I guess because of \" [13:11:42] your method for parsing quoted strings uses up a stack level for every backslash [13:11:49] $ php eval.php [13:11:51] > JavaScriptDistiller::stripWhiteSpace('"' . str_repeat('\\',100000)); [13:11:52] Segmentation fault [13:12:18] I tried to fix it using lookbehind assertions, but it didn't work [13:12:35] Yeah lookbehind assertions in general seem to break ParseMaster [13:12:45] no, I fixed ParseMaster so that they work [13:12:55] Oh [13:13:04] but "\\\\" doesn't work [13:13:27] because I had (? so it doesn't match [13:13:53] you need to know whether there was an odd or an even number of backslashes [13:13:58] Right [13:14:25] you can't do that with fixed-width lookbehinds [13:15:00] maybe I can just support a few backslashes before the quote, expand it out [13:15:54] If you were to convert the whole thing to strcspn, I'm sure this could be done in a nice (as in performance) way [13:16:03] it's already done, it's JSMin [13:16:08] Handling escape sequences probably isn't trivial, but it's doable [13:16:20] I converted it from single-character loops to strcspn [13:16:35] when you consider rewriting JavaScriptDistiller, please note that it's not trivial to safely minify js when allowing arbitrary user input [13:16:44] I can just rewrite it without looking at the original source [13:16:49] that's because of the division/regex conflict in js's grammar [13:16:56] then the license restrictions will be gone, right? [13:17:16] unfortunately this can not be resolved only by looking at the previos token: [13:17:35] (a+b)/(c+d)/g [13:17:36] pawelx: if someone submits broken JS and gets broken output, that is not a big problem [13:17:41] vs [13:17:45] if someone submits broken JS and it breaks the server, that is a problem [13:17:53] if(1)/(c+d)/g.exec(... [13:18:14] You can also just use the "when in doubt, armor it and don't minify" approach [13:18:24] but if someone submits correct js and minification breaks it, that's bad [13:18:28] Which is what simplifying some of the original JSPacker regexes basically meant [13:19:09] I guess you could; if the tactics to do escape sequences with strcspn and such weren't in JSMin and you use the minification operations from JSDistiller, you're fine [13:19:17] now that we have multi-line strings, any string parsing error just runs on forever [13:19:30] all the strings turn into non-strings, and all the non-strings turn into strings [13:19:34] so you can't just ignore it [13:21:15] for now I guess I'll revert to your method, and we can have segfaults [13:21:44] It'd be nice to rewrite it, but you don't have to finish that tonight ;) [13:23:48] I'll add a pcre.recursion_limit hack [13:40:42] I'll put my comments on the bug, so the one who rewrites it can have a look at it. [13:52:35] pawelx: hast du ne idee, warum der JS-distiller pl??tzlich die zeilenumbr??che in der quickbarbox wegmacht? gestern warnse noch da... [13:53:25] I'm committing my fix now [13:53:49] maybe Roan will review it and deploy it, so that everyone can stop complaining [13:55:03] PDD: es gab einige ??nderungen am Minifier, um Fehler zu beheben, vermutlich wurden dadurch wiederum andere Fehler verursacht, sollte bald behoben sein [13:55:27] .oO(ich hoffs mal...) [13:57:02] PDD: I speak German but I'm not /that/ good. What exactly disappeared from the quickbar? [13:57:29] RoanKattouw: the line breaks, and people are now starting to do "fixes" like thsi one: http://de.wikipedia.org/w/index.php?title=Benutzer%3ACa%24e%2Fmonobook.js&action=historysubmit&diff=85449527&oldid=85448980 [13:57:36] argh [13:58:07] Wrong diff? [13:58:29] looks right to me [13:58:48] I don't understand how that could possibly be a minifier issue [13:59:12] me neither, but since this appeared today, it must be related to the changes in the minifier [13:59:19] something is wrong with formey [14:00:03] We changed more things than just the minifier [14:00:24] okay, then it must be realted to the changes ~~in the minifier~~ [14:00:29] Other changes went live today as well, let me look at the server admin log [14:00:43] I wish people would investigate the cause of bugs first before immediately blaming the minifier [14:01:48] PDD: Can you explain the bug in more detail? Which skin, which links, where, etc. [14:03:57] RoanKattouw: Well, if it works with debug=true and the problem is whitespace disappearing from strings in js files, it seems likely to be related to the minifier [14:03:59] RoanKattouw: if I had anything like firebug here at work I could investigate but I don't; that's why I was gently poking pawelx :-) [14:05:43] there was a problem with single-line comments with slashes in them [14:05:48] the following line would be corrupted [14:05:58] that's what I just fixed, is it possible that's the problem you were seeing? [14:06:03] pawelx: Aah OK. The bug with whitespace disappearing from "quoted strings" is fixed now, people just need to whitespace-edit the page the JS is on and wait 5 mins, that should fix it [14:07:13] TimStarling: Does one of those modifiers (/Sxs) say "ignore whitespace" ? [14:07:26] x [14:07:58] OK [14:08:07] r82384 [14:08:25] !r 82384 [14:08:25] --elephant-- http://www.mediawiki.org/wiki/Special:Code/MediaWiki/82384 [14:08:25] I put it in a separate commit so that the diffs would make more sense [14:08:55] OK [14:10:15] + '\\\\.' . // an escaped dot [14:10:37] Maybe the fact that you misunderstood the regex shows how bad it is [14:10:51] It really means 'a backslash followed by any character' [14:10:59] oh yeah [14:11:20] I noticed that later and fixed that comment, but the fix got reverted when I reverted back to repeating subpatterns from my version [14:11:58] Ah [14:12:34] I'll fix it [14:15:33] I'm already on it [14:15:39] Fixed that comment and another one in my local copy [14:16:17] Bleh, you were first [14:18:06] pawelx: Aah OK. The bug with whitespace disappearing from "quoted strings" is fixed now [14:18:12] if you say so [14:18:27] try this: JavaScriptDistiller::stripWhiteSpace( " // /' \nx=' i '" ) [14:18:30] Well the test case provided by the fawiki people was fixed [14:18:45] that's the one that I just fixed [14:20:07] Right [14:20:13] the space slash at the start was seen as the start of a regex [14:20:30] then the second slash was the initial character of the regex, which is not allowed to terminate it [14:20:43] so the third slash is the end of the regex, and then the quote is the start of a string [14:21:03] then for the rest of the script, everything that is inside a string is seen as outside it, and vice-versa [14:22:27] Alright, merging and deploying those fixes now [14:23:35] might want to bump the cache version [14:23:42] Yes [14:26:12] Done [14:31:34] RoanKattouw: line breaks are back, so it was the minifier :-) [14:31:44] Yay [14:31:44] thx for fixing it. [14:31:48] Thank Tim [14:32:53] TimStarling: Thanks for helping out, you rock. Now get some sleep ;) [14:34:06] thx TimStarling :-) [14:34:18] no problem [14:34:23] good night [16:35:47] RoanKattouw: module=user is also loaded when logged out but it seems to be blank (content-length:0). [16:36:03] Any reason not to get rid of that for anons ? [16:36:13] There's a more general bug about that [16:36:19] Not loading it when there are no user JS/CSS files [16:36:25] Ah, right. [16:36:56] well, atleast for anons (temp. solution) would already save a lot of hits, right ? [16:37:09] Possibly [16:37:17] I'll work on it later [16:37:21] 1 http request on all pages for all anonymous user. [16:37:22] I'll add a comment about anons on that bug [16:37:26] k [16:37:36] *RoanKattouw needs a break after last week's craziness [19:09:03] we have a few thousands of svg images which are broken because they do not pass the new XML checks [19:09:19] I'm thinking about reverting, what do you guys think? [19:14:58] *Bryan pokes RoanKattouw [19:15:46] *RoanKattouw is working on school stuff and would like not to do much WMF stuff today, beyond maybe quickly reviewing and deploying some things [19:16:06] Someone relaxed the XML checks regarding namespacing, I think that was deployed [19:16:32] yeah, but previously a lot of stuff that does not validate as proper XML was uploaded [19:17:07] Rigth [19:25:31] for anyone who didn't see the bit on #mediawiki: http://www.mediawiki.org/wiki/Bugmeister/Bugzilla [19:44:37] hmmm I just inspected a random set of 100 svg and none were mallformed [19:44:48] *Bryan wonders about statistics