[10:32:54] brion: It seems to me that the only at-rule we need/want to support is @media. Maybe @supports? (Is that even supported by any UA? I don't recally having ever /seen/ one) [10:33:20] @media definitely, lemme check the docs on the others [10:34:52] Coren: yeah something fancy might want @keyframes for custom transitions but meh. no rush there [10:34:56] @media is super important though! [10:35:36] I've got a fairly decent parser atm - well, "lexer" mostly. Reducing to tokens suffices to extract rules cleanly enough to filter and rewrite. [10:37:44] Doing full-blown validation would require a better parser though and it'd be ugly and slow in PHP. [11:44:42] Coren: cool :) yeah for prototyping the main thing is extracting the selectors (to prefix them) and the rulesets (to copy them) [11:45:00] and preserving the @media block structure [12:39:31] brion: A side effect of the way I rebuild the CSS from the tree merges @media blocks when appropriate and would allow me to be smarter about mergin rules even. [12:39:51] hmmmmm should work as long as same queyr rules [12:40:04] order can matter though :D [12:40:29] brion: Sure, but we need to document that order is not guaranteed since templates can appear in arbitrary order on any one page. [12:42:10] Coren: between separate templates we should be safe since we prefix them yeah [12:42:32] I do preserve rule block order though. [12:43:21] within one you might still have multiple blocks that you want to override, where order changes could change the behavior [12:43:24] but those should be rare [12:43:25] :D [12:44:47] Ima do a commit for the class that does this in a bit so you can take a look at it. Do we have a corpus of stylesheets somewhere I can throw at it? [12:45:14] cd mediawiki/resources && find . -name \*.css ;) [12:45:32] lemme check if any have good @media though [12:45:35] Heh. I suppose that'd be trial by fire. :-) [12:45:42] hehe [12:46:56] UniversalLanguageSelector/lib/jquery.uls/css/jquery.uls.mobile.css has a couple @media blocks mixed in. styles aren't that exciting but it might be a good test [12:53:39] Hm. Do we have a 1-class per file rule in mw? [12:54:00] * Coren needs to actually find and read our style guide. :-) [12:54:22] if we do we sure don't stick to it reliably ;) [13:31:32] brion: https://gerrit.wikimedia.org/r/#/c/282155/ [13:31:49] Needs moar commends, but has a nice little cli testing thingy. [13:31:54] comments* [13:37:43] There's an obvious spot to allow at-rules, and an obvious spot to filter either keys or values in rule blocks (right now, I fiilter none but url() is an obvious one to add) [13:38:05] Ima probably make that latter blacklist parametrizable. [13:43:51] regex tokenizer, cycle through consuming tokens, looks sane enough in principle :) [13:44:11] Coren: i'd consider keeping blacklist-like stuff separate from the parser, though doing them in one go feels convenient :D [13:45:27] Well, the parser doesn't care; the blacklisting would be done during render or just before on the tree. [13:45:36] spiffy [13:48:12] I've ran this on a number of stylesheets (including the one you pointed to) and afaict, the result matches the expectations - though it's a bit hard to veryfy that they are strictly equivalent. I'll create a few valid and broken stylesheets for CI testing. [13:49:00] Right now the parser just gives up and ignores a declaration or rule at the first sign of trouble - that seems to be the sanest behaviour. [13:50:06] (Also, there are a couple "\n" in the rendered output that are there only for the benefit of humans that could probably be elided. [13:50:47] Coren: yeah ignoring malformed rules is per spec so :D [13:51:31] ooh.. [13:51:31] PHP fatal error: [13:51:31] Invalid operand type was used: cannot perform this operation with arrays [13:53:26] eek [13:53:59] * Coren returns to work on the extension part of the extension. :-) [13:54:37] https://commons.wikimedia.org/wiki/Special:Contributions/Reguyla [13:54:41] gives: [13:54:42] PHP fatal error: [13:54:43] Invalid operand type was used: cannot perform this operation with arrays [13:54:46] [13:54:50] what is wrong? [13:55:05] Odd, wfm. [13:55:41] Cause it's been fixed [13:56:22] now works [14:00:17] Coren: the content rules in this file misparse https://brionv.com/misc/ogv.js/demo/iconfont.css [14:00:28] (the font-faces etc are correctly ignored, that's fine) [14:00:44] but the comments get merged with the strings which looks messed up [14:01:51] hmm actually it's not removed either, it was just offscreen ;) [14:06:26] Coren: ah i think it's a greedy capture [14:06:38] Might be. [14:06:38] from ' .... through the second ' into /* ' a comment ' */ :D [14:06:40] * Coren checks. [14:09:22] Wait. [14:09:48] That CSS is actually nonconforming; from my reading of the standard multiline comments are not actually supported. [14:09:58] * Coren double checks. [14:10:14] (I'm clearly not handling it right though [14:10:42] hmmm i'm pretty sur emultiline comments are cool. at least they are common ;) but lemme check the spec [14:11:00] No, nevermind. I'm the confused one. [14:11:12] Strings break on newlines; comments aren't supposed to. [14:11:36] My regex actually erroneously prevents newlines. [14:13:46] aha [14:16:42] how can i manipulate output page from handler file? [14:17:06] the imagepage file calls certain functions from handler file [14:17:54] but i just cant output stuff from, for example, getWarningConfig function can I [14:18:05] because its not designed for that [14:19:08] hmmmmm not easily no Harri [14:19:20] you could do some work arounds though [14:19:55] Harri: you can use general output page hooks (or if there's anything ImagePage-specific) from your extension... [14:20:11] ... which could check if the file belongs to your handler and then do custom checks/output [14:20:18] yeah, that makes sense [14:20:35] Harri: TimedMediaHandler for video/audio files does that to add the transcoding information at the bottom of the File: page [14:21:29] such as on ImageOpenShowImageInlineBefore hook [14:22:53] hmm yeah okay [14:23:23] so, my handler handles everything when 3D-model in STL format is uploaded [14:24:02] have created the viewer for it and tested it (in core :P) and it works fine, but now I just need to refactor stuff [14:25:52] https://www.mediawiki.org/wiki/Manual:Hooks/ImageOpenShowImageInlineBefore [14:25:59] sounds like exactly what i need [14:26:24] awesome [14:26:26] :D [14:55:26] alright something went amiss [14:55:29] "Class StlHandler contains abstract method (doTransform) and must therefore be declared abstract or implement the remaining methods" [14:56:05] hmm [14:56:18] Harri1: missing the doTransform override on the subclass? [14:56:49] i guess so, since my class doesnt contain doTransfrom method [14:57:31] what does this method do exaclty though [15:01:15] Harri1: i believe that's the one that makes the thumbnail image [15:01:18] lemme double-check [15:01:21] yeah [15:01:25] ill save you the trouble [15:01:28] :D [15:01:35] and we'll get to that part [15:01:38] :) [15:01:39] because [15:01:44] yeah just stub that out for now [15:01:52] the fun part is the lient-side viewer ;) [15:01:52] i also ineed to implement thumbnail creation for 3d models [15:01:58] *client [15:02:16] but i think its correct for my class to extend bitmaphandler? [15:02:33] oh wait, i've put it to extend imagehandler [15:20:44] Hm. OutputPageParserOutput lets me addHTML easily, but within the mw-content-text div. [15:22:39] aha. addScript() [15:25:57] hi :), does somone know if it is possible to change $wgAbuseFilterConditionLimit for commons? https://phabricator.wikimedia.org/T132048 [15:26:50] brion: Odd. If I add css with OutputPage::addInlineStyle() it ends up being in the output twice. [15:27:15] Hm. Either that or the OutputPageParserOutput hook gets invoked twice. [15:27:18] Steinsplitter: you're running into it? :O [15:27:35] that's a lot of conditions… [15:27:39] MatmaRex: yes :-(, we run some very ... complex filters [15:28:12] i also disabled all unimportant filters a while ago. [15:29:47] (related: https://phabricator.wikimedia.org/T129171) [15:30:14] Coren: there might be two things getting rendered or some crazy-ass.... ;) [15:32:10] brion: Huh. Happens on Special:Version but on on Main Page. [15:32:34] Coren: OutputPageParserOutput can totally be called multiple times [15:33:19] MatmaRex: Changing the confition limit will make the wiki slower? [15:33:22] And not Special:UserRights. Allright, so I need to figure out a way to determine whether I've been called before otherwise that'll get unwieldy fast [15:33:26] when saving a edit i mean [15:34:06] Steinsplitter: potentially [15:34:44] the problem is as well that i can't find any way to see how much conditions are used. mhm :/ [15:43:46] Steinsplitter: yeah… i'm looking at the code right now and i can't figure out what is it actually counting [15:51:48] anomie, i had a question for you if you are around now. [15:52:02] subbu: What's the question? [15:52:24] so, i am running an import of xml dumps on labs vm running mw-vagrant ... [15:52:32] and on a eswiki import .. i ran into this error [15:52:41] PHP Fatal error: Maximum function nesting level of '200' reached, aborting! in /vagrant/mediawiki/extensions/Scribunto/engines/LuaCommon/LuaCommon.php on line 419 [15:52:48] i can gist you the pull trace if you want. [15:53:30] i was wondering what that indicates .. is that fixable with a config setting? or is something else broken with the m/w install on the vm? [15:56:01] subbu: That indicates that something has a really big backtrace, which might indicate runaway recursion or might be something expected. I'd have to see the full trace to have more of a guess as to which it is. It's possibly fixable by increasing the value for xdebug.max_nesting_level. [15:57:36] anomie, https://gist.githubusercontent.com/subbuss/38d03db74313346f1775b2efbb0d921e/raw/4d34cd7ba4e1e3bc4d5d34af2160c233829d1e19/gistfile1.txt is the full gist. [16:04:39] subbu: That stack trace looks like what you might get if the wikitext has something like "{{#invoke:Module1|func1|{{#invoke:Module2|func2|{{#invoke:Module3|func3|{{#invoke:Module4|func4|{{#invoke:Module5|func5|...}}}}}}}}}}". Or else something is broken somehow so that trying to do the equivalent of expanding {{{1}}} inside the {{#invoke:...}} is instead expanding the whole {{#invoke:...}} again. [16:06:08] this is an import of a dump from eswiki .... so, one possibility is that some update is happening prematurely before all the relevant modules and templates have been imported. [16:07:19] anomie, is that plausible? [16:08:17] subbu: If a module or template isn't imported, it shouldn't go recursive like that. It should just report an error in the parser output that the template/module wasn't found. [16:09:17] right .. that is what makes sense .. but i am clutching at straws to figure out why this might happen given that this is an import of titles that are "functional" on eswiki. [16:09:51] in any case, i could probably delete the offending titles from the dumps for now. [16:11:22] subbu: I may be wrong, but I think our production hosts don't even have xdebug installed/enabled, and I think that's what gives the "Maximum function nesting level of '200' reached, aborting!" error you're seeing. [16:12:08] this import is not running on a production host .. this is on a labs vm. [16:12:26] subbu: Exactly. That would explain why it works in prod but not on your VM. [16:12:33] ah, i see. [16:13:15] so, you are suggesting that this page is "broken" in production, but probably in a different way. [16:13:57] anyway, thanks for your input .. if i can find out what title is causing this, i'll let you know if you are interested in investigating more. but for my purposes, i might just purge it from the dump. [16:14:15] I'm saying that the prod hosts don't care if it has 1000000 levels of recursion as long as things stay within the time and resource limits, while your VM has xdebug installed which adds a check for that. [16:16:36] ok. but separately, given that this is a dump import, why would the parser be involved? i.e. why is the import process trying to render the page .. [16:17:56] oh, probably for link tables and the like. [16:18:10] That's what I was going to say, link tables. [16:41:38] so, Class StlHandler contains abstract method doTransform and must therefore be declared abstract [16:42:06] i defined an abstract method doTransform in my class tho [16:50:34] ah well you can't instantiate a class that has an abstract method [16:50:56] they exist only to make child classes from [16:56:50] okay, so suggestions? [17:16:54] * Coren needs to figure out why vagrant is so ridiculously slow on his box. [17:17:43] Coren: could be several things. if you are on linux you might want to try running it with LXC instead of virtualbox [17:18:06] Yeah, I'm thinking lxc might be a win.\ [17:18:30] it works pretty well on my debian testing box [17:18:50] But also, I'm on Xenial and that might actually be an issue at that level. [17:19:43] I haven't tried with ubuntu newer than 14.04 yet but getting it all up and running on 14.04 was easier than on debian [17:32:44] * Coren facepalms. [17:43:20] so it would appear my stlhandler's ontransform had one parameter less [17:43:28] so it's also facepalm-worthy [17:43:45] Call to a member function getDisplayedFile() on a non-object (null) [17:44:04] im trying to get the full url of the image using $full_url = $imagePage->getDisplayedFile()->getFullUrl(); [17:59:53] whatever, Capitalization error. Now it seems my extension is not being registered anymore [18:20:08] anomie, bd808 https://es.wikipedia.org/wiki/Proteinosis_alveolar_pulmonar is the page on which the dump import fails with the too deeply nested stack exception. [18:20:53] so, maybe i just need to increase xdebug.max_nesting_level and see if that fixes it [18:21:18] subbu: it would be worth looking to see what we set the to in prod [18:21:29] s/the/that/ [18:21:47] anomie thinks xdebug might not be enabled in production [18:22:07] ah. it certainly wouldn't be [18:22:09] bd808, is there a way to turn it off in mwvagrant? [18:22:16] via a role? [18:22:23] or i have to mess with configs [18:24:07] subbu: you need to edit puppet/modules/php/manifests/init.pp and comment out "include ::php::remote_debug" [18:26:05] ok. [18:26:10] and reprovision right? [18:26:25] yes, then run `vagrant provision` [18:26:41] I think that will clean up the ini file [18:26:53] but, so, this will be a local change to the vagrant repo? [18:27:13] i guess i can revert after the import is done. [18:27:31] subbu: yes. In /srv/mediawiki-vagrant on your host VM [18:27:43] got it. [18:29:48] subbu: actually, it may be easier to just delete /etc/php5/cli/conf.d/20-xdebug.ini inside the VM [18:30:01] ah, too late. :) [18:30:07] already reprovisioning [18:30:08] that will get recreated the next time you run vagrant provision [18:30:11] coolio [18:35:04] actually, i don't think i can revert this change since I want the page to render for visual diffing purposes. [18:35:49] subbu: Remember how I said that stack trace looks like what you might get if the wikitext has something like "{{#invoke:Module1|func1|{{#invoke:Module2|func2|{{#invoke:Module3|func3|{{#invoke:Module4|func4|{{#invoke:Module5|func5|...}}}}}}}}}}"? https://es.wikipedia.org/w/index.php?title=Plantilla:OMIM2&action=edit [18:36:33] anomie, nice .. :) so .. I guess I should probably simply increase the allowed nesting depth. [18:37:00] oh gross [18:37:08] that's why we have lua right? [18:37:17] subbu: Yeah. Locally, 1000 seems to be sufficient. [18:37:37] ok .. can you give me a config setting line that I can plop in to a config file? [18:37:56] xdebug.max_nesting_level=1000 [18:38:00] thanks. [18:38:39] bd808, do i need to add that to wiki-specific config files in mw-vagrant (my local wtexp_wiki.pp) or in the /vagrant/Localsettings.php file? [18:39:15] it's actually a php5 config you need to add [18:39:19] The line I told you would go in php.ini or equivalent. [18:39:25] ah, ok! [18:39:39] * subbu isn't thinking [18:40:07] * subbu is hungry and should eat lunch [18:40:12] I'm not sure if we have a nice mechanism for managing that or not [18:40:47] * anomie could use a nice mechanism for managing lunch [18:41:09] heh .. after I commented out that module, see what I get ..."PHP Fatal error: Maximum function nesting level of '100' reached, aborting! in /vagrant/mediawiki/includes/parser/Preprocessor_DOM.php on line 1059 [18:41:09] " [18:41:18] so, 100 now instead of 200. :) [18:41:31] lol [18:41:39] so, i guess tweaking that xdebug setting is the way to go. [18:42:18] subbu: oh! 'xdebug.max_nesting_level' => 200, is in puppet/modules/php/manifests/remote_debug.pp [18:42:26] on that note, i should eat lunch ... [18:42:27] so just raise it there [18:42:34] bd808, alright! [18:42:44] and submit a patch, we may as well raise it everywhere [18:42:47] ok. :) [18:53:07] ping c [19:31:20] Hm. Is there anything else I need to do beyond $parser->getOutput()->setProperty() within the tag handler to add a page_prop? [19:31:30] Nothing in the log, no error... no row in the db. [19:32:54] ... wait, actually, templatedata props aren't there either. Issue with vagrant? [19:33:46] you save it to parser cache [19:33:53] then links update picks that up (or reparses) [19:34:01] and saves it to the tables [19:35:09] Actually, the page_props table is entirely empty in my vagrant instance - that seems wrong. [19:35:39] nah [19:35:44] core doesn't write to it [19:35:49] only a few extensions do [19:36:10] Hm. [19:36:24] Well, TemplateData doesn't either. [19:36:27] * Coren digs further. [19:36:45] like PageImages, Wikibase [19:37:00] sort keys might also end up there [19:37:03] disambig stuff [19:39:38] Hm. Nothing totally broken: putting in a __NOINDEX__ did add the property. [19:40:06] So it's definitely something I'm doing wrong. [19:47:37] Hm. The sample code in the documentation doesn't work either. [19:47:47] * Coren guesses *something* changed. [19:57:47] Gaargh. [19:57:54] * Coren strangles Mediawiki. [20:02:05] Coren: Having fun? [20:03:09] James_F: I'm having an issue I'm increasinly suspicious is not real. Otherwise, and but for that bit, I'd have template styles working. [20:03:57] Wow. [20:04:00] Also, boo. [20:04:14] Attempts to add a property to the page from the tag handler fails silently. I'd be sure the error is in my code were it not for the fact that TemplateData in my vagrant instance *also* fails to put things in page_props. [20:04:58] For that matter, the sample code in the documentation doesn't work either. [20:05:12] Hence, "not sure is true". [20:05:44] Coren, you need to make sure your jobs are being processed [20:05:52] Yeah, that's my first thought too. [20:05:56] cuz it's done via jobs these days [20:06:00] Hm. [20:06:07] TemplateData etc. write their pageProps via DeferredUpdates. [20:06:35] James_F, all LinksUpdate crap uses jobs now [20:07:01] Putting __NOINDEX__ on the page adds the prop though. Not DeferredUpdates? [20:07:20] meh [20:07:35] I would just put a bunch of var_dumps everywhere [20:13:01] Hmph. Well, I got no pending jobs so that's not it. [20:14:09] ah-ha! [20:14:12] Eureka! [20:15:17] Amusing. One of my configs in settings.d started with ' Ha. [20:15:38] So the extensions were there for apache, but not for mwscript and jobs [20:15:44] ... quietly. [20:38:02] James_F: Success! [20:42:53] Yay. [20:58:46] James_F: brion: https://gerrit.wikimedia.org/r/#/c/282155/ [21:01:40] Coren: Ooo. [21:02:16] Oh, I should probably include the template CSS on the actual template page too. :-) [21:02:29] Ha. [21:02:39] Also, `Bug: TXXXXX`. :-) [21:03:09] Hrm. Can I get the page ID from a ParserOutput? [21:04:05] Umm. [21:04:51] https://doc.wikimedia.org/mediawiki-core/master/php/classParserOutput.html doesn't suggest it directly. [21:05:09] Yeah, that's where I was looking. [21:05:55] Can you not use RequestContext? [21:07:02] From an OutputPageParserOutput hook? I suppose so. [21:08:02] Ah, OutputPage has a context. [21:18:08] hi, https://test.wikipedia.org/wiki/Module:ScribuntoTitleProtect Why "editprotected" ? this is bug or not? [21:27:04] rxy: I'm quite sure it's not [21:27:13] there's a permission called editprotected [21:28:00] see https://test.wikipedia.org/wiki/Special:ListGroupRights [21:29:46] my vagrant shows 503 service unavailable [21:29:47] rip [21:40:52] Platonides: hmm .. why "create" and "edit/move/upload" results are different? [21:42:19] James_F: Turns out, getting the template's css while on the template page is nontrivial. [22:08:48] James_F: So, I think the performance cost should be bearable - there is a select during rendering and a bit of CSS juggling. Nothing too catastrophic, but it does add a DB read for every page on which there are transcluded templates [22:09:32] James_F: Also, this whole scheme obviously could never work for subst:ed templates - but that's unavoidable [22:11:03] Coren: Yeah. Except subst:-ed into other templates, I suppose? [22:11:13] (But that might be too magical?) [22:12:06] James_F: Not even; you'll get the non-subst'ed enclosing templates though. One could always put a transcluded style-only template in the substituted one - but that's getting beyond clunky. [22:12:30] (And also mostly defeats the point of subst: in the first place) [22:12:33] True. [22:12:59] I don't think the important use cases include substitutions anyways. [22:13:16] I was thinking that {{subst:A}} on [[Template:B]] would magically merge A's tag into B's inside a set of A-substed-specific wrappers. [22:13:29] But yeah, too complicated and magical and no real use-case. [22:14:59] The missing feature atm is enclosing tags for faux-scoping; but I'm pretty sure that the current iteration is already useful enough. Besides, since that feature can only work for balanced templates, we need to solve that general problem first. [22:15:14] (And there is nothing that prevents users from scoping themselves if they want to) [22:16:02] On the plus side, that extension has zero cache impact. [22:16:12] Nice. [22:16:23] Are we sure we want to do this as an extension and not in core? [22:16:41] brion seemed to think it wiser, at least for now. [22:16:47] The follow-up concept of changing the Parser to ignore inline styles would be… challenging to do in an extension. [22:16:53] Hmm. OK. [22:17:12] I don't think we can realistically remove inline styles for a long, long time. [22:17:43] In templates? Or in articles? [22:18:01] Give it six months of production use before we provide a tool highlighting templates with inline styles in a "must fix" list. [22:18:08] Give it another year, throw the switch. [22:18:22] Either. Remember that there will always be revisions having them that must not break too badly. [22:18:38] But yeah, the decision to make that mandatory is a PM thing. :-) [22:19:07] We are really very bad at supporting reading of old revisions related to templates. [22:19:13] *Very* bad. [22:19:24] True dat. [22:19:25] E.g. deleting images and templates that are "no longer used". [22:19:35] Or transcluding current rather than then-appropriate revisions of content. [22:19:37] Or … [22:19:53] So the use case of "revisions having them that must not break too badly" is… eh. :-) [22:20:03] Yeah, true. [22:54:16] James_F: There's also another good reason to keep this to an extension for the forseeable future; that code is likely to go through a number of revisions and the rythm of core changes may not be ideal. [22:54:24] Sure. [22:54:31] We can always core-ify it. [22:54:41] But doing it as an extension means all the CI bollocks. [22:54:42] :-) [23:06:29] andre__: I can haz TemplateStyles projekts? [23:06:36] (in phab) [23:08:42] Coren: I can make it. [23:10:12] Coren: https://phabricator.wikimedia.org/project/view/1932/ [23:15:37] Danke schoen. [23:16:05] * Coren sets up a wikitech page. [23:22:42] Coren: You mean MW.org? [23:23:03] Yes, yes I do. Brain fart. [23:23:04] Coren: I already made https://www.mediawiki.org/wiki/Extension:TemplateStyles … [23:23:28] Bleh. I've been editing it for a while now. Ima going to trample your changes I guess. :-) [23:49:42] James_F: Mine is bigger than yours. :-P