[14:00:17] Technical Advice IRC meeting starting in 60 minutes in channel #wikimedia-tech, hosts: @Lucas_WMDE & @cscott - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting [14:50:17] Technical Advice IRC meeting starting in 10 minutes in channel #wikimedia-tech, hosts: @Lucas_WMDE & @cscott - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting [14:50:29] ok [14:59:13] good morning from the US East coast! [14:59:17] When's the session supposed to start? [14:59:21] good morning! [14:59:23] in 1 minute ;) [14:59:27] in one minute by my watch :) [14:59:29] Because I had a question [14:59:35] cscott joined just in time :P [14:59:41] being about something on English Wikisource [14:59:56] https://en.wikisource.org/w/index.php?title=Template:Lang&action=edit [15:00:30] In this template an ugly construction is used to split between a DIV and SPAN style presentation based on a supplied parameter [15:00:49] Lint detects a "Stripped" SPAN that shouldn't be present [15:01:21] So I in a recent revision attempted to ensure the generated TAG was complete by using a #tag: [15:01:23] function [15:01:31] just a second, let me get the header out first: [15:01:40] Welcome everyone to today’s Technical Advice IRC Meeting! [15:01:44] looks like we’ve already started :) [15:01:49] okay, go ahead [15:02:05] However, when i updated the template to use a tag: function instead [15:02:15] it broke many other pages [15:02:36] Such as https://en.wikisource.org/wiki/Page:Beowulf_(Wyatt).djvu/28 [15:02:55] which is currently warning about 'misnesting' [15:03:16] https://en.wikipedia.org/w/index.php?title=Template:Lang&action=edit uses a lua module it seems [15:03:33] it does? [15:04:00] because what I am seeing is template syntax [15:04:08] Albiet a rather complex one [15:04:17] I think you’re looking at different pages – cscott did you intentionally go from Wikisource to Wikipedia? [15:04:26] ShakespeareFan00’s link was to Wikisource [15:04:36] yes, i was trying to see what the equivalent construct looked like on other wikis [15:04:37] I see the evil tag hacks there [15:04:39] ok [15:04:52] english wikipedia seems to have a quite complicated lua module: https://en.wikipedia.org/w/index.php?title=Module:Lang&action=edit [15:05:18] (sorry, indulging my periodic "i wish we had global templates already" dreams) [15:05:31] Lucas_WMDE: When I tried to remove the ugly tag hack [15:05:35] cscott: 🙏 [15:05:36] by using #tag; instead... [15:05:48] it broke the template :( [15:06:04] because a page like :- https://en.wikisource.org/wiki/Page:Beowulf_(Wyatt).djvu/28 [15:06:07] were you using a conditional as the first argument of the #tag? [15:06:14] yes [15:06:43] If was doing #tag:{{if:{{{1}}}|div|span}} [15:06:57] Does that approach not look correct? [15:07:34] My revision was - https://en.wikisource.org/w/index.php?title=Template:Lang&action=edit&oldid=9452256 [15:08:10] Because this apparently broke stuff I reverted [15:08:47] What I am also not understanding is why https://en.wikisource.org/wiki/Page:Beowulf_(Wyatt).djvu/28 [15:08:58] is now saying there are badly nested tags [15:09:11] because everything SHOULD be balanced up [15:09:16] I’m not seeing any error messages on that page [15:09:25] do you have a user script or gadget that shows lint errors? [15:09:36] Linthint.js [15:09:47] it could work, it's just a little bit evil. as a parser guy i always prefer to see full HTML constructs not fragments, though. forgive my broken syntax, but my first attempt would be to refactor the guts of the template (the attributes) into a helper, and then do {{if:{{1}}|
{{lang-contents}}
|{{lang-contents}}}} [15:10:08] (passing appropriate parameters to lang-attribs and lang-contents, of course) [15:10:41] Lucas_WMDE: https://en.wikipedia.org/wiki/User:PerfektesChaos/js/lintHint [15:10:49] Is the script I use [15:10:56] that way each template generates "balanced" (ie, complete) output. but that's probably not immediately what's breaking your version. [15:11:39] On https://en.wikisource.org/wiki/Page:Beowulf_(Wyatt).djvu/28 [15:11:52] the error suggest something is mismatched [15:12:00] But I can't see anything immediately [15:12:17] other than the template contains many
\n type lines [15:12:24] i think `lang="foo" ` is a little dodgy [15:12:32] especially the closing > [15:12:57] i don't think the closing > is actually in there at all, but maybe i'm not seeing it [15:12:59] Collapsing out all the comments is feasible, but make for an unreadable template [15:13:23] you could use around them so they don't show up in the wikitext [15:13:55] when I run `document.body.innerHTML = ' style="color: red">Hi!'` on about:blank the comment doesn’t work properly [15:13:57] but i think the actual 'bug' is the missing > [15:13:59] but I don’t know what tidy/remex does with it [15:14:10] cscott: the closing -->{{{attr|}}}>{{{attr|}}}}}{{Documentation}} [15:15:20] which doesn't have a closing > [15:15:21] I reverted back to an earlier version [15:16:05] https://en.wikisource.org/wiki/Template:Lang [15:16:25] which problem are we currently debugging – that the #tag didn’t work, or that Beowulf still has lint errors after the revert? [15:16:37] Lucas_WMDE: good question! ;) [15:16:50] OKay time to take a step back [15:16:51] the missing > would have been a problme in the #tag version, I think, but shouldn’t affect the current Beowulf page [15:17:03] anyone lazy and willing to do quick CR? (guaranteed quickness) https://gerrit.wikimedia.org/r/c/mediawiki/core/+/523890 [15:17:15] Lucas_WMDE: https://en.wikisource.org/wiki/Page:Beowulf_(Wyatt).djvu/28 [15:17:35] is still showing Misnsted tags after I reverted the Wikisource Lang to a stable version [15:17:41] That's problem No1. [15:18:02] revi: done! [15:18:06] Problem No2. - How to make Lang work as the current logic intends without the ugly construction [15:18:30] Problem No3. - Inconsistent 'start of line processing' and end of block recognition [15:18:56] (No3. is something already noted many times on phabricator. [15:18:57] ok, i can answer question no 1 [15:19:10] cscott: ty :) [15:19:12] idea for problem 2 – always use the same tag but conditionally change the style: inline/block? though that might break a lot of other things depending on the tag name [15:19:19] the parsoid lint errors are generated when restbase re-parses the page, they might not occur immediately after you save [15:19:37] let me check w/ the parsing team to see how long is "reasonable" to wait for updated lints [15:19:53] additional <3 to Lucas_WMDE and tgr [15:20:27] On bothe No1. and No2. I also have a concern that the bahviour of
\n inside a generated SPAN sometimes isn't expected [15:20:39] *behaviour [15:20:51] because mediawiki attempts to 'tidy' up [15:21:14] despite the markup being valid in HTML which removes whitespace. [15:21:38] Inside mediawiki

tags might be inserted where they weren't necessarily expected to be [15:22:06] (or intended to be) [15:22:16] https://en.wikipedia.org/wiki/Wikipedia:Linter says, "After editing, pages are rechecked for lint errors, but the delay can take from seconds to hours. If lintHint says you fixed one or more lint errors, you almost certainly did fix them, even if page information and the specific lint errors page aren't updated yet." [15:22:38] Okay.. [15:23:00] However, and I may have already explained this badly earlier [15:23:06] ok,
inside . let me think about that one. [15:23:21] Not plain
,
\n [15:23:44] line1
line2 would be valid [15:24:04] and in HTML the HTML backend would just remove the \n from the user input [15:25:17] that seems to work in wikitext as well: [15:25:25] hmm [15:25:31] $ (echo 'a
' ; echo 'b') | php maintenance/parse.php [15:25:31]

a
[15:25:31] b [15:25:31]

[15:25:45] but you're saying when you use {{#tag}} this is breaking? [15:25:45] Yes.. [15:25:53] Yes [15:26:12] But as I reverted I'm still not sure why the Beowulf page is generating a stripped tag [15:26:43] Hello tech people :) [15:26:43] cscott: On Wikisource Page:s have a header, body and footer [15:26:57] Vermont: hi :) [15:27:21] $ (echo '{{#tag:span|a
' ; echo 'b}}') | php maintenance/parse.php [15:27:21]

a
[15:27:21] b [15:27:21]

[15:27:48] Odd. [15:27:57] i think what's really happening is that your template is letting a newline into the output wikitext [15:28:10] Or not [15:28:10] so your \n is becoming a \n\n, which becomes a

, and

breaks out of a span [15:28:26] cscott: That would be No3. point [15:28:35] $ (echo '{{#tag:span|a
' ; echo; echo 'b}}') | php maintenance/parse.php [15:28:35]

a
[15:28:35]

b [15:28:36]

[15:29:11] partly.. It would be nice to have ONE handling rule so contributors don't need to know the minutiae of whitespace when nesting templates [15:29:36] I have had this 'whitespace' leakage issue with other templates [15:29:59] or having to insert an \n to get Mediawiki to wrap a block based template correctly [15:30:11] (Known issue noted in Phabricator for some time) [15:30:36] Related is {{template}\n{{othertemplate}} [15:30:58] And on Wikisource Proofread Page when processing a Page: adds it's own \n [15:31:07] when processing headers and footers in some instances. [15:31:28] or spaces.. [15:31:49] I can't immediately think where Lang is leaking though [15:32:06] unless the new line is occuring because something like [15:32:25] {{blockcenter/s}}{{Lang....}}{{block center/e}} [15:32:43] is somewhere putting in a \n that Mediawiki is trying to tidy [15:33:08] I would appreciate someone else looking at the template concerned [15:33:14] I suspect it is T14974, which (almost) always inserts a newline before a template [15:33:15] but thanks for the hints [15:33:15] T14974: The newline added to a template, magic word, variable, or parser function that returns line-start wikicode formatting (*#:; {|) causes unexpected parsing - https://phabricator.wikimedia.org/T14974 [15:33:28] cscott: Thanks [15:33:41] So to summarise... Not yet easily solved [15:33:54] so something like `a
\n{{template}}` as the contents causes the \n to become \n\n and thus a

and breaks things. [15:33:54] but the #tag: approach would have been a reasonable solution? [15:34:09] Yep [15:34:16] That was my thought as well. [15:34:38] I also note on some pages that I am seeing constuctions that put a block level element [15:34:43] inside a Reference span [15:34:55] That may be another reason why some Lint warnings arise [15:34:56] i think so. but honestly, so would just refactoring the both into a {{lang-attributes}} helper and then making the template `if inline then {{contents}} else

{{contents}}
` [15:35:07] and (to me) it seems the latter is easier to read than then {{#tag}} version [15:35:33] cscott: The thing is that lang already has a lang-block equivalent [15:35:37] on Wikisource. [15:35:46] that is, instead of going to extremes with {{#tag}} to avoid repeating the middle part, you could just pull the middle part into its own template and repeat yourself a little bit. [15:35:59] and the inline parameter is used to tweak the code [15:36:05] well, then lang-block could reuse the {{lang-attribs}} part too ;) [15:36:10] Yes [15:36:15] That was where I was going [15:36:27] cscott: Thanks [15:36:46] I'll have a look into these approaches and see if they resolve the problems by the next technical meeting [15:37:11] I did have one other point I'll raise here [15:37:26] if you find a specific place where the \n is breaking the i'd be happy to look at it in more depth, but i'm almost certain you'll find either an escaping \n in the template definition or an implicit \n at the start of a template expansion [15:38:06] Related : https://phabricator.wikimedia.org/T49544 [15:38:23] And my other point was on something else entirely [15:38:47] the implicit \n at the start of the template expansion is The Worst, but it's been around for over a decade now. You can sort of work around it sometimes, but a proper solution will probably have to wait for a little while longer; the parsing team has Lots of Ideas. [15:39:34] once we’re back to a single parser, everything will be rainbows and unicorns, or so I hear [15:39:57] starting the template with is a good way to ensure that you won't get an extra newline inserted; it is only inserted if the first character emitted by the template is * # : ; {| [15:40:19] Lucas_WMDE: alas, getting to a single parser only means we can *start* to address the more fundamental issues [15:40:32] shh, don’t ruin my illusion [15:40:32] which right now would require hacking two parsers at once to address [15:40:48] Okay my other query was - [15:40:49] https://phabricator.wikimedia.org/T196440 [15:40:49] I’m still excited for it ^^ [15:41:15] i'm excited that folks are actively using the linter to remove old wikitext cruft! [15:41:21] To adress T196440 I wrote a somewhat complex template on Wikisource to force a value [15:41:21] T196440: Provide a clearer way to distinguish between "absent" and empty/blank parameters when handling them in templates and parser functions - https://phabricator.wikimedia.org/T196440 [15:41:39] However my view was that this REALLY should be something in the parser [15:41:55] #cook:{{{param|}}} for example [15:42:17] So you are garunteed a sensible return [15:43:51] I'm mentioning this because of your interest in the parser [15:44:40] Over on Wikisource I have the situation of certain templates that do pre-processing for a lower level one and being able to #cook: inputs would be rather useful [15:45:01] especially when the 'null' and default logic in a #switch need to different [15:45:22] *'null', 'empty' , 'blank' or default sorry [15:46:11] (commented on T49544) [15:46:12] T49544: list item must not wrap the text in - https://phabricator.wikimedia.org/T49544 [15:47:25] does anybody else have questions, by the way? [15:47:51] cscott: It'd be nice if you didn't say "sure" on a task I've declined. :-P [15:48:26] James_F: it seems it was reopened by Powers That Be after you declined it [15:48:26] Would there be any interest in these meetings being logged like Office Hours are? [15:49:03] sario258: {{vote|Support}} for logging technical advice sessions [15:49:09] James_F: i'm not weighing in on whether this is "as designed" just that properly enforcing the block or inline context is something we've thought a lot about, and is likely to show up in the parser at some point. [15:49:12] cscott: No, it was edit-warred on. We generally don't feed anger by firing back, per long-standing Phab policy. [15:49:54] Obviously we just output the entire DOM in custom tags with class-based shadow DOM styling. ;-) [15:50:04] James_F: My suggestion on T49544 was to keep ref as SPAN but have as the DIV based one [15:50:14] subbu|busy: said 'sure' first ! ;) https://phabricator.wikimedia.org/T49544#4124810 [15:50:43] sario528: they’re already logged by wm-bot (https://wm-bot.wmflabs.org/browser/index.php?display=%23wikimedia-tech), would separate logs be useful? [15:50:53] James_F: I will also in passing note that there are also instance on Wikisource, where you have Sidenotes [15:51:16] if the outcome is likely to be useful to others, we sometimes try to record it on Discord as well (https://discourse-mediawiki.wmflabs.org/) [15:51:24] for which there isn't currently a stable long term implementation [15:51:27] though I don’t think that applies to this tag discussion [15:51:28] "Might be irrelevant once changes happen" and "we'll intentionally break the output to fix some bad content" are different responses, and you know it. :-) [15:52:19] Whilst on references , I'll also mention something I don't think Mediawiki does yet.. [15:53:04] which is references defined in a list at the start of content, but to which link backs are placed later in a work [15:53:17] can't handle these... [15:54:26] I've noted that on some ancient items of legislation, on the UK SLD, sometimes what would otherwise have been sidetitles are collapsed into a section header at the start of a section [15:55:28] having a that could be defined in advance would be useful for implementing a responsive design like this at Wikisource [15:55:43] I'll get my own strait-jacket now :) [15:55:51] Lucas_WMDE: I could see a benefit for actual wiki pages of each meeting log, as a reference [15:56:13] sario258: With the more colorful invective edited out? [15:56:23] :lol: [15:56:32] James_F: edited T49544 to more carefully not take a position except a technical parsing one. [15:56:34] T49544: list item must not wrap the text in - https://phabricator.wikimedia.org/T49544 [15:56:34] Rather then having to comb through the complete log [15:58:16] * James_F nods. [15:58:25] cscott: Thanks. :-) [15:59:12] cscott: Mostly I think we need to scrap the Cite extension and replace it with several things that work better for the different use cases. [15:59:21] But we've had this conversation many times. :-) [15:59:34] sario528: well, IIRC most TAIM days there’s not much action in this channel outside of the meeting [15:59:51] ShakespeareFan00: so T196440 reminds me of one of the loose ends of heredoc arguments: https://phabricator.wikimedia.org/T114432#4717140 [15:59:52] T196440: Provide a clearer way to distinguish between "absent" and empty/blank parameters when handling them in templates and parser functions - https://phabricator.wikimedia.org/T196440 [16:02:16] Lucas_WMDE: True, but if we convert to a wiki page, rather than the raw text, we could add categories to discussed subjects and it would show up in search [16:02:26] true [16:03:46] anyways, we’re out of time, that concludes today’s Technical Advice IRC Meeting! [16:03:51] thanks to all who participated :) [16:04:06] don’t forget you can always leave questions over at the Wikimedia Developer Support forum: https://discourse-mediawiki.wmflabs.org/ [16:04:11] or keep talking here, of course ;) [16:04:30] cscott: Can we continue the Lang discussions? [16:04:43] I've started the split down of the tempalte [16:04:48] *template [16:10:49] ShakespeareFan00: I commented on T196440 as well. Not a full solution yet, but it's definitely something i've been thinking of (mostly in the context of the new heredoc syntax) so almost certain your use case will be accounted for in whatever we eventually come up with here [16:10:51] T196440: Provide a clearer way to distinguish between "absent" and empty/blank parameters when handling them in templates and parser functions - https://phabricator.wikimedia.org/T196440 [16:11:19] i'm giving another "let's rewrite the template system from scratch" talk at wikimania this year, i'll probably work in the default-arguments use case into that as well ;) [16:11:51] ShakespeareFan00: and sure, hit me up with any continued questions about the lang template [16:11:58] Will do [16:12:09] or hop over to #mediawiki-parsoid, where I "live" [16:12:21] cscott: Outside of the meeting but I have another parsoid related thing 've recalled [16:12:32] TO do with subst ing [16:12:39] Let me find the ticket [16:14:02] cscott: https://phabricator.wikimedia.org/T185379 [16:14:22] which came out of a use-case at Wikisource again [16:14:25] thanks for asking all the parsing questions! ;) makes me feel like a useful co-host. [16:14:49] In the ticket I've linked I mention a specfic template and use-case [16:14:59] Current subst is litteral [16:15:07] It doesn't examine parser functions [16:15:16] it just puts them in directly [16:15:45] In the use case an #out: function which subst based on processing the Parser functions would have been very powerful [16:16:19] I coudl write say {{playing-card|2|h}} and it subst into the page the full File:syntax needed [16:16:38] whilst still letting the underlying template use the full extent of parser functions and magic words.. [16:16:58] Sorry {{#out:playing-card|2|h}} [16:17:36] What get's subst out of the templates mentioned currently is a Pain to clean up [16:18:04] Some people use templates like text macros... [16:18:52] and being able to more cleanly subst the Output, which is typically what is actually desired would be better than having to subst all the parser function code as well [16:21:53] cscoot: BTW don't be afraid to look through some of my old phabricator tickets:) [16:22:03] There are some ideas I closed due to lack of support [16:22:16] but you might want to look at again [16:22:18] :) [16:22:28] I commented on the bug. I don't think it would be too hard to implement. The Q would be, what to call it, would it cause other problems, is this a feature we want to support forever and ever, etc. [16:22:53] cscott: Many of my tickets should be moved over to being feature request tasks [16:22:58] I don't see any obvious problems there, but then again i'm just a parser hacker not an editor on the projects [16:22:59] (When that is possible) [16:23:22] cscott: I also had a ticket about TemplateStyles allowing custom properties [16:23:30] looks like James_F triaged this as "enhancements" not "terrible crazy idea" which is a positive sign. ;) [16:23:31] so called CSS var.. [16:24:16] offtopic re: your nick -- do you have any thoughts/feelings about cymbaline? [16:24:29] cscott: Not a play I've seen yet sadly [16:24:37] And the handles is something of an accident [16:24:45] the shakespeare in the park group here where i live is performing it this summer, and i realized it's one of the plays I barely remember reading, and i'm pretty sure i've never seen performed [16:24:54] I set it up when trying to get an IRC drama project going and it never happened. [16:24:55] the wiki article indicates it used to be quite popular, though [16:25:17] Essentialy the handle was related to a project that never happened [16:25:38] but by that time it was already associated with a lot of accounts so wasn'teasy to change [16:25:42] :lol: [16:26:15] cscott: I had one other idea, but it's best taken to PM possibly as it's not actually Mediawiki related yet [16:28:05] but was more to do with a scriptwriting program called trelby [16:49:59] cscott: On Lang , short of turning it into a Module, you meant something like? [16:50:00] https://en.wikisource.org/wiki/Template:Lang/new [16:50:36] I am thinking my tag approach was along the right lines [16:58:52] whew, yeah i see what you mean about passing default arguments! [16:59:18] just pulling the attribs part into its own template is pretty annoying [17:00:08] but that's what i was thinking, yes. I think you've got an unclosed