[04:26:26] [[Tech]]; 132.3.57.80; /* tydero */; https://meta.wikimedia.org/w/index.php?diff=15834721&oldid=15830600&rcid=8133673 [07:35:33] [[Tech]]; Stryn; Reverted changes by [[Special:Contributions/132.3.57.80|132.3.57.80]] ([[User talk:132.3.57.80|talk]]) to last version by ArchiverBot; https://meta.wikimedia.org/w/index.php?diff=15834977&oldid=15834721&rcid=8134032 [17:09:45] Platonides: hii, did you accidentally ban me from #wikimedia-operations? [17:13:02] ottomata: it seems so [17:13:11] it's now unbanned [17:13:49] thanks :) [17:18:46] Fun [18:09:41] Hi everyone. [18:10:06] I have a wikitext from which I try to remove the references in Lua [18:10:15] I'm trying something like mw.text.killMarkers(mw.getCurrentFrame():preprocess(description)) [18:10:32] in mw.log it works just fine [18:11:00] but on preview, I still have the references list [18:11:39] not the references themselves (the numbers in the text) just the list [18:11:46] any ideas on how to get rid of that/ [18:11:52] a reflist will always be shown at the end of the page on preview, as if there was a tag at the end of the wikitext [18:12:29] yes, but since the references where removed, why does the tag still see them? [18:12:47] isn't that part made after the markers are replace? [18:13:10] no idea [18:13:28] i think the refs are "gathered" when parsing the tag [18:13:42] and it doesn't matter if you remove it from output [18:13:56] e.g., something like {{#if: foo | a | b }} [18:14:08] no ref is shown on the page, but probably there will be one in the reflist [18:14:08] hmm, that would suggest the only solution is to remove the refs with regex or similar [18:14:12] I would expect killMarkers to remove more than just refs [18:14:13] references are weird. :/ [18:14:35] Platonides: yes, that's what I want [18:14:50] what's your use case? [18:15:21] I'm trying to automatically generate an excerpt for the FA on the first page [18:15:39] oh, cool [18:16:00] so I take section 0 and try to remove the images, refs and whatever else doesn't belong [18:23:22] strainu: if you want to suppress the reflist, i think you could make it parse the wikitext '' and discard the output. [18:23:46] MatmaRex: no luck, it's not in the wikitext [18:23:54] it's probably added afterwards somehow [18:34:19] strainu: but that's just for the preveiw I think [18:34:52] nope, check out https://ro.wikipedia.org/wiki/Utilizator:Strainu/Pagina_principal%C4%83 [18:36:55] hmm [18:37:02] I think it's a new feature [18:37:27] that it automatically adds the references if a wasn't in the wikitext [18:38:06] it may be better to remove them from the wikitext [18:39:27] strainu: hm [18:40:05] strainu: you're doing preprocess() twice, that is probably doing something silly [18:40:28] strainu: if you add `unused = mw.getCurrentFrame():preprocess('')` after preprocessing, the references disappear for me. [18:40:44] strainu: do you mind if i edit "Modul:Pagina principală"? [18:40:50] no, go ahead [18:42:05] https://ro.wikipedia.org/w/index.php?title=Modul:Pagina_principal%C4%83&diff=10706792&oldid=10706784 [18:42:18] that seems to work [18:46:24] MatmaRex: very nice, thank you! [19:16:48] haha, MatmaRex's solution puts the page in "Pages with reference errors" :)) [19:16:54] as you said, references are weird [19:17:04] I'll just remove them from wikitext [19:17:39] it does? blergh [19:17:47] references are terrible. [19:18:24] yep, just notices [19:18:33] noticed [19:18:37] LMAO [19:38:03] Platonides: I'd love to talk about the history of Poolcounter with you if/when you're available [19:38:17] oh [19:38:18] sure [19:38:29] is now a good moment? [19:38:37] yeah, absolutely [19:38:53] btw I think you were going to tell me something some time ago... [19:39:20] ah, it was this [19:39:36] yeah, we just hadn't had time to sync up [19:39:52] but I think they already published a blog entry about it [19:40:30] they did, but I think we might be ripe for a followup. as I recall, we didn't talk much (or at all?) about your involvement [19:40:59] I think I wasn't even mentioned xD [19:41:07] d'oh! [19:41:46] you wrote most (all?) of the original code for it, right? [19:42:20] yes [19:42:30] there was a proof of concept in python by domas [19:42:54] and Tim had done a bit of the php extension side [19:43:24] I took some code from memcached [19:43:40] and made a daemon from it [19:43:48] neat! [19:43:49] and adapted the php [19:44:15] what inspired you to get involved solving that problem? [19:44:30] that's a good question [19:44:35] I don't remember xD [19:44:57] my guess: peer pressure! :-) [19:45:03] it was years ago! [19:45:49] I based on memcache code [19:46:09] so that there would be no efficiency concerns [19:46:27] if it was efficient enough for memcached, it would be for this, too [19:46:44] seems it was [19:46:54] it's hard to convince people that things would scale to wikipedia scale [19:47:10] so there was a risk that i wouldn't be deployed [19:47:21] which people were hard to convince? [19:47:23] s/i/it/ [19:48:09] actually, there was no such problem [19:48:19] but it was something I had in mind [19:48:41] :-) that's a good thing to be overly cautious about [19:48:55] heh [19:48:56] (especially in 2009) [19:49:05] being just a volunteer developer [19:49:29] who would spend time reviewing it and seeing if it performed? [19:49:38] Tim, probably [19:49:50] but it's not as if he had plenty of time for reviewing other's work :P [19:50:03] looking at the history, seems it was in August 2010 [19:50:17] https://phabricator.wikimedia.org/rEPOC55b72b3de4e2ad2ffab020d373020edc9d69cd54 [19:50:22] * robla looks [19:50:56] that's the initial commit [19:51:40] https://www.mediawiki.org/wiki/Special:Code/MediaWiki/r71805 [19:52:01] ah, Special:Code :-) [19:52:19] * robla looks for the scaptrap tag [19:52:51] I think there wasn't back then [19:53:41] it wasn't a scaptrap anyway [19:53:53] as it was a new extension [19:54:01] mediawiki wasn't using a poolcounter [19:54:14] although there was a bit of stub support [19:54:52] that was when I had just started meddling with Tim's time (or trying to) [19:55:02] hehehe [19:55:56] in early 2011, Tim had tried some experiments with ehcache that didn't pan out, and then said "well, I should probably at least deploy PoolCounter" [19:56:15] to which I responded "what's PoolCounter?" [19:56:31] good times :-) [19:57:21] :) [19:57:27] it was nice that it got deployed [19:58:30] it was a piece of code made by me, running in one of the most visited websites on Earth [19:58:37] I was looking at https://lists.wikimedia.org/pipermail/wikitech-l/2010-August/thread.html [19:58:52] to see if there had been a Michael Jackson problem around that date [19:59:11] Michael Jackson was the year before [19:59:30] I see that there had been some discussion about parser caching [19:59:41] and I had been working there [19:59:52] yeah, I don't *think* we had a quite the same traffic spike for a while there [20:00:19] I touched quite a bit the ParserOptions class [20:00:41] seems to have been at that time [20:00:43] https://www.mediawiki.org/wiki/Special:Code/MediaWiki/r70783 [20:01:13] and that's clearly a reaction to "wikipedia is one of the slower sites on the web " thread by Aryeh [20:01:34] it's possible that I came across the PoolCounter concept due to that [20:01:57] I think it wasn't documented anywhere [20:02:11] and found it when changing the code [20:04:03] there was a thread "[Wikitech-l] error lasted more than 10 minutes...." thread where we wished we were using PoolCounter for one part of the code [20:04:14] * robla looks up link [20:04:52] http://www.gossamer-threads.com/lists/wiki/wikitech/260772?do=post_view_threaded [20:04:58] * robla misses GMane [20:06:23] Tim deployed PoolCounter for the parser cache in early 2011, but it took a while before it found more widespread use [20:06:42] (and in fact, there are probably plenty of places where we should be using it, but aren't) [20:07:08] where else are we using it? [20:07:26] I think that for a long time less than a handful of us knew about it [20:07:42] that thread is from 2011 [20:10:17] re: "where else are we using it?" I'm actually not sure. I suspect that that "error lasted..." thread resulted in more use in ResourceLoader [20:12:00] oh, wait, that was about the FundraiserStatistics code [20:12:30] (the thread, that is) [20:13:15] I saw Roan's name all over the thread, and mentally assumed (based on what he was working on at the time) that it was about ResourceLoader [20:14:43] Is the exclamation mark when you patrol pages no longer supposed to be red? The "Mark as patrolled" link also seems smaller. [20:14:44] you know, Roan did more things than ResourceLoader [20:15:29] regarding the problems "convincing people" [20:15:49] I remember for instance how almost any proposal that required a schema change [20:16:26] was quite throwed back because that needed eg. adding a column [20:16:50] I don't remember how many db servers we had per cluster at that time, though [20:19:01] Srdjan_m: I'm not sure what's changed about patrol pages, but if someone else does, please break in [20:19:11] looking at https://lists.wikimedia.org/pipermail/wikitech-l/2009-June/thread.html [20:19:21] I don't find the report about Michael Jackson [20:19:50] * robla looks for the MJ thing himself [20:20:02] robla - I think it has to do with this https://gerrit.wikimedia.org/r/#/c/303353/ [20:21:28] Because I know it was working fine before 1.28/wmf.14 [20:23:20] MatmaRex: did you test it? [20:24:11] it's a bit unsetting tha the author said: The whole change is completely untested by me. Please test. [20:27:11] Platonides: what? [20:27:27] that change that Srdjan_m mentioned [20:27:40] 20:12 < Srdjan_m> Is the exclamation mark when you patrol pages no longer supposed to be red? The "Mark as patrolled" link also seems smaller. [20:27:48] 20:17 < Srdjan_m> robla - I think it has to do with this https://gerrit.wikimedia.org/r/#/c/303353/ [20:28:48] pretty sure i did [20:29:48] probably caused by another revision, then [20:30:00] it's possible i missed this [20:35:07] Platonides: Srdjan_m: eh, yeah, i guess i didn't test it well enough [20:37:13] Platonides: Srdjan_m: want to file a bug? or should i? [20:37:29] Srdjan_m: great debugging! [20:37:49] i think i'll be reverting that, since now that i look at it more closely, mediawiki.page.patrol.css groups styles for three rather unrelated things… [20:40:48] hehehe [20:40:54] MICHAEL JACKSON INSPIRED US [20:40:56] I think [20:40:58] Platonides: ^^ [20:42:11] robla: Platonides: https://dom.as/2009/06/26/embarrassment/ [20:42:40] hi domas ! ah, yes, that post :-) [20:43:16] there's a comment underneath [20:43:26] python poc - https://code.launchpad.net/~domas-mituzas/+junk/poolcounter/ [20:44:29] thanks for pointing that out! I skimmed the comments too quickly the last time I saw your post [20:45:30] though, sweat equity matters most and Platonides is the one who actually addressed the problem, hehehe [20:46:49] well, thank you for your patience with my clowny-ness ;-) I don't think I would have learned about what cache stampedes are without the 2010-11 badgering from y'all [20:47:41] Also, what happened to parser profiling data? I remember it being available at the bottom of articles when you went into edit mode. [20:47:52] * robla still vividly recalls in 2011-ish when Domas pulled him aside to explain the importance of performance work [20:48:57] robla: apparently whining is my biggest skill [20:49:05] xD [20:49:58] Srdjan_m: I'm not sure [20:50:37] domas: stop humblebragging ;-) [20:50:44] on launchpad? [20:50:50] robla: hehe [20:50:52] funny [20:50:54] robla: I just don't know how to write code [20:50:57] guys got any core bugs you need sorted out real quick [20:51:08] Platonides: I used to like bzr for how egalitarian repos used to be :) [20:52:21] I wasn't keen on the move to git at the beginning [20:52:31] then came to love it [20:52:56] git doesn't scale! [20:53:04] git was a pain the buttocks [20:53:06] I giggle each time someone ponders wikipedia-on-git [20:53:14] they have not seen how large git repos break [20:53:49] git isnt the easiest thing to set up i think we should purly use github and gerrit together and take git and give it the good ol statue of liberty boot [20:54:13] git storage format is theoretically very simple [20:54:17] How do you use github and gerrit without git? [20:54:30] Platonides, robla: https://tools.wmflabs.org/bash/quip/AVZ7YmlXaH8PnNb4Dvwc [20:54:42] Platonides: text files are simple too [20:54:43] you find the deflated-blobs-with-sha1-name description everywhere [20:54:50] Platonides: doesn't mean you can run wikipedia off a text file [20:54:51] :) [20:54:56] Reedy: u have a bot transfer github files back and forth [20:55:07] RoanKattouw: Platonides : yes, yes, I know :-) [20:55:09] if you did so, the storage requisites are high [20:55:16] Zppix: How does that help? [20:55:20] You dont really need a bot [20:55:37] domas: that's what dokuwiki? used [20:55:53] Platonides: they used many files [20:55:59] yes [20:56:04] when you use many files, file system matters [20:56:08] until the filesystem breaks :P [20:56:18] remember, swift file storage at wikipedia relied a lot on file system at first and that was a disaster [20:56:26] Reedy: one you wouldnt require an SSH key [20:56:38] Why? [20:57:02] Zppix: note that I talk about using git locally [20:57:06] gerrit is another beast [20:57:13] oh i se Platonides [20:57:22] nvm then [20:58:17] domas: well, once we got the SSDs in for sqlite file indexes, it became workable [20:58:57] well, it didn't even use sqlite at some point [20:59:06] the hash files in each directory were killing all the io [20:59:15] there's very basic math there [20:59:28] me and sql dont agree with each other ever hence why i refuse to work with dbs [20:59:29] still, a basic git sharing over ssh is simple [20:59:47] Zppix: the good thing about databases is that they often get the storage right (unless it is called MongoDB) [20:59:53] Hahah [21:00:20] domas: I think they used the classic open source model of "let's try something simple" "crap, that didn't work, how about this?"...etc [21:00:21] you can do one I/O per operation or you can do twenty [21:00:26] Don't get me wrong i love databases but setting them up is (insert french here) [21:00:26] robla: yeah [21:01:20] it can easier to fix things when you err on the side of "too simple" than when you overengineer [21:01:43] I'm thinking about writing some more documentation to put in mw/core repo is there a process i have to go through? [21:03:24] not really [21:04:21] Zppix: in the mw/core repo? is there a patch you have waiting, or are you asking about how to submit one? [21:04:29] Making sure before i get an angry mob of devs coming outside my doorstep [21:04:48] robla: no i was making sure i didnt have to do anything special to write brand new documentation if i needed to in the future [21:06:05] I'm a newer MediaWiki developer so i'm still learning some stuff hence why i'm not writing new features... that and I'm sure WMF likes there server less explosive :P [21:06:26] well, English text is easier to review, too :) [21:06:33] and copyedit :P [21:06:39] i love copyediting and helping with l18n :P [21:06:44] Oxford comma! [21:06:53] ? [21:07:10] there's ample space for bikeshedding, too [21:07:17] https://en.wikipedia.org/wiki/Serial_comma [21:07:26] Platonides: are you feeling ok? [21:08:05] xD [21:08:34] :-D [21:09:01] Platonides: i accidently reported myself to AIV using huggle before :P [21:09:02] robla: there's that, but then there's using too simple things where they're not good enough :) [21:09:42] lol Zppix [21:09:56] that's typical of you, domas [21:10:03] finding out people doing that [21:10:06] I'm sure an admin saw that and was like... facepalm [21:10:31] Platonides: I'm just an accountant [21:10:32] and janitor [21:10:40] I count things and then clean them up from time to time [21:10:49] I'm a very active editor so when i reported my self accidently AIV well grow pretty big [21:11:09] you managed to report yourself to AIV? [21:12:43] robla: I still miss those times when I could 'tail -f error.log' [21:12:45] on wikipedia [21:12:48] and go after everything [21:13:03] doing that at FB is a bit more challenging [21:13:36] you can still tail the exception log ;) [21:15:17] whats the defference between fetch and prune? [21:17:49] oh domas [21:17:52] you were at facebook [21:18:06] Platonides: when? what? [21:19:03] I didn't remember you worked at facebook [21:19:11] Platonides: I am at facebook [21:19:13] still. [21:19:16] old fart here by now [21:19:33] joined november2009! [21:19:46] I recently got a bit angry with whoever wrote a mysql client extension there [21:19:55] squiggle maybe? [21:20:13] squangle? [21:20:15] that ends up in a loop [21:20:17] probably [21:20:50] and after following the code, you end up find a TODO: comment there [21:21:22] hehe [21:21:32] yes, squangle [21:23:42] really, does facebook use AsyncMysql ? [21:24:02] (in production I mean, not in some toy project) [21:24:14] I think we use it everywhere [21:24:22] :) [21:24:23] kinda [21:24:36] legoktm: are you around? [21:25:06] Platonides: nowadays we even have a 'db client' team [21:25:37] hmm [21:25:38] what's that? [21:25:44] a team [21:25:46] that works on db clients [21:25:46] :) [21:25:51] and proxies [21:25:53] and protocols [21:25:55] and what not [21:26:54] where does wikipedia store db map nowadays? [21:26:56] still db.php!?!?!? [21:27:13] https://noc.wikimedia.org/db.php [21:27:21] nice! [21:27:41] we've a dbtree! [21:27:45] 9 replicas?!!?!?! holy shit you have lots of money!!!! [21:27:53] I run one per datacenter! [21:53:16] question can i tag 2 phab tasks in a gerrit patchset [21:53:35] nvm [22:05:56] Repeatedly getting "An error occurred while attempting to preview your changes. [22:05:56] HTTP error: timeout [22:05:56] " errors when trying to preview edits/diffs today. Known issue? (Saving them works fine, and I'm using live preview.) [22:06:05] (dewiki) [22:10:25] Is there a way to get the exact result of {{subst:#lsth:pagename}} in Lua? [22:10:48] I tried frame:callParserFunction, but it gives me all the templates expanded, while subst does not expand templates [22:11:13] pajz: I'm not aware of anything (but that doesn't mean much) [22:11:28] strainu: subst replaces the wikitext [22:11:33] if you are using lua [22:11:40] it's as a transclusion