[07:55:33] [[Tech]]; ArchiverBot; Bot: Archiving 1 thread (older than 30 days) to [[Tech/Archives/2015]].; https://meta.wikimedia.org/w/index.php?diff=14908471&oldid=14855576&rcid=7079121 [10:17:14] Graph images are broken, like https://www.mediawiki.org/api/rest_v1/page/graph/png/Community_metrics/0/a53d3a77c8f8f8897153b2200ca083cf36dd47a0.png on https://www.mediawiki.org/wiki/Community_metrics [10:27:20] ouch https://meta.wikimedia.org/w/index.php?title=Special%3AMessageGroupStats&x=D&group=agg-2015_Community_Wishlist_Survey&suppressempty=1 [15:15:03] [[Tech]]; DennisCohick; /* Tech */ new section; https://meta.wikimedia.org/w/index.php?diff=14914239&oldid=14908471&rcid=7080777 [15:18:27] [[Tech]]; Dan Koehl; Undo revision 14914239 by [[Special:Contributions/DennisCohick|DennisCohick]] ([[User talk:DennisCohick|talk]]); https://meta.wikimedia.org/w/index.php?diff=14914327&oldid=14914239&rcid=7080812 [15:38:35] [[Tech]]; Jackonclark57; /* @@@@@!!!USA 1888.413.3809 gateway computers tech support email usa canada */ new section; https://meta.wikimedia.org/w/index.php?diff=14914710&oldid=14914327&rcid=7080952 [15:39:14] [[Tech]]; Matiia; Reverted changes by [[Special:Contributions/Jackonclark57|Jackonclark57]] ([[User talk:Jackonclark57|talk]]) to last version by Dan Koehl; https://meta.wikimedia.org/w/index.php?diff=14914721&oldid=14914710&rcid=7080956 [15:41:46] Krinkle, word. [15:41:53] :) [15:41:55] thanks for le blinkenlites [15:42:12] Krinkle, so you're on the WMF ops team? [15:42:16] I am not [15:42:18] ok. [15:42:23] But I pay attention :) [15:42:25] are most of them volunteers/ [15:42:26] ? [15:42:43] i heard WMF finally got a clue that it could pay money to people for services [15:43:01] after bragging for years how it worked volunteers to a heart attack [15:44:02] Krinkle, you were saying "we" so what's your role [15:44:36] I work in the technology department on the MediaWiki software. [15:44:44] The core software that powers Wikipedia. [15:44:52] Currently focussing on performance. [15:44:55] application performance [15:45:04] and front-end page load performance etc. for user experience [15:45:19] I do work for WMF. [15:45:36] hmmmm [15:45:38] well done [15:45:41] that is da bomb [15:45:44] i'm proud of ya, son [15:45:57] So who are you, what's your interest. [15:45:57] let me tell ya, i have never thought of wikipedia as slow [15:46:01] the user scripts, yes. [15:46:03] but not the site. [15:46:06] Hehe [15:46:09] the user scripts are devilishly slow [15:46:12] screw that. [15:46:19] but i gotta have em. :/ [15:46:31] twinkle and whatever [15:46:32] I've actually been focussing on that (user scripts) a fair bit. [15:46:33] omg so slow [15:46:35] glacial [15:46:40] agonizing [15:46:47] can i speed em up in Chrome? [15:46:49] Nowadays they can be enabled by default on a per-wiki basis [15:46:56] so their SLA does matter [15:47:11] my interest is as a kick-ass editor of maximal quality content that you're serving up hot and fresh every day there [15:47:17] individual scripts may use poor code to bootstrap themselves (e.g. the js code itself), but at least the delivery and storage pipeline is important to us [15:47:22] and my background is as a sysadmin [15:47:28] Because e.g. Common.js is delivered through that system [15:47:30] (ResourceLoader) [15:48:18] I can't make scripts go faster, but we do focus a lot on making sure the retreival of those fake files (from wikipage database) get streamed, cached, combined and minified as fast as possible. [15:49:02] btw, Blinkenlights, now I feel old. [15:49:15] wat! [15:49:22] blinkenlites is eternal [15:49:28] Yes, cheers to that [15:49:37] burned into our eyes [15:49:50] all are changed forever by their visage [15:49:59] remember telnet://towel.blinkenlights.nl btw? [15:50:04] possibly [15:50:08] ohyeah. [15:50:14] and i have a friend at xs4all.nl [15:50:21] he prolly told me [15:50:27] also telnet Star Wars. [15:50:59] okay so what's making userscripts so slow on every single loading of every page on wikipedia? [15:51:11] can i cache those stupid files or what [15:51:18] That's a done deal. [15:51:25] It used to be a big problem [15:51:28] #1 perf blocker [15:51:37] But not anymore [15:51:42] well. i'm lookin at it. [15:52:00] every single page load, the page does several dances and reshuffles its content. [15:52:10] half the time, i do a false click [15:52:39] and Twinkle sometimes does a false revert because it has NO CONFIRMATION [15:52:47] what are they thinking? [15:54:09] i have soooo many ideas for new apps for content and account management. [15:54:53] there needs to basically be a giant wikipedia control panel or user portal. i need a realtime feed of my watchlist and all kinds of things. i have it all written down. ;) [15:55:03] which i guess is community-tech [15:55:10] i dont know. i just found that today. [15:55:37] wikipedia's user experience is one of the worst things i've ever seen in my life by any standards [15:55:47] not your fault; you inherited it a thousand times over [15:56:12] Like I said, I don't work on Twinkle and can't make it perform better by making generic changes to the infrastructure. [15:56:22] But well-written scripts that can perform well used to perform poorly as well [15:56:30] perhaps you can talk to him [15:56:33] because of them being loaded in a synchronous fashion, thus blocking the page [15:56:36] page render [15:56:42] dtm: http://i.imgur.com/X1j5oSg.png [15:56:53] That is the history graph of first render time [15:57:01] let's put it this way. i'm talking about an unloaded 2.66GHz i7 quad core system with Chrome, struggling to death on every page load [15:57:13] The change in august is what my team worked on which effectively changed JS to be loaded asynchronously [15:57:25] mainly, from what i can visually see as a user, rendering Twinkle and such menus [15:57:52] so i'd love to hear anything i can do about it on my system. i'm not a programmer, sorry [15:58:27] oh wow so your team dropped the page load time by about 1/3 [15:58:37] Most time is spent waiting typically in those scripts. And it often does a lot of unintentional layout trashing and read/writes interleaved, causing it to make the browser render partial content while it is rendering, instead of building it and rendering it just once. [15:58:48] So it goes in and out of the DOM much more than it has to [15:58:59] dtm: Sad to hear that you heard of Community tech just today. :( We are having a massive survey to see what our user's needs are, but the proposal phase is over. You can come and vote though! https://meta.wikimedia.org/wiki/2015_Community_Wishlist_Survey [15:59:29] And let me introduce myself! I'm Niharika and I work on Community Tech. :) [15:59:42] Niharika, although i've searched for it repeatedly countless times over the years, and wrote myself little essays and wishlists as condolences in the meantime over the years, i only heard of it today because someone gave me a link to a list of irc channels. [15:59:47] and #community-tech is empty [16:00:00] dtm: #wikimedia-commtech :) [16:00:03] so i followed the link to the site. and was baffled by how MANY sites [16:00:11] for some bizarre reason [16:00:21] dtm: And in our defense, we posted the announcement on all Village Pumps. :) [16:00:23] i dont know if i found the real one in the house of mirrors [16:00:33] Haha. [16:00:57] Village Pump. ....riiiiight.... ^_^ [16:00:58] dtm: Nice talking, my code has compiled so I'm back to work :) [16:01:17] Krinkle, uh oh! okay i shall always take delight in your achievements so do keep in touch [16:01:23] good jerb [16:01:40] Thanks, much appreciated. [16:01:52] dtm: But we're planning on repeating the survey annually, so watch our pages! [16:01:57] Krinkle: Hello! [16:02:23] Sadly my code doesn't take more than 30 seconds to compile ever. [16:02:47] dtm: if you want to improve load speed of userscripts, you'll have to rewrite user scripts :) [16:03:35] and importantly, try to at least use Gadgets instead of userscripts, if they are available. [16:04:20] thedj, oh hey well alrighty then [16:04:36] * dtm uploads his userscripts to.... the c l o u d [16:07:40] * Nemo_bis sprinkles appropriate chemicals to make the sky christal clear [16:10:52] !!!!! [16:13:34] hey would you guys like to examine my ridiculous cacophony of userscripts for any culprits? [16:13:35] https://en.wikipedia.org/wiki/User:Smuckola/common.js [16:13:43] https://en.wikipedia.org/wiki/User:Smuckola/vector.js [16:14:19] and can i configure Twinkle to request confirmation on reverts? [16:14:30] it only prompts me on AGF [16:15:42] https://en.wikipedia.org/wiki/User:Smuckola/twinkleoptions.js is there something i can put in here? [16:15:52] Niharika, what say u [16:16:43] wait can i have Twinkle as a gadget rather than a userscript? [16:16:45] for performance [16:17:41] dtm: Twinkle, the gadget is available on enwiki. Try that. [16:19:30] oh i found the options here https://en.wikipedia.org/wiki/Wikipedia:Twinkle/Preferences [16:23:51] Niharika, is it the same functionality as a gadget? why is it faster? [16:24:31] i have it checkmarked there in my Gadgets list. [16:24:45] i mean i always have. [16:26:18] dtm: I think there's some ResourceLoader magic at play, but not sure. I haven't played with many gadgets. [16:26:32] dude. no wonder it's slow.... [16:27:12] this is in the top 3 of gigantic userscript pages... [16:27:27] :-o [16:28:13] eacht script, is a separate request to the server and separate setup etc etc. this is slowing you WAY down. [16:28:43] k [16:29:28] thedj, is there an easy way to profile them? like 'top' for each page [16:29:42] to list the runtime of the scripts on the page [16:29:54] the total runtime [16:30:30] it's not about runtime in this case. it's about load time. you just have too many load requests. you can never really optimize this. [16:30:41] i had two of my big scripts loading twice lol [16:30:51] thedj, so i can't cache them? [16:31:07] you're saying it's loading the file containing each script? [16:31:26] well they are cached (to some degree), but it's one giant waterfall of actions [16:31:44] in series huh [16:35:06] so are there any megascripts that replace any of those userscripts i have? [16:35:27] like i was saying, there needs to be a single wikipedia equivalent of Reddit Enhancement Suite [16:35:45] to continue to help bring it out of 1995 [16:35:52] It's called "don't switch any damn scripts or gadgets on". :-) [16:35:58] It works beautifully quickly. [16:37:50] actually, it would not be a bad idea to move the majority of scripts in a more general store and build browser plugins around them instead of loading them like we do now. it's probably the only way to make them more effecient. [16:37:59] :-I [16:39:03] though in general, bad scripting is still the biggest problem. [16:39:27] which people cannot be blamed for fully, writing good scripts is REALLY hard. [16:39:40] there should be a code review of common enhancement scripts like this, while encouraging them to consolidate into a suite framework [16:39:53] or migrating them into gadgets [16:40:07] a lot of the ones i have are basic UI functionality [16:41:18] has there ever been any work on a comprehensive enhancement suite like that? to do for the whole general experience what Twinkle does for CVU and such? [16:41:34] no, not really. [16:41:57] does anyone at WMF operate on the feedback from any user experience studies or polls? [16:42:10] or any such volunteers etc [16:43:18] slightly related, https://meta.wikimedia.org/wiki/2015_Community_Wishlist_Survey comes to my mind but that's popularity (poll?) and not UI study [16:43:24] plus wider [16:44:35] how about this https://en.wikipedia.org/wiki/Wikipedia:Vada [16:44:51] "Vada is a multithreaded, Wikipedia framework based in the browser, designed to make it easy to develop tools such as AWB, Huggle and STiki that run in the browser." [16:47:00] when's the last time there's been a usability study? i dont know if it's possible to quantify all the levels upon which wikipedia would fail that, or how it could be calibrated for normalcy. i'd like to read someone's attempt. [16:47:37] https://www.mediawiki.org/w/index.php?title=Special%3ASearch&search=usability&fulltext=Ricerca [16:47:42] dtm: the real problem is how you transition. not what you find. [16:47:54] okay. [16:48:00] see also Flow, amirite [16:48:12] i dont know the status of Flow. or if it's statusless. [16:49:17] Flow is being maintained, but not being actively rolled out anywhere beyond where it is now. [16:49:20] thedj, you're right and i would think it would require comprehensive top-down leadership edicts of Jobsian proportions to make it happen huh? [16:49:32] that's just my impression [16:49:41] dtm: no, that will cause you to be hunted down and lynched by the community [16:49:53] that's part of the problem :) [16:50:04] well no i'm talkin bout Jimbo pullin a Steve. [16:50:19] yes, murder would ensue [16:50:20] for him to maybe tone down the anarchy for a second [16:50:29] the anarchist policy etc [16:50:49] the experiment in chaos theory that is euphemized here as "consensus" [16:50:58] not possible, no one has that power in our communities. [16:51:01] okay. [16:51:05] especially not Jimbo [16:51:10] not even WMF or nobody? [16:51:13] no [16:51:23] so it's pure cat herding across the cosmos [16:51:28] exactly [16:51:31] it's undoing the big bang [16:51:33] REVERT [16:51:50] so that's kinda why i was thinking "reddit enhancement suite" [16:52:32] which people generally consider something that can't be lived without [16:52:36] reddit is orders of magnitude simpler than Wikimedia [16:52:41] yeah. [16:52:57] just. that's the idea of a comprehensive standardized framework. [16:52:57] i have to go... [16:53:04] thedj, okay thank you for your feedback [16:53:13] BE WELL [16:53:58] Nemo_bis, it would appear that vada is a comprehensive wikipedian framework, and A930313 is a master jedi [17:04:04] dtm: seems like many others [17:14:52] Nemo_bis, orly. [17:15:16] are many others widely deployed or written by core infrastructure people like him? [17:15:32] or as comprehensive? i know nothing of them. [17:15:55] does Twinkle or whatever have an easy form for automatically submitting to AIV? [17:16:28] Widely deployed? O_o [17:17:08] * Nemo_bis is having trouble finding a definition of "core infrastructure people" which might apply for https://en.wikipedia.org/wiki/User:A930913 [17:17:46] Nemo_bis, he comaintains cluebot [17:17:48] last i knew [17:18:07] so i guess that's relatively core. i meant "high profile" [17:18:14] or "utterly necessary" [17:18:53] i'm just asking if there are highly viable projects that could be consolidated [17:19:07] There are papers which demonstrated ClueBot is not necessary :) [17:19:14] or asking how generalized they are as a platform for becoming a userscript suite [17:19:18] what?! how [17:19:26] And anyway en.wiki is just a small corner of Wikimedia [17:20:16] http://wikipapers.referata.com/w/index.php?search=cluebot&title=Special%3ASearch&fulltext=1 [17:20:34] http://wikipapers.referata.com/wiki/When_the_levee_breaks:_Without_bots,_what_happens_to_wikipedia's_quality_control_processes%3F [17:21:59] is there somewhere I can go to check the API maxlag level? not sure how it works, does it differ from user to user? [17:22:00] with a sampling of one week four years ago, that's not comprehensive but ok [17:22:42] MusikAnimal: is https://www.mediawiki.org/wiki/Manual:Maxlag_parameter unhelpful? [17:22:42] that's interesting [17:23:41] Nemo_bis: I just want to check how much the servers are currently lagged [17:23:45] without an API operation [17:24:10] I was hoping there was a status page or something, that shows what the bots are going to run into [17:24:14] e.g. not http://status.wikimedia.org/ [17:24:44] my bot is continually hitting maxlag errors, and is programmed to retry three times, waiting the amount of time the server said to wait [17:24:52] I've never seen it fail after three tries [17:24:58] but it's failing over and over [17:25:43] "without an API operation" makes it look like I suggested to travel in a far place [17:25:59] https://dbtree.wikimedia.org/ shows the actual lag, which might not be what MediaWiki believes [17:26:09] http://status.wikimedia.org/ is clueless about everything [17:26:25] MusikAnimal: what wiki are you talking about? [17:26:31] enwiki [17:26:42] Just ignore status.wikimedia.org [17:26:47] I guess I found this https://en.wikipedia.org/w/api.php?action=query&titles=MediaWiki&format=json&maxlag=-1 [17:27:31] great! so the manual page achieves its purpose :) [17:28:06] yes sorry, no need to be rude about it, though [17:28:36] Sorry, didn't intend too. It's truly valuable to test the docs with real docs consumers [17:28:50] Nemo_bis: Then not fix them becuse they have issues? :D [17:29:23] ok, now says {"servedby":"mw1199","error":{"code":"maxlag","info":"Waiting for 10.64.32.23: 6 seconds lagged","*":"See https://en.wikipedia.org/w/api.php for API usage"}} [17:29:36] 17, 19, 23 [17:29:42] dunno what's going on [17:29:58] https://en.wikipedia.org/w/api.php?action=query&meta=siteinfo&siprop=dbrepllag&sishowalldb= [17:30:00] dbtree says db1053 Lag 20 [17:30:35] Reedy: yes but they were all 0 a moment ago [17:31:40] db1053 was worked on this morning *shrug* [17:35:47] very busy catching up or what? https://ganglia.wikimedia.org/latest/?r=day&cs=&ce=&m=cpu_report&c=MySQL+eqiad&h=db1053.eqiad.wmnet&tab=m&vn=&hide-hf=false&mc=2&z=medium&metric_group=NOGROUPS depool seems in order either way [17:46:33] I've pinged jynus [18:25:32] bd808: thanks for https://commons.wikimedia.org/wiki/File:ELK_Tech_Talk_2015-08-20.pdf ; is there a way in logstash to find specific channels or error types which proportionally gre more? [18:25:37] * grew [18:26:14] Like, "Set every line at 100 for its value on $timestamp and then graph the evolution" [18:32:46] Nemo_bis: It's the dumps [18:33:10] Hah! [18:38:12] bd808: or something similar to mostDeviant https://graphite.readthedocs.org/en/1.0/functions.html#graphite.render.functions.mostDeviant [18:42:57] Nemo_bis: there really isn't a most deviant sort of thing that I can think of that can be done in Kibana [18:51:13] bd808: ok thanks [18:54:33] Nemo_bis: you might be able to find some "most deviant" sorts of things using the related graphite data -- https://grafana.wikimedia.org/dashboard/db/production-logging [20:30:22] is it possible to set the page that comes up when i go to http://wikipedia.org? i have to select english every time [20:30:30] it should go to my user page [20:30:46] no [20:31:49] Just go to en.wikipedia.org [20:35:00] k [20:39:48] It's surprising how many people have a "hah!" moment in workshops when I tell them to type "it.wikipedia.org", people never pay attention to the domains. :( [20:43:02] dtm, depending on your browser, you should be able to set a "keyword", e.g. I've got "w" set (http://i.imgur.com/RfzqSeo.png ), so I can just type "w banana" to go straight to an Enwiki search for banana. [20:45:20] ++ [20:46:20] quiddity: btw, do you too dislike the new (Firefox 42) arrangement of search engines by icons? I have multiple with similar icons or no icon, without text things are very difficult [20:48:40] Nemo_bis, I customize the layout, to remove things like the search box. http://i.imgur.com/rVYKcP8.png I just create bookmark-keywords, for anything I search regularly. "mw foo" "m foo" "calvin foo" etc. [20:48:56] ? [20:49:08] Oh. Well, I guess you know what my first name is now. :P [20:49:12] Carry on. [20:51:08] foo? [20:51:32] quiddity: yes but I don't search everything regularly [20:51:46] Not foo, Calvin. :P [20:56:28] Nemo_bis: only when you expand the search engine menu I hope [20:57:44] I don't usually switchn between search engines [20:58:19] Platonides: the search menu now self-expands, that's the matter [20:58:34] And you can't select a search engine unless you type something [20:58:37] let me test… [20:59:06] Nemo_bis, Also, I turn off favicons/site_icons via about:config because they bloat up our bookmarks.json with data:image/png cruft. (mine is 3mb without favicons). So I'm used to not having any! [20:59:28] I see [20:59:42] yeah, quite useless for me [20:59:52] quiddity: I keep faviconsfor historical reference :) [21:00:09] seems an attempt to allow many search engines [21:00:30] there's probably a hidden preference somewhere to disable it [21:00:48] Not very many... there is less space now [21:01:00] you end up needing to change lots of knobs to continue having what used to be default behavior [21:01:02] Unless there is a way to scroll the icons [21:01:24] Oh for sure, I'd never dare doing a clean install and losing my ten years of customisations ;) [21:01:28] :) [21:01:46] I have a pending task of documenting all the things I customize [21:02:00] note that the search engine text is shown onmoseover [21:02:04] *mouse [21:02:18] True but that takes ages [21:05:11] you can disable all search engines [21:05:26] it still expands, but with no iconse [21:06:39] Platonides: what setting is that? [21:07:16] Would be fun to see if such things have any effect on the amount of &sourceid=Mozilla-search searches [21:07:34] not exactly a setting [21:07:43] about:preferences#search [21:07:52] (or the direct link in that expanded menu) [21:07:57] then uncheck all search engines [21:08:05] (Something to be added in http://discovery-beta.wmflabs.org/metrics/ ?) [21:08:33] "The search bar lets you search alternate engines directly" [21:08:43] The word "directly" must be one of the most abused by marketing [21:08:53] hehe [21:09:22] I didn't know that page [21:09:37] For me, that just makes the icons disappear. I want my text back [21:09:47] * Nemo_bis will never udnerstand all this frenzy for icons [21:10:46] yes, that's what it does [21:10:53] https://en.wikipedia.org/wiki/Poor_Man's_Bible needs an update [21:10:55] "with no icons" [21:11:18] as they were present only in icons form, they are gone [21:11:44] omg https://www.wikidata.org/wiki/Q681856 is a nightmare [21:12:25] https://en.wikipedia.org/wiki/Ansgar is the true father of modern UX, apparently [21:13:48] This statue represents how he preconized the usage of grey instead of black https://en.wikipedia.org/wiki/File:Erzbischof_Ansgar_01_KMJ.jpg [21:16:10] Curious: without the Esperanto Wikipedia, people would hardly know Encarta existed https://eo.wikipedia.org/w/index.php?title=Speciala%C4%B5o%3ASer%C4%89i&search=encarta&fulltext=Ser%C4%89i [21:17:10] quiddity, interesting. i have Chrome. [21:18:01] Nemo_bis: I'm quite surprised of that [21:18:25] it's just that the easiest way i have to bring up wikipedia is openapple-t and w [21:18:55] so. i'll take a look at that [21:19:39] Default Chrome takes multiple seconds to open a new tab, due to the silly tiles preloading [21:20:05] (Yet another thing which is equally annoying, but much easier to fix, in Firefox) [21:20:30] Nemo_bis, mine's instantaneous [21:20:33] always has been [21:21:24] for both of them [21:21:56] You must have been lucky then. Chromium sometimes decides to load multiple MB of stuff for each new tab and then you are irrecoverably damned [21:23:15] Presumably the purpose is to make sure that people never learn to type URLs in the location bar, a very pernicious and anti-American habit [21:23:21] kill them ;) [21:23:52] Nemo_bis: that way people would be able to bypass their wise search engine [21:24:08] as well as the ads they sell inside it [21:26:13] Yes that's what I was talking about ;) [21:34:11] Nemo_bis, Platonides I keep a minimal user.js for most of the tweaks i need in new installs. https://dpaste.de/Jv90/raw in case you hadn't tried any of those already. :) [21:36:08] "browser.tabs.tabMinWidth" I don't need, I use TabMixPlus [21:37:37] "browser.newtabpage.enabled" is essential indeed [21:38:10] browser.newtabpage.introShown is false by default I'm told? [21:39:16] "browser.tabs.animate" I didn't have this, turns out TabMixPlus disables it by default [21:39:45] "browser.tabs.autoHide" rotflol I never have a single tab :P [21:40:10] "browser.tabs.closeButtons" also supported by TabMixPlus ;) [21:40:11] re: introshown - not sure, I copied from the linked forum thread. [21:40:47] It seems we have perfect agreement on Firefox config [21:41:27] :> [21:44:24] quiddity: https://www.mediawiki.org/wiki/Special:Contributions/Veodor15 <- one of your friends? [21:47:59] ty [22:17:53] I'm not sure if https://support.mozilla.org/es/questions/1033968 is related to what were were talking about earlier [22:19:25] quiddity: they seem sensible defaults :) [22:46:14] you may also want not to send to Google anti-malware info (filename, url, hash…) of the unsigned binaries you download