[04:27:09] neilk_ you are working on statistics for the UploadWizard ... do you already have some preliminary figures ? [04:27:33] nothing yet, sorry [04:27:39] I want to get you more uploads from other languages [04:27:47] I'm going to look at survivorship rates once we go default [04:28:00] it's too skewed with volunteer testers [04:28:07] me too! [04:28:16] we were just talking about that today. Did Kaldari contact you yet? [04:28:23] no [04:28:54] Are you involved with translations? [04:29:03] given this post of mine http://ultimategerardm.blogspot.com/2011/05/malayalam-loves-wikimedia-interview.html it is easy to get Indian localisations [04:29:12] I think we were going to talk to siebrand & nikerabbit about how we go about getting more localizations [04:29:31] I do the promotion for translatewiki.net [04:29:32] unfortunately we need a Resource Loader patch for RTL [04:29:49] or, I'll have to do something Very Clever [04:29:53] Not for the Indian languages [04:29:56] I know [04:30:10] that's a great pic [04:30:27] ooh, they're all pretty great. [04:30:28] can I have a screenshot with a thumbnail from you [04:30:42] they are some of the best of a competition [04:30:57] sure, but I'm going home now [04:31:00] tomorrow [04:31:11] :) have a nice evening then [04:31:47] I think I just fixed the issue with commons thumbnails [04:32:00] it's going to try, try again until it works or give up with a broken img. [04:32:10] so it should be good for me then ? [04:32:28] after wednesday push, which will be late in the day your time I think. [04:32:34] Where are you? Europe I think? [04:32:38] Europe [04:32:47] it is early in my morning [04:32:53] 06:40 [04:33:17] germany? [04:33:48] ah, netherlands [04:33:50] nm [04:33:57] :) [04:34:11] same timezone [04:35:16] europeans have that crazy one timezone, multiple countries thing [04:35:27] on this continent we do single countries, multiple timezones [04:35:48] ok, good night,ttyl [04:41:00] morning guillom [04:41:12] moin moin [04:44:35] can you provide me with a screenshot of the uploadwizard with a good thumbnail? [04:44:48] hi guillom [04:46:10] GerardM-, we have that wonderful free media library called Wikimedia Commons :) http://commons.wikimedia.org/wiki/Category:UploadWizard-0.1 [04:46:13] hey andrew [04:48:05] guillom NICE [05:07:55] morning [05:49:09] guillom http://ultimategerardm.blogspot.com/2011/05/commons-has-uploadwizard-ii.html [05:49:22] thanks [17:09:03] RoanKattouw: TrevorParscal: alolita howief awjr http://eiximenis.wikimedia.org/ArticleFeedbackV3 x2003 [17:09:13] Yeah, coming [17:29:56] RoanKattouw: do you remember the ResourceLoader 10+ sec response issue we had once upon a time [17:30:13] we figured it out eventually, that it was something with a 10 second timeout [17:30:21] that was always failing [17:30:31] do you recall what was timing out? [17:30:32] I... don't [17:30:38] There's 0 open bugs/feature requests for ArticleFeedback and any of the extensions it depends on [17:30:40] yay [17:30:55] Krinkle: shhh... people might start filing them again [17:31:08] You guys obviously overdid quality [17:31:11] most notable changes: Page titles linked on the dashboard; Not loading it on inexisting pages; Configurable namespace(s) [17:31:16] I don't know what in the world would have a 10s timeout in RL [17:31:33] it was like memcache, or something [17:31:53] *TrevorParscal tries to knock something loose in RoanKattouw's brain [17:31:54] Oh, yeah there was something about memcached timeouts for parser cache [17:31:59] yeah [17:32:47] I don't remember any such issue in RL though [18:34:35] mediawiki:Tagline is not working in fawiki , any possible solution ? [18:48:25] http://www.boingboing.net/2011/05/02/who-is-osama-bin-lad.html [19:52:56] neilk_ the localisation status for the UploadWizard can be found here .. https://translatewiki.net/wiki/User:Multichill/stats [20:07:07] GerardM-, I'd created https://translatewiki.net/wiki/User:Reedy/UW fof neik before :P [20:33:53] RoanKattouw: got a sec to chat about database stuff for aft? [20:34:07] Sure [20:34:14] neat [20:34:54] ok, so i'd like to put an index on aa_timestamp in the article_feedback table, and potentially on the aa_page_id field as well [20:35:00] do you see any problems with this? [20:35:58] Well [20:36:13] If you have a query on fields A, B and C you can't just create an index on A, one on B and one on C and be done with it [20:36:21] You have to create a combined index in the right order [20:36:26] aa_timestamp should be first for sure [20:36:40] But I'm not sure what the order has to be for a query where the GROUP BY and the ORDER BY are different [20:36:41] well im not positive we really need an index on aa_page_id [20:36:51] but yes, aa_timestamp definitely [20:37:05] I think it should be something like (aa_timestamp, aa_pageid, aa_rating_value) [20:37:08] i was thinking about getting rid of the order by completely [20:37:19] GerardM-: thanks [20:37:22] The query was WHERE aa_timestamp > N GROUP BY aa_pageid ORDER BY aa_rating_value DESC , right? [20:37:32] GerardM-: why is UW in those stats, is that a config thing [20:37:32] Oh hmm [20:37:36] The DESC messes with stuff anyway [20:37:42] yes [20:37:45] awjr: It would be great if that ORDER BY could die [20:37:52] That would make stuff easier [20:38:13] we'd have to get rid of the limit then as well and then do the rest of the calculations in php [20:39:00] if we go that route we almost may as well get rid of the group by as well, and select all of the records form the last 24 hours and do the stats calculations in php [20:39:25] Hmm [20:39:32] Let me see how many rows we're talking about then [20:39:59] I'm kind of a sad panda because this thing is inherently unscalable [20:40:00] and remember we'll be scaling up pretty quick [20:40:09] :( [20:40:13] It scales linearly instead of logarithmically [20:40:34] yeah [20:40:41] neikl_ it is a place where several things I am interested in are combined [20:42:04] one advantage, it is updated daily [20:43:02] awjr: Hmm the last 24 hrs only saw 5772 ratings [20:43:09] So for now the rating volume is tiny [20:43:21] and we are dsipalying on 3k articles? [20:43:35] I don't know [20:43:48] Anyway [20:44:13] It's unfortunate we can't easily do these stats on the toolserver [20:44:16] well, assuming we are and we go up to all 3.6 million... [20:44:24] Yeah well [20:44:32] If the script takes forever that's fine [20:44:46] yeah, that's why im wondering if it will be saner to just handle the stats in php [20:44:56] Probably [20:45:02] Minimize the load on the slave [20:45:08] You'll still need the index on aa_timestamp [20:45:11] select all the ratings in the last 24 hours, process in php cuz who cares [20:45:11] yes [20:45:16] Aye [20:45:17] ... how about doing this for the other languages ? [20:45:39] If the # of ratings in 24 hrs is insane we'll have other problems before this goes wrong [20:45:45] Like actually writing that volume [20:45:46] hehehehe that's very true. [20:45:54] I don't think we can handle a million ratings per day on the write side anyway [20:46:12] GerardM-: Were you talking to us? [20:46:33] yes [20:46:46] We only have ArticleFeedback on enwiki too [20:47:01] Expanding it to other wikis is an interesting proposal [20:47:04] why ?? [20:47:13] But it should probably be suggested to alolita or Howie [20:47:21] "We just work here" ;) [20:47:42] sounds like Befehl ist Befehl [20:47:49] But you're right if we wanna go to all enwiki articles why not go to other wikis as well [20:47:55] Well, to some degree [20:48:00] I know [20:48:01] hmm if we maintain the same write-rate for 3k articles, we'd be looking at what, 7.5million ratings/day? [20:48:08] for all 3.6 million articles, that is [20:48:08] you are not in court [20:48:08] The other half of it is "if it's not technical I don't care" [20:48:21] There's no way we'd get the same write-rate [20:48:26] [20:48:33] The articles we're running it on now are all substantial and popular and what not [20:48:44] ah i see [20:49:04] Of those millions of articles, there's gonna be lots like [[Eleven Cities Cycling Tour]] [20:49:15] yeah, for sure [20:49:17] You and I are probably the only ones that read it this week [20:49:29] Or, you know, even more obscure stuff [20:50:02] RoanKattouw is it an extension ??? [20:50:05] Yes [20:50:17] ArticleFeedback ? [20:50:21] Yes [20:50:24] in that case a better way to guess the write-rate would be to determine the % of people who rate compared to page views [20:50:25] thanks [20:50:31] awjr: Good point [20:50:41] We should ask Dario for those numbers [20:50:51] do we have actual page view stats tho? [20:50:56] Of course we do [20:51:00] stats.grok.se [20:51:01] huzzah [20:51:17] The way it's collected is kind of a hack but whatever [20:51:21] udp2log? [20:51:32] Yes [20:51:36] heh yeah [20:51:38] Sort of [20:51:47] Squid logging over UDP is only 1:1000 [20:51:58] You'd have to have kickass hardware and networking to do 1:1 [20:52:02] yeah [20:52:09] fo real [20:52:28] What we do do is keep counters [20:52:45] Actually [20:52:50] where are the counters? [20:52:51] Oh wait, yeah [20:52:56] The counters are on dammit.lt [20:52:59] And stats.grok.se pulls from there [20:53:20] ahha [20:53:34] (I think) [20:53:46] I once dissected this completely for Pete [20:53:53] But I've kind of forgotten these things since [20:54:04] http://dom.as/2010/11/25/death-to-mituzas/ [20:54:27] someone got tired of the trolling i guess [20:55:27] so i presume dammit.lt is domas' server? just out of curiosity, do you know how the counters work? [20:55:30] I love the replies of various Mituzases [20:55:36] Sort of [20:56:29] Let me dig up my e-mail to Pete [20:57:26] oh, if it requires effort, don't worry about it, it was just idle curiosity :p [20:57:31] "Domas's tool processes all requests made to our servers, and looks at [20:57:31] all hits for URLs that look like [20:57:33] http://language.project.org/wiki/Pagename . It counts these requests [20:57:35] and produces two list files per hour: one with per-page counts and one [20:57:38] with per-project counts. These files are published at [20:57:38] http://dammit.lt/wikistats/ (dammit.lt is Domas's web site). The [20:57:40] per-page lists are gzipped because they're so large; even gzipped [20:57:41] they're between 50 and 80 MB each. The per-project lists are only 15 [20:57:45] to 20 KB and aren't gzipped." [20:57:45] Basically, it's a hook into the Squid logger that udp2log also hooks into [20:57:56] But it works differently [20:58:08] ah ok, makes sense [20:58:17] It sends a UDP packet (I think?) to some server process, which keeps counters [20:58:23] And spits out hourly reports [20:58:31] That are then rsynced or something to dammit.lt [20:58:45] The important part is that, although it's 1:1, it doesn't do any disk IO [20:59:16] groovy [20:59:22] that's pretty cool [21:00:28] (Or, you know, doesn't cause any disk I/O per request) [21:00:42] yeah [21:18:56] I remember someone saying time ago that the page view logging contained one hit per request now [21:21:22] Yes, it does [21:21:36] I was just saying that the architecture meant we don't actually do disk I/O per request [21:21:49] Because that would be insane and kill our disks [21:28:59] Brion Thank you !! [21:29:06] :D [21:29:25] are you happy when I liberally sprinkle it with illustrations ? [21:30:00] RoanKattouw: it's so much more complicated doing this outside of mysql :( [21:30:26] Dude, multidimensional arrays [21:30:34] I wrote a migration script for the AFT revisions table [21:30:46] It maps rows -> 4D array -> 3D array -> rows or something [21:31:22] interesting, i should take a look - populateAFRevisions? [21:31:30] Yeah that one [21:31:40] It was kinda mind-boggling, I wrote it at 1am [21:31:43] iparticularly as we scale to 100% im more concerend about memory [21:32:13] Wouldn't worry too much [21:32:20] If your script uses a couple hundred megs, who carse [21:32:28] oh that's legit? [21:32:33] well then [21:32:38] that makes life easier :) [21:32:54] hume has 8G raw [21:32:56] *ram [21:32:59] so no one will care if i have an array with a few million members? [21:33:01] GerardM-, pictures are always good. :D i didn't think of any specific ones offhand so pick what you like ;) [21:33:20] that just made this way easier [21:33:21] awjr: Well a few million is too much probably :) [21:33:27] doh [21:33:29] But a couple hundred K is fine [21:33:43] Besides we agreed we weren't talking millions earlier, right? [21:33:49] (Other things would melt earlier) [21:33:54] well we agreed we weren't talking 7.5 million [21:34:09] i asked dario to come up with an estimate of how many ratings we could see in a day [21:34:28] he's going to get back to me, but the timing is rough since i need to get this cranked out pretty soon and i dont think he'll get to it til EOD [21:34:47] then again i can write it assuming it'll just work and we can tweak it before monday if we need to [21:34:55] Yeah [21:35:10] But I mean srsly, look at edit volume [21:35:36] 164k in the past 24 hrs on enwiki [21:36:00] that is instructive [21:36:27] alright, i'll stop worrying [22:36:29] http://ultimategerardm.blogspot.com/2011/05/interview-with-brion.html [22:36:44] enjoy [22:37:32] \o/ [22:37:41] hah i like the cartoon GerardM- :D [22:38:10] :) [22:39:33] that magician should be added to pages explaining why adding magic words is evil [22:41:12] what does "sed jen video de mia prelego pri la rolo" mean? [22:54:35] TrevorParscal: hey is there a reason not to use the test 'foo' in bar ? does this run into ownProperty strangeness? [22:55:00] 'foo' in bar.... [22:56:08] um, well, run 'toString' in Object [22:56:26] it will say yes [22:58:57] TrevorParscal: right, I knew there was a reason not to [23:00:05] for ( key in bar ) { console.log( key ); } only outputs things other than the obvious contents if you extend Object.prototype [23:04:17] TrevorParscal: are you interested in web font technologies? [23:04:29] if so, have you seen this? http://sprungmarker.de/wp-content/uploads/webfont-services/ [23:04:57] i'm only dimly aware of how they could be used by us, but i know we're talking to some people at redhat(?) about it. figured you would know more. [23:05:29] it's hilarious to me that people are making money off of this... [23:06:29] it's about as easy to do as image scaling.. but then again, there's lots of services that host images too - maybe we should talk to redhat about that as well [23:06:47] <^demon> Maybe they can do our diffs too! [23:07:28] dude, outsourcing this stuff just makes sense! [23:08:26] for instance, why are we storing article text? cant someone else do that? It's kinda complicated! that's a bummer dude... [23:09:02] my understanding was that we were trying to get someone to opensource their webfont code so that we could use it ourselves. [23:09:10] we should akamize everything. [23:10:17] seriously: http://www.fontsquirrel.com/fontface/generator [23:11:02] wtf are these "services" doing? taking your money because you suck at using google? [23:13:02] <^demon> People've been doing that for years. [23:13:19] <^demon> In fact, the entire seo "profession" was built on that. [23:13:40] It's the old AOL model... "You are so dumb, you will pay me $20 a month to stay ignorant!" [23:18:43] TrevorParscal: I sent you a bunch of stuff to review (ducks) [23:19:01] ha ha [23:19:06] when do you need that by? [23:19:11] tomorrow 1pm [23:19:15] omg [23:20:15] ok, well I have 5 things on my todo list for articlefeedback, none of them trivial, and all of them due by friday, early enough to get reviewed, deployed to prototype, and tested - or I'm going to miss my release on Tuesday... [23:20:27] so - am I the only one who can do that review? [23:20:33] alolita: poke [23:20:54] i said n/p earlier, because that was before I had these 5 things to do [23:20:55] trevor: hi [23:21:14] neil needs help with UW review, but I'm pretty backed up with AFT stuff all of a sudden [23:21:21] is there anyone else that can help with that? [23:22:01] trevor: whom would you recommend [23:22:08] right I understand [23:22:14] well, how much of it is CSS? [23:22:19] none [23:22:23] all JS [23:22:43] who do we consider capable of reviewing JS [23:22:50] it could be Kaldari, Timo or Roan [23:23:02] neilk: who else is capable - timo could; ryan also could [23:23:07] Neil is also on that list, but it's his code :) [23:23:13] roan is probably asleep [23:23:18] yes, he is [23:23:20] also, he's doing two pushes tomorrow [23:23:22] of course neil could :-) but its his code [23:23:26] Timo is also a sleeping dutch person Krinkle ? [23:23:33] right [23:23:37] right [23:23:39] jan paul? [23:23:47] not for review [23:23:48] well, Roan has done speed-reviews before pushing, but it's not ideal. [23:23:49] jan paul could too [23:23:49] Unofficiallly [23:24:02] Unofficiallly sleeping [23:24:07] What's up [23:24:10] timo: can you help? [23:24:12] maybe I could call out beyond the WMF [23:24:13] Sure [23:24:20] Krinkle: we are in need of JS review [23:24:23] neil: you have timo [23:24:49] Krinkle: it's like 1:32am there... [23:24:56] <^demon|dinner> Why don't you ask in #mediawiki if anyone's got time? [23:25:02] yeah [23:25:07] arg... [23:25:11] how many revs are we talking? [23:25:12] I know, but I needed to repair some bot for the countervandalism network monitoring 600 small wikis. fixed now [23:25:22] 5 or 6 [23:25:24] revs [23:25:25] So, what needs to be reviewed that I can do or help with. [23:25:29] http://www.mediawiki.org/wiki/Special:Code/MediaWiki/tag/trevor [23:25:34] they are tagged as trevor right now [23:25:42] I can do some [23:25:51] you said "a bunch" so I was like, uh oh! [23:25:52] there's only one that's substantial. [23:25:58] Hm.. UploadWizard, I know the code base a fair bit, but I don't have a local install. [23:26:02] 87314 [23:26:05] i will take the substantial one first [23:26:06] the others are trivial [23:26:07] trevor: timo can help [23:26:15] timo can try and do the others [23:26:18] race! [23:26:19] :) [23:26:23] and languageStats.php is not even relevant [23:26:23] :-) [23:26:32] it's a script that isn't used in MediaWiki, just on the command line [23:26:40] :O http://www.mediawiki.org/w/index.php?title=Special:Code/MediaWiki/status/fixme&path=%2Ftrunk%2Fextensions%2FUploadWizard [23:27:04] Krinkle: kaldari is on it already [23:27:10] I can handle those [23:27:43] neilk_: what does this pubsub plugin give you that custom jquery events doesn't? [23:27:59] you know about bind and trigger being usable for arbitrary even names I assume... [23:28:00] TrevorParscal: it's 10x faster [23:28:05] interesting! [23:28:05] k, UploadWizard installed [23:28:08] trigger and bind are very slow [23:28:19] that's why lots of people write pubsub libs [23:28:26] love it [23:28:39] also, trigger and bind don't always work in older versions of jQuery / older browser [23:28:46] this seems more universal [23:28:49] and and [23:28:58] you don't need to bind to an element, or really, to anything. [23:29:04] you just register. [23:39:37] After browsing and clicking "Upload" button in the wizard I get two exceptions in the console [23:39:38] ok, that all looked sane to me [23:39:43] jquery.mwCoolCats.js TypeError: Result of expression 'cat.replace' [undefined] is not a function. [23:40:00] (r87314) that is [23:40:06] mw.UploadWizard.js TypeError: Result of expression '_this.details' [undefined] is not an object. [23:40:26] {3x} mw.Log.js mw.UploadWizardUpload::getThumbnail> thumbnail error or missing information [23:41:10] function _insertCat( cat, isHidden ) { cat.replace [23:42:32] _this.details.populate() [23:43:53] I'm getting errors like: false has no method "replace" [23:44:02] and Cannot call method 'populate' of undefined [23:44:22] ooh exciting [23:46:24] someone is calling _insertCat( false, false ) [23:47:30] "// Disable ResourceLoader support by default, it's currently broken \//$wgUploadWizardDisableResourceLoader = true;" [23:47:32] eh ? [23:50:10] I've reviewed a few revs. In the mean time the #countervandalism server is back up. Gonna go now [23:50:13] good luck [23:50:14] nn [23:57:53] *werdna prods alolita [23:58:23] somebody in physical proximity want to give her a nudge?