[02:39:13] Snowolf_: you're a steward right? [02:39:50] sorry, wrong room [05:54:14] hello [05:55:07] wikipedia isn't in HTTPS by default ? [05:55:46] for logged in users [05:56:09] ori-l why not for all ? :P [05:56:31] https://blog.wikimedia.org/2013/08/01/future-https-wikimedia-projects/ [05:56:38] it's complicated [05:57:14] SSL/TLS adds bandwidth overhead and is not available for all users [06:07:50] ori-l because it's slow ? [06:07:57] with SSL ? [06:09:09] Negotiating the SSL handshake adds at least 2 round trips between the client and the server and often 3 [06:09:39] cortexA9: it's not as simple as "it's slow"; it's complicated to roll out and the total impact on our users is unknown [06:09:43] Also there are countries that block access to Wikipedia (and other sites) over SSL [06:09:57] mmm [06:10:01] ok [06:10:53] But I can highly recommend https://www.eff.org/https-everywhere for those of us who can use it [06:11:11] yea i know thanks bd808 [06:13:25] another question: If i wanna make a mirror of the entire wikipedia on my own web server. It's possible ? [06:14:23] yup [06:14:36] let me find a link [06:14:41] legoktm but what about the web server. [06:14:48] you'd need to run that yourself. [06:15:39] hm, can't find the link. you can get dumps from http://dumps.wikimedia.org/ though [06:15:42] legoktm: the web server needs apache right ? [06:16:14] well, how are you planning to run the mirror? if its just for you locally you should use kiwix [06:17:04] legoktm: how many TB required ? [06:17:21] it depends, do you plan on also storing images? [06:17:35] legoktm: yea all. [06:17:37] there are also meta pages that aren't articles, but also part of wikipedia. [06:18:13] no clue, a lot I'm guessing. [06:18:18] i just curious. [06:18:20] hehe [06:18:28] *i am [06:20:02] so it's not possible legoktm to make a local mirror in my web server of wikipedia. [06:20:22] it is. it would just be difficult. [06:20:34] if you want a local mirror for your own personal use, look into kiwix [06:20:45] https://meta.wikimedia.org/wiki/Kiwix [06:21:15] yea legoktm but it's an offline reader. [06:23:29] legoktm: I mean reading wikipedia with a browser locally. [06:26:09] it's a special web server right legoktm ? [06:31:06] http://www.extremetech.com/computing/114387-how-to-set-up-your-own-copy-of-wikipedia [06:31:08] i found this [06:31:26] seems it's possible. [06:33:25] Its very possible, and you should talk to apergos [06:33:35] but it takes a lot of HDD space [06:33:58] even without images from memory [06:34:36] p858snake|l: mmm ok thanks :) [06:35:53] http://www.extremetech.com/computing/114387-how-to-set-up-your-own-copy-of-wikipedia/2 [06:35:56] i am interested in this part. [06:36:10] Install Apache, PHP, MySQL, and phpMyAdmin [06:38:12] so it's very possible :-) [06:38:20] there isn't really a need for phpmyadmin [06:39:20] p858snake|l: mmm idk.. [06:39:48] its just a gui for the mysql commandline [06:40:07] and i would be shocked if it didn't break at the size of the wmf databases [06:40:28] no you don't need phpmyadmin [06:40:30] good grief [06:40:35] hey apergos [06:40:37] :) [06:40:40] hey [06:40:59] do you need the historical revisions or only the current articles? [06:41:11] also do you want user pages and talk pages etc or can you skip those? [06:41:41] apergos: what is the minimum space i need ? [06:41:51] see my questions [06:42:05] apergos: just all articles. [06:42:14] apergos: in the EN. [06:42:21] and do you want media with those articles or not? [06:42:52] apergos: not necessarily. [06:43:36] apergos: u have a web server locally ? U have tested ? [06:44:11] I have imported subsets (I don't have the patience to do the whole thing, and I don't need it) [06:44:20] but I maintain the dumps and sme import tools [06:46:08] I have not tested this but I know some folks use it: http://www.nongnu.org/wp-mirror/ [06:47:23] I do of course have several local mw installations, as a dev/ops person [06:51:06] apergos: cool. [06:53:58] apergos: i am just curious, just in case wikipedia doesn't work :P [06:54:35] it's time for me to take another foray into setting up a subset, now that we have parsoid and wikidata to deal with [06:54:42] need to see how those impact local setups [07:01:27] another question: where can i see the development of mediawiki ? [07:01:59] cortexA9: #mediawiki-dev and gerrit.wikimedia.org [07:02:14] cortexA9: sorry, #wikimedia-dev and #mediawiki :p [07:02:35] thanks mutante [07:02:39] yw [07:56:45] [[Tech]]; TeleComNasSprVen; /* Thanking on Login Wiki */ new section; https://meta.wikimedia.org/w/index.php?diff=6772371&oldid=6743528&rcid=4752457 [08:05:45] [[Tech]]; Legoktm; /* Thanking on Login Wiki */ cmt; https://meta.wikimedia.org/w/index.php?diff=6772422&oldid=6772371&rcid=4752461 [09:12:36] I wonder if tidy is broken... https://it.wikipedia.org/wiki/Utente:Vale14orla [09:13:10] we already had that non-nowiki'ed breaking a mediawiki.org page, which is weird; this never was so common AFAIK [13:13:27] We have issues with File description pages not being created at Wikimedia Commons. [13:13:41] The files upload but the description pages are not created. [13:13:59] The incidence increased over the last past weeks [13:14:12] Can someone please have a look at https://bugzilla.wikimedia.org/show_bug.cgi?id=15430 [13:14:54] AaronSchulz ^^ [16:27:06] what PHP version did we upgrade from? [16:27:18] I also don't see anything on SAL about it :/ [16:28:00] huh? [16:28:15] in labs? [16:28:28] no, production [16:28:42] http://lists.wikimedia.org/pipermail/wikitech-l/2013-December/073645.html "There will be a new version of PHP deployed to the Wikimedia servers on Monday as well" [16:28:45] i saw an email something about precise [16:29:16] that was labs though I think. [16:29:46] oh, ok ... i see the mail [16:30:25] * aude wonders which version [16:30:40] special:Version says 5.3.10-1ubuntu3.8+wmf2 [16:30:50] I assume thats the new one, now I want to figure out what the old one was :P [16:30:57] seems so [16:31:10] that's the old one [16:31:24] i was supposed to update it, but i got a toothache instead and asked ops to help [16:31:35] i should have sent an update, sorry [16:31:47] legoktm: Should be fix for the exif bug and a couple of security updates [16:32:30] Reedy: https://bugzilla.wikimedia.org/show_bug.cgi?id=55541 ? [16:32:50] Yup [16:32:54] ori-l: so is there a scheduled date for when it will happen, or just sometime in the future? [16:33:26] shortly after MassMessage is fixed [16:33:36] :< [16:33:48] just trying to figure out what I should put in the signpost [16:33:56] :) [16:34:05] oh, I see. um, paravoid? [16:35:09] * aude happy to have technology report back [16:35:19] dunno then, it depends on his availability [16:35:40] ok, I just said "soon" [16:35:53] thanks [16:45:16] Reedy, AaronSchulz: https://bugzilla.wikimedia.org/show_bug.cgi?id=22390 was merged/deployed, is it fixed? still says patch to review [18:10:41] awjr, do we have a hangout for the overview? [19:47:52] manybubbles: does it.wikt or one of the others count as load-testing/stressing for the elasticsearch cluster, or are they negligible? [19:58:05] Nemo_bis: at this point the cluster isn't really under much load at all [19:58:24] manybubbles: ok, when will you start stressing it? [19:59:31] two things are holding us back: the total throughput of the job queue (probably temporary) and switching users to searching using it. [19:59:40] I really don't have enough people searching it [20:00:55] true :| [20:01:10] after chrismass I'll start switching more wikis over to it as primary [20:01:26] as per job queue, this is not related to cirrus is it? https://ganglia.wikimedia.org/latest/graph_all_periods.php?c=Miscellaneous%20pmtpa&h=hume.wikimedia.org&v=823574&m=Global_JobQueue_length&r=hour&z=default&jr=&js=&st=1365625056&z=large [20:01:44] oh, it went down a few millions O_o [20:02:22] hopefully, we'll see some load there. right now keeping the indexes up to date is much more load then actually searching [20:02:32] It probably is cirrus related [20:02:38] we now handle a good portion of the page updates [20:02:50] and I'm starting to notice us falling behind on our page update jobs [20:03:04] I've actually paused indexing commons while I investigate [20:09:48] part of that is that we build wikis by jaming a bunch of jobs in the queue and letting it drain [20:09:59] but we try not to jam quite so fast [20:11:54] marktraceur: ping [20:12:40] though I doubt that spike around 12/13 was us [20:12:50] so maybe that is why I think we're falling behind.... [20:18:53] marktraceur: do you have time to fix a "inconsistent state within the internal storage backends"? :P [20:25:02] manybubbles: those huge spikes happened at the same time as parsoid deployments [20:25:33] but I was wondering if maybe something cirrus-related not logged on SAL also happened [20:26:44] Steinsplitter: Uh...maybe not. [20:27:19] k [20:28:48] 99/12 [21:03:23] this gives a weird query error: https://en.wikipedia.org/w/index.php?namespace=118&tagfilter=&title=Special%3ARecentChanges [21:10:56] * Reedy blames Gloria [21:10:59] * Reedy waits [21:12:00] jackmcbarn: What error? [21:12:01] WFM? [21:13:21] Reedy: A database query error has occurred. This may indicate a bug in the software. Function: SpecialRecentChanges::doMainQuery Error: 0 [21:19:45] there in the logs.. [21:20:28] I wonder what error 0 is [21:26:34] Is it not just a timeout? [21:28:28] marktraceur: i've tried it 5 times, it runs a long time then fails with that error every time [21:30:25] marktraceur: Seems likely [22:14:08] Nemo_bis, parsoid deployments are pure JS and don't touch the PHP cluster at all [22:15:35] ah, job queue- IIRC that was commonswiki and wiktionary mainly, PHP refreshLinks jobs after some huge template editing spree [22:15:48] parsoid was not affected as not enabled on those wikis [22:15:56] gwicke: I only mentioned correlation, not causation [22:16:13] afaik you said so also last time parsoid added a few millions jobs in the queue ;) [22:16:17] but ok [22:18:14] we are tracking all edits, so will add the same number of jobs as refreshLinks [22:18:24] Nemo_bis: Parsoid didn't add the jobs. Parsoid removed the jobs. Editing added the jobs. :-) [22:18:31] we used to be very slow in dequeing them [22:19:38] we currently only track edits on wikis the VE is enabled on though, which is why we didn't need to process the commons/wiktionary edits [22:20:09] gwicke: We should probably start thinking about dealing with wiktionary and a few other sister projects soon, BTW. [22:21:13] James_F: before variant conversion? [22:21:26] the main blocker is LST [22:21:29] afaik [22:21:32] gwicke: Yeah, true. [22:21:50] gwicke: We really should chat at some point on our various blockers. [22:22:14] James_F: sure, just ping us whenever you have time [22:22:17] gwicke: E.g. for MetaWiki, or (in VE) editing for Commons. [22:22:19] * James_F will. [22:23:05] translate is being fixed by the i18n folks [22:23:15] and Inez has done work on gallery [22:23:38] LST looks more like a reimplementation though [22:28:10] gwicke: I'm not quite sure that's correct (about ) [22:28:43] Nemo_bis: Yeah. :-( [22:28:50] actually, I'm sure it's wrong [22:29:08] Nikerabbit> well, you can relay my message that I'm not fixing until someone helps with the core parser issues [22:31:41] well, at least he tried a bit [22:31:55] with some help we might get further [22:36:22] yep :) [22:44:11] /away food, checkout 'memoserv' if you dont know it yet, and feel free to use that for a reliable IRC ping in general [22:45:42] Hi, I'd like to make a research in deleted revisions texts, I can make it by API, but using the dumps would be much easer and faster, is there any way to I get access to this private dump? [22:49:27] Krinkle: Incidentally your wikipage for Intuition has the wrong include path :) [22:49:50] which wikipage? [22:52:19] jarry1250: Aye [22:52:27] https://github.com/Krinkle/TsIntuition/wiki/Documentation [22:52:36] jarry1250: What should which one be? [22:52:50] No /home ? [22:53:00] data rather than home on Labs [22:53:11] --> /data/project/... [22:53:44] Ah, right. [22:54:04] fixed [22:54:05] Thanks:) [22:54:34] Krinkle: np. I have a pull request as well (Github's feature seems very convenient) [22:54:55] Yep, saw it. Will look at it in a few hours, (unless siebrand beats me to it) [22:57:04] Krinkle: Sure. Incidentally, I thought http://tools.wmflabs.org/jarry-common/?uselang=nl was supposed to display the odd string I have translatable in Dutch [22:57:41] http://tools.wmflabs.org/jarry-common/?userlang=nl [22:57:47] use*r*lang, dont ask [22:58:02] Krinkle: Ah, thanks [22:58:06] (the bottom is now translated on that link) [22:59:54] Krinkle: Yup. I'll be adding a few more strings as well [23:00:47] danilo: What private dumps? [23:00:54] Oh.. deleted revisions.. [23:01:15] yes [23:02:33] I'm guessing the only way to get this data would be to contact the foundation and possibly have to sign an NDA or similar [23:03:00] danilo: I would be surprised if you can do it via the API... [23:03:08] unless you have the researcher right... [23:04:39] p858snake|l: the bureaucrats in my wiki (ptwiki) agreed in give a temporary flag to I make this research [23:05:08] … [23:08:04] it's not so uncommon [23:08:31] IIRC someone else was granted a global flag for this before, do you need it on pt.wiki only? [23:09:11] * Nemo_bis also wonders where https://www.mediawiki.org/wiki/Extension:BayesianFilter/GSoC_2013 went [23:14:21] danilo: ah, while you here: have you tried huggle 3.0? it also has change patrolling now, should help with pt.wiki's patrolling backlog https://github.com/huggle/huggle3-qt-lx#building [23:16:47] Nemo_bis: there is a bug, I can't install in openSuse, I have not used it yet [23:17:33] aww