[02:42:14] Elsie: have you ever imported a WMF wiki dump into a local wiki? [02:44:15] mwalker: Probably. [02:44:18] mwalker: What's your question? [02:44:25] Pro-tip: use a small wiki. [02:44:35] The Beta Labs folks have done this repeatedly, I think. [02:44:36] how do I do it :) the documentation indicates that I should convert it to SQL first [02:44:40] As have whoever's running test2wiki, I think. [02:44:42] but all the tools seem very old [02:44:51] Are you using dumps.wm.o? [02:44:55] There are SQL dumps there. [02:45:13] * mwalker rummages some more [02:45:33] there are SQL dumps; but not it seems for page content; but I could be wrong [02:45:35] http://dumps.wikimedia.org/mediawikiwiki/20131102/ [02:45:54] We have public dumps of all content in XML and SQL format. [02:45:57] I believe. [02:46:04] If we don't, please let me know. [02:46:08] I know we're missing a few tables. [02:46:48] Elsie: xml only not sql? [02:46:53] I guess we don't dump the revision table publicly? [02:46:53] i think [02:47:05] Yeah, this may be dumber than I thought. [02:47:12] God I hate the XML dumps. [02:47:21] there's scripts to rebuild sql from xml [02:47:32] It looks like text and revision aren't dumped as SQL. [02:47:45] mwalker: I used maintenance/importDump.php [02:47:48] or somesuch [02:47:49] Publicly, anyway. [02:47:59] worked fine when importing all of testiwikidatawiki [02:48:06] Right. There's also mwdumper, I think. [02:48:07] just xmk [02:48:09] Meta-Wiki has docs. [02:48:23] yeah but for something small enough importDump is probably good enough [02:48:23] mwalker: We already have a bunch of test wikis, though... [02:48:25] What are you working on? [02:48:39] You probably should be using a test wiki, Labs, or vagrant. [02:48:45] But it's difficult to know without context. [02:49:12] I'm in labs :) I'm setting up a test wiki that I can debug and interact with some a new backend we're writing for the collections extension [02:49:31] Hmmm. [02:49:36] Will you touching mwlib? [02:49:40] Or just the MediaWiki side? [02:49:40] sadly [02:49:42] Heh. [02:49:55] I spent about 3 hours today getting that setup in labs [02:49:56] I thought you did fundraising tech. [02:50:02] not for the next two weeks [02:50:09] You're updating Collection? [02:50:12] *nods* [02:50:15] Isn't that maintained by some shady company? [02:50:16] PediaPress? [02:50:21] *nods* [02:50:29] What are you upgrading? [02:50:46] we're rewriting mwlib effectively [02:50:55] I think mwlib is currently its own mini-parser. [02:50:59] it is [02:51:00] I'd recommend... not doing that again. [02:51:10] nope; we're taking the RDF from the parsoid [02:51:17] Nice. [02:51:19] and then manipulating that [02:51:22] Who's we're? [02:51:36] anomie, maxsem, and cscott [02:51:53] Interesting line-up. [02:52:07] basically; it's whoever was free and crazy enough to try and do it [02:52:18] but we need to get some new solution before we shutdown the tampa datacentre [02:52:25] because we aren't going to move the current PDF boxes [02:52:30] I'm not really complaining about improving mwlib. [02:52:33] Just curious who's pushing for this. [02:52:38] Ah. [02:52:40] Erik; and Ops [02:52:49] There it is. Okay, that makes more sense. [02:52:55] Yeah, that's a maintenance headache. [02:53:03] Well, yay for mwlib getting better. [02:53:08] Perhaps it can move into Gerrit too. [02:53:13] It's currently only on GitHub, I think. [02:53:38] https://bugzilla.wikimedia.org/showdependencytree.cgi?id=31552&hide_resolved=1 [02:54:00] oh; that few [02:54:06] Well, [02:54:08] * mwalker is surprised [02:54:10] that's just a random tracking bug. [02:54:14] I'm sure there are more. [02:54:28] I figured for sure several hundred [02:54:28] Plus probably a bunch on GitHub, where the code is? [02:54:56] https://github.com/pediapress/mwlib/issues [02:54:58] Not terrible. [02:55:38] I vaguely remember RTL support being super-wonky. [02:55:44] I wonder if Parsoid will help with that. [02:55:54] yes it will [02:56:06] Nice. [02:56:07] we're doing minimal manipulation of the DOM; and then having PhantomJS render it [02:56:28] so; basically; whatever Parsiod and Webkit support; so will we now [02:56:31] Will it recognize Print.css? [02:56:43] maybe :P [02:56:51] (Does it currently?) [02:56:51] I can't remember how much it actually prints and how much it just half-assed parses. [02:57:12] it parses everything currently; but it does attempt to respect print media css commands [02:57:14] and so will we [02:57:44] Yeah... the whole idea of a PDF generation service always seemed bizarre to me. [02:57:51] Given the ability of any modern OS to do it. [02:57:58] But I guess it's somewhat popular. [02:58:04] 200,000 requests a day [02:58:15] with 100,000 downloads a day [02:58:30] Right... well, it's in its own sidebar section. [02:58:38] On a huge site. [02:58:51] I wonder what the actual breakdown is. That could be half bots, eh. [02:59:04] *shrugs* I didn't dig that deep [02:59:14] One day we'll have an analytics infrastructure. [02:59:18] One day more. [03:00:10] heh; I wrote my own crappy scripts to help with it https://git.wikimedia.org/tree/wikimedia%2Ffundraising%2Ftools/2f6f28d9a80524bdd32a784739acc08f7f8105ec/SquidRipper [03:00:56] Is fundraising still using Squid? [03:01:08] Everyone seems to be switching to Varnish, though I'm not sure anyone likes either. [03:01:14] no; but I originally wrote these back when we were still on squid [03:01:24] the site is now completely served by varnish [03:01:25] Ah. [03:01:39] but squids aren't repurposed yet [03:01:39] In varnish we trust, all others pay cache. [03:01:52] we *could* switch back [03:01:58] Elsie: That was pretty clever. [03:02:05] jeremyb: I want to throw a party when we finally decommission them [03:02:22] mwalker: i thought some are being converted to varnish or ssl or something [03:02:22] A tribute? [03:02:26] not decomm [03:02:31] Squid has served us well, I think. [03:02:45] jeremyb: probably true; but I'm not sure [03:03:08] How many requests did Squid serve at peak? Do we know? [03:03:18] I think it was... a lot. [03:03:29] mwalker: at least in ams. tampa's maybe more likely to truly decom [03:04:03] Elsie: errr, more than varnish probably? michael jackson and barack obama were in squid era [03:05:16] according to stats.wm.o somewhere > 8.2 B objects / day [03:05:31] though; that includes bits [03:05:34] so... *shrugs* [03:05:42] call it > 6 B? [03:06:00] We should salute Squid. [03:06:06] bits was never squid i thought? [03:07:04] mwalker: i wonder if we could get steve to paint the pdf boxes black [03:07:36] jeremyb: correct; bits has always been varnish [03:19:03] mwalker: hrmmmm, it's awfully close to december to loaning from FR :P [03:19:25] *nods* [03:19:36] but I'm one of the crazy people who's fiddled in collection before [03:27:20] http://shapecatcher.com/ [03:32:22] Is it wrong that I'm using ASCII characters as test cases [03:35:35] me too -- it's all I know :( [07:02:31] mwalker|away: what documentation are you using? [07:02:57] You should use https://meta.wikimedia.org/wiki/Data_dumps/Tools_for_importing and subpages [07:04:05] (please add there anything you discover; import is still a science for few alchemists, it needs it) [07:31:06] Argh, why is this edit not autopatrolled? Kghbln is in editor group which has autopatrolled permission. https://www.mediawiki.org/w/index.php?title=User_talk%3AAlterego&diff=805523&oldid=710507 [07:55:19] https://bugzilla.wikimedia.org/show_bug.cgi?id=56989 [08:52:29] erp [09:19:05] hm [09:19:13] such a n00b [09:19:22] no it's not [09:19:35] better so :) [09:20:26] I am seeing issues for enWS "Error: 503, Service Unavailable at Wed, 13 Nov 2013 09:19:46 GMT " is something underway? [09:20:48] sDrewth: AFAIK the 503 error spree is going on since a few days [09:21:08] is it? [09:21:22] I have seen occasional issues in saving big pages, but not just trying to call up a page [09:21:36] sDrewth: where are you located? [09:21:44] Oz [09:21:55] paravoid: according to https://gdash.wikimedia.org/dashboards/reqerror/ yes :) [09:22:09] sDrewth: Oz? [09:22:58] paravoid: Australia [09:23:02] Oz [09:23:58] we had some intermittent issues with the SF datacenter (again) and Australia is usually pointed there [09:24:12] but we switched DNS back to Ashburn, half an hour ago [09:25:57] we do still have some servers in Amsterdam right / [09:25:59] ? [09:26:41] GerardM-: we do [09:27:45] has there ever been a quantitative analysis for cost and benefit ? [09:28:55] I don't know. [09:28:56] GerardM-: that's a good question to ask, but it's probably not the best time to ask it [09:28:59] if we had a second site like Amsterdam, would cost go down for Amsterdam because it is replicated [09:29:18] we have 4 sites [09:29:27] ori-l: why not ? [09:29:42] two of them are caching centers, and one of them is being replaced in the next 6 months or so [09:29:44] Three in the USA right [09:29:55] yes [09:30:37] moving the second to India or Africa would serve us better I would guess [09:30:43] because people's minds are likely to be trained on the immediate issues rather than the big picture. but i'll just stand back since you're getting your answers anyway. [09:30:55] why india or africa? [09:31:10] both of them have really crappy network connectivity [09:31:20] and the datacenter market is really not advanced either [09:31:40] We had a discussion at the diversity conference in Berlin about traffic growth ... we see it comming from a new functionality that is on the horizon [09:31:51] I don't remember that [09:32:01] oh, diversity conference [09:32:02] it is local language search for Commons [09:32:03] no I wasn't there [09:32:12] I was [09:32:24] datacenter selection has nothing to do with diversity, I'm afraid [09:32:42] it's very market-oriented [09:32:50] when all kids in a country can find pictures in their language ... it will grow our traffic hugely [09:32:52] because it's all about good connectivity, quality of services and money [09:33:17] I doubt that very much [09:33:25] it has everything to do with the capacity of cables to Africa and Asia [09:33:29] because ? [09:33:42] because broadband penetration in Africa is miniscule [09:33:51] because internet penetration is pretty low too [09:34:11] you may have millions of people, but they're all going to come via super slow links [09:34:30] the internet backbones in Africa are really good.. it is connectivity to outside sources that sucks [09:34:46] so have the cache inside Africa [09:34:46] I don't think so [09:35:10] I spoke with a network architect of Orange / France Telecom about this [09:35:18] https://en.wikipedia.org/wiki/List_of_Internet_exchange_points_by_size [09:36:03] the only african entry shows 48 members, max throughput of 2gbps [09:36:36] and that's south africa, which typically is way above average compared to the rest of the continent [09:37:08] I spoke to the president of the SA chapter... local search of Commons will be hugely popular [09:37:18] I'm not doubting that [09:37:26] I'm doubting that it will make any difference traffic-wise [09:37:50] how much traffic is in a Wikipedia search compared with a Commons search? [09:37:55] (I'm not doubting if commons will be hugely popular because it's not my area and I really have no clue) [09:38:22] but if traffic from Africa becomes significant, I'm sure that an caching center there will be considered [09:38:38] ok ... I am asking about numbers of Amsterdam so that I can think on what it COULD do elsewhere [09:39:03] fwiw, a few months back I switched Africa & Middle East to Amsterdam [09:39:06] I was there at the start of the Amsterdam centre [09:39:17] and it didn't even appear in graphs [09:39:36] I introduced Kennisnet, the original donor of the Amsterdam centre to Jimmy Wales [09:39:41] it was just within the normal parameters [09:39:52] They did this because schools complained about our service [09:40:10] people complain about us in Africa and India but they are used to it [09:40:41] do they complain about us especially? [09:40:57] i.e. do other sites work alright, but we are especially problematic? [09:41:04] as the crow said "faaaaaark" [09:41:10] or are people complaining about internet connectivity in general? [09:41:12] enWS gone again [09:41:25] sDrewth: really? [09:41:49] sDrewth: could you please do a "host en.wikipedia.org" (Linux/OS X) / "nslookup en.wikipedia.org" for me? [09:41:59] or en.wikisource [09:42:01] it doesn't matter really [09:42:31] we are a foreign site and these generally suck ... the googles et al have local services [09:42:37] GerardM-: no they don't [09:42:51] google has GGC, but in Africa is not used much [09:43:03] and google has no DC in africa as far as I know [09:44:05] Non-authoritative answer: [09:44:05] Name: wikipedia-lb.eqiad.wikimedia.org [09:44:05] Addresses: 2620:0:861:ed1a::1 [09:44:05] 208.80.154.225 [09:44:06] Aliases: en.wikipedia.org [09:44:08] wikipedia-lb.wikimedia.org [09:48:18] it is back [09:48:55] try restarting your browser [09:49:02] as crazy as that sounds :) [09:50:02] we have issues with the SF datacenter and we switched traffic to Ashburn; the output you sent me above shows that this is indeed the case for you too and your ISP isn't doing something silly with caching records beyond their intended time to live [09:50:26] paravoid: sorry was not aware you are in the middle of an issue [09:50:33] but browsers are fighting each other for performance and hence have become usually very aggressive in caching [09:51:07] which makes your browser prefer our SF datacenter, because that was the response we gave until almost an hour ago [09:52:12] back and functioning [09:52:45] so, SF is down again, but if you restarted your browser you shouldn't notice :) [15:22:10] hi [15:22:22] anomie, I have data [15:22:38] It's a Wikimedia server issue. [15:23:04] I'll email the files shortly. [15:23:07] You say that... [15:24:06] Reedy, ? [15:48:19] Reedy I guess it is LUA related [16:39:50] hi all. [16:39:53] couple of questions: [16:39:55] 1.- How can I conditionally link text in wiki markup depending on anonymous/registered users. Needed for linking or not to [[[[Special:EmailUser/Aerispixel]] [16:39:57] 2.- Can I rename my user myself? just a capitalization problem (now: Aerispixel, preferred: AerisPixel) [16:39:59] TIA [16:40:45] amontero: rename – yes, see: [16:40:48] !e Renameuser [16:40:48] https://www.mediawiki.org/wiki/Extension:Renameuser [16:40:58] Noting that this is also off tpic in here! [16:41:13] (ah, yes, that's assuming your own wiki :) ) [16:41:24] i sit in too many channels. [16:42:48] MatmaRex: thanks. Is this extension available and allowed to regular Wikimedia Commons users? [16:42:50] Reedy: Sorry. I tried it on #wikimedia after looking at the IRC channels wiki help page. did my best but, where would be appropiate? [16:42:56] #mediawiki [16:43:36] amontero: this channel is for WIkimedia-wikis-related questions, mostly. #mediawiki is for non-Wikimedia wikis, mostly. [16:43:55] amontero: on Wikimedia Commons you should ask a bureaucrat, there's probably some specific rename requests page. [16:44:08] Reedy: will shoot #1 there, thanks. but for #2, I imagine that it will depend on which extensions are configured/enabled. [16:44:31] MatmaRex: "bureaucrat"? ok, I'll try looking for that. [16:44:57] is it just me, or is page saving horribly slow right now? [16:45:02] trying to edit meta, currently [16:46:00] seems random - sometimes ist's fast, sometimes it takes more than a minute for a preview of a nearly empty page [17:36:06] Reedy: yay for archive population :) [17:37:45] Our servers are currently experiencing a technical problem. This is probably temporary and should be fixed soon. Please try again in a few minutes. [17:37:54] http://commons.wikimedia.org/w/index.php?title=Template:Technique/list&action=submit [17:40:01] Hi Laura1822 please ask it in here, already notified them Tech support on here :) [17:40:19] Hi, thanks! [17:40:42] I am getting an error when trying to save an edit to https://commons.wikimedia.org/w/index.php?title=Template:Technique/list&action=edit [17:40:54] Request: POST http://commons.wikimedia.org/w/index.php?title=Template:Technique/list&action=submit, from 208.80.154.9 via cp1065 frontend ([10.2.2.25]:80), Varnish XID 1132488710 [17:40:55] Forwarded for: 24.72.169.216, 208.80.154.9 [17:40:55] Error: 503, Service Unavailable at Wed, 13 Nov 2013 17:40:36 GMT [17:42:41] ugh [17:42:54] https://bugzilla.wikimedia.org/show_bug.cgi?id=18903 same bug? [17:42:59] Sorry, actual page url is page is https://commons.wikimedia.org/wiki/Template:Technique/list [17:43:00] just that or any other pages? [17:43:30] I don't know, haven't tried any other pages since getting this error maybe 20 minutes ago [17:43:32] Laura1822: Could you try other pages.. [17:43:39] sure, brb [17:45:47] later apergos :) [17:46:40] ok, I'll likely be gone but hopefully someone else can look, if it's a general problem [17:46:48] we haven't had any other reports today that I know of [17:47:14] I am having the same [17:47:25] issue that is.. [17:47:48] I successfully edited a File and the page https://commons.wikimedia.org/wiki/Template:Technique/en [17:49:31] Alright probably was a lag [17:49:33] I don't know if it's the same bug 18903 but I did see "Your cache administrator is nobody" on the error page. [17:49:57] Cool thanks Laura1822 :) Lets us know more if it occurs on here [17:50:18] gah 503's! [17:50:45] Explain [17:50:59] More of the same error? [17:51:13] No, I am still getting the error on https://commons.wikimedia.org/wiki/Template:Technique/list [17:51:40] Request: POST http://commons.wikimedia.org/w/index.php?title=Template:Technique/list&action=submit, from 208.80.154.9 via cp1066 frontend ([10.2.2.25]:80), Varnish XID 2006432294 [17:51:41] Forwarded for: 24.72.169.216, 208.80.154.9 [17:51:41] Error: 503, Service Unavailable at Wed, 13 Nov 2013 17:50:40 GMT [17:51:41] Your cache administrator is nobody. [17:51:51] Laura1822: Alright someone will look at it on here [17:52:10] on any other pages though? [17:52:29] or jsut this one page? [17:52:31] Thanks! No other pages so far, but will let you know if I find another. [17:52:48] Thanks apergos later :) [17:53:01] ok [17:57:45] Hello. [17:58:33] I've got a small question concerning the mobile site... Why is it mandatory to login to do a modification from the mobile site? Is it a feature? [18:00:38] anyone can give me a clue about why this image is not displayed in the Google Maps link inside the info box? [18:00:40] https://commons.wikimedia.org/wiki/File:Desembocadura_del_Ebro.jpg [18:00:42] AFAICT, I've followed https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Geographical_coordinates to geotag the file page, but this was some time ago and it still doesn't appear in any toolserver request [18:01:33] Thehelpfulone: could you help Laura1822 it's a bug I believe.. [18:21:42] Fae on the main channel suggested my bug might be related to https://bugzilla.wikimedia.org/show_bug.cgi?id=55219 [18:22:40] This is possible as it takes a long time after pressing "Save page" to get to the error page, like a timeout, and also the page I'm editing could probably be fairly described as "complex" [18:23:10] so perhaps there is a connection between the two bugs [18:26:35] also some discussion of it here: http://commons.wikimedia.org/wiki/Commons:VP#HTTP_Error_503:_Service_Unavailable [18:27:00] hi [18:27:16] i cant access any settings tabs [18:27:21] other than general [18:28:04] but only on test wiki' [18:41:00] kaldari, is the ui hack day in 31 or do we just stay where we are? [18:41:02] what's the main channel [18:46:47] When I said "main channel" I was referring to #wikimedia-commons [18:46:49] hi jgonera, i just sent you an e-mail. :) [18:47:08] MatmaRex, just replied ;) [18:47:32] oh, yay [18:47:33] thanks [18:47:45] the things this is blocking are: [18:47:53] * the other patch i linked there [18:48:06] ya'll stole the techops meeting room! [18:48:07] * the bug i created as a dependent one to the main one [18:48:13] * and the last comment on https://gerrit.wikimedia.org/r/94276 [18:48:37] (which i was meaning to convert to a bug, but got distracted and now there isn't much point) [18:48:49] (tl;dr; buttons and s need vertical-align: middle;) [19:08:05] ori-l, am I doing something stupid or views are no longer under Views tab in ganglia? [19:17:33] you're doing something stupid [19:23:45] greg-g, btw, do you work on the Wikipedia servers? [19:24:24] Cyberpower678: I don't touch the servers, that's ops ;) [19:24:44] greg-g, the problem seems to be a server issue. [19:24:47] <^d> We have Wikipedia servers? [19:24:47] I work for the Wikimedia Foundation: https://www.mediawiki.org/wiki/User:Greg_%28WMF%29 [19:24:52] ^d: :) [19:25:08] Cyberpower678: did you reply to the mailing list thread? I'm not the person you should be talking to here [19:26:07] greg-g, I sent Coren an email and asked him to forward it to the appropriate staff. [19:26:19] is there a reason it is not on the mailing list? [19:27:40] It's got attachments and I didn't know how it would affect the mailing list. The attachments are fairly large. [19:27:59] I don't want to blow up everybody's inbox. [19:28:14] I am still unable to edit/save the page https://commons.wikimedia.org/wiki/Template:Technique/list [19:28:27] after a wait long enough to be a timeout, I get [19:28:33] Cyberpower678: It smells like it's ulsfo caching issues; have you tried it in the past couple of hours? [19:28:41] Request: POST http://commons.wikimedia.org/w/index.php?title=Template:Technique/list&action=submit, from 208.80.154.9 via cp1052 frontend ([10.2.2.25]:80), Varnish XID 452601987 [19:28:41] Forwarded for: 24.72.169.216, 208.80.154.9 [19:28:41] Error: 503, Service Unavailable at Wed, 13 Nov 2013 19:28:01 GMT [19:28:41] Your cache administrator is nobody. [19:29:04] Haven't found the error on any other pages yet. [19:29:19] Laura1822, is spitting out the exact same problem my bots are having. [19:29:34] How does that equate to having large attachments [19:29:35] ? [19:30:05] We're in an ops meeting, but the right person to ask (in about 30m) is paravoid [19:30:50] Coren, greg-g: http://pastebin.com/uprp4Y2X is what is being returned by the API, and needless to say, what is causing the unserialization errors. [19:30:56] 154.9 is ssl1003.wikimedia.org [19:31:10] Cyberpower678, is that bug 18903 or bug 55219? [19:31:52] dunno [19:32:09] Do you have a bug report for your error on bugzilla? [19:32:19] I don't, myself [19:32:37] I haven't gotten around to filing one yet. I have Bugzilla open though. [19:32:51] At least, not that I know of. Those are the bug numbers that helpful people have suggested to me. [19:33:42] I don't have an account for bugzilla, and don't know if there are any other reports of this bug. [19:34:43] ori-l, not really, there must have been some error in JS that is now gone that prevented the jquery.ui tabbed thing to work [19:35:05] ori-l, views were still there but styled as regular radioboxes and inline so that's why I missed them [19:35:20] ah [19:37:10] Laura1822, Coren, greg-g, paravoid: 57026 [19:37:17] on bugzilla [19:37:28] Cyberpower678: so, hi there, I don't need to be pinged on this issue anymore, thanks. [19:37:50] greg-g, I have unpinged you. :p [19:38:06] Cyberpower678: thankyou sir ;) [19:38:29] greg-g, no problem. Glad I can be of service to you, [19:38:35] :p [19:48:05] Thanks, Cyberpower678! [20:01:38] where is MzMcBribe when I need him? :( [20:02:27] * Reedy kicks Elsie [20:04:26] Elsie: http://lists.wikimedia.org/pipermail/wikitech-l/2013-November/073077.html :D [20:04:41] Elsie: tldr: wmf has a contract for a technical writer. [20:07:09] hey hashar! [20:07:17] good morning [20:07:29] o_O [20:07:43] where are you, Hawaii? [20:08:37] DanielK_WMDE: ugt [20:08:49] bora bora right now [20:08:54] relocated last week [20:09:02] :P [20:09:23] still waiting for the Satellite internet connection then I will sail in the pacific ocean [20:10:45] :D [20:10:57] trying to relax by triaging the thousands of mails [22:41:19] MatmaRex: fyi: Project:RENAME is usually a redirect to the right page on most WMF wikis. [22:42:02] okay, finished scrollback [22:52:40] true, I created many of those back in 2008 :) [22:52:51] or was it before SUL [22:52:53]