[00:25:14] Elsie, Reedy, TimStarling: Is something up w/ the world's BGP? I've been unable to trace routes for various IPv4 websites, like Toolserver [00:25:30] I know it's not my end b/c other IPv4 sites work fine. [00:25:41] How does that mean it's not your end? [00:25:46] And you're really asking teh wrong people [00:25:55] traceroute does not use BGP [00:26:04] seems like I can't find routes to Toolserver [00:26:30] do you mean that you can't access those sites? [00:26:44] can't ping, can't trace, can't access, nothing. [00:27:02] not a problem for Labs, only toolserver it seems [00:27:10] fwiw I can ping and traceroute Toolserver over IPv6 [00:27:26] (but my browser thinks the IPv6 is too broken and tries to connect to it over IPv4) [00:29:24] labs and the toolserver are at completely different locations [00:29:53] hence why I think it's a routing issue [00:30:39] is anyone else reporting this? [00:30:43] does 'telnet website 80' work? [00:30:49] or just traceroute? [00:31:03] WFM on my home connection [00:31:04] TimStarling: idk, hence why I'm asking around [00:31:09] Jasper_Deng: ^^ [00:31:38] domas: nope - really weird [00:31:50] maybe it is a firewall!?!? [00:31:55] well at least I eliminated the chance that Wikimedia's routes are working correctly and it's my ISP's routes that are at fault [00:32:09] obviously wikimedias routes drop constantly [00:32:11] is Mark going to Wikimania? [00:32:14] thats the first assumption I'd make [00:32:18] TimStarling: are you going to Wikimania? [00:32:21] no [00:32:23] domas: it works for most IPv4 sites, but others (those that seem to be in Europe, it seems) can't even have routes traced. [00:33:28] TimStarling: :( [00:33:31] well, I'm not going either [00:33:32] heee [00:34:09] He doesn't look to be on the lists [00:34:48] I dropped out of the perfect attendance club [00:35:18] domas: Angela is 37 weeks pregnant [00:35:26] TimStarling: wheee [00:35:34] so I have an even better excuse than Gdansk, when she was only 7 months or so [00:35:44] yes indeed [00:36:30] I have no excuses [00:36:53] apart from the fact that my care level is not high enough to cram myself into a plane for 30 hours [00:37:46] Jasper_Deng: can you ping 91.198.174.232 ? [00:37:47] TimStarling: fun hack on the clock! [00:38:04] TimStarling: yes [00:38:34] domas: yup [00:38:41] TimStarling: interestingly I can't even ping eqiad [00:38:44] on IPv4 [00:39:02] TimStarling: can't wait for you to hack power wheels for your kids to have 10x more power :) [00:39:15] Jasper_Deng: unless a lot of people are affected, I don't think I can justify waking up Mark for this [00:39:28] nah, it's probably not that bad [00:39:40] Jasper_Deng: what is your ip? [00:41:35] doesn't look like BGP issue [00:41:41] more like "personal firewall gone nuts" [00:42:00] I'll go reboot my router anyways, but my router's own ping function didn't work [00:42:57] well, assuming that I can reach deep into your network from both eqiad and pmtpa [00:43:00] and FB and what not [00:43:17] .. [00:44:34] do I have access to toolserver? [00:44:37] is there a looking class? [00:44:38] g [00:46:05] <--- I just used "GLB" when referring to Wikipedia's DNS balancing [00:46:18] retirement! [01:02:59] clonebot? [01:03:29] alright, Jasper_Deng's issue has been resolved [01:03:38] but let's wake up Mark anyway [01:04:32] domas: nah [01:04:35] it's purely on my side [01:04:38] don't bother him [01:04:42] WAS A JOKE [01:05:31] !joke is Heck, even I got that one... Hahaha... -_- [01:05:31] Key was added [01:26:32] domas: I thought you were presenting this year at Wikimania. ;-) [01:26:40] About how self-important the movement is. [01:26:57] The children in Africa aren't going to save themselves. [01:28:13] heh [01:28:58] Who says the children of Africa need to be saved anyway? [01:29:22] Jimmy Wales and the Wikimedia Foundation, of course. [01:29:29] They're in the Global South®. [01:29:36] What about the children of Detroit? [01:29:50] they have US passports [01:29:52] Children in the Global North are problem-free. [01:30:06] is Australia in Global South? [01:30:52] Their Internet speeds certainly make it seem like it. [01:31:56] Oh, so the internet will fix Africa? [01:32:39] I guess it could be important for learning [01:32:46] Yep. Africa is sorely lacking in the three Ps: poker, pills, and porn. [01:32:57] The Internet offers all three, at extreme discounts! Act now. [01:33:10] Hi ctwoo. [01:33:38] so youu don't want poker, pills, or porn Elsie? [01:34:10] As a resident of the capital of the Global North, I'm all set. [01:34:38] What is the capital of global north? [01:34:55] Washington, D.C. (though some would say New York City). [01:35:15] I have never seen it on a paper map called "global north" [01:35:42] you're missing out [01:35:52] it is off latitude and longitude for true north [01:36:01] true north™ [01:36:12] I guess africa won't mind then [01:36:37] north is west of africa [01:37:17] the capital of Global North is obviously Reykjavik [01:37:56] Liquid magma is the capital of Reykjavik [01:38:32] see I could help teach africa too [01:38:44] we can save them [01:39:04] bgp issue was a pebkac? [01:39:09] or anything I can look at? [01:39:31] domas: ^ [01:39:46] user issue [01:39:49] k [01:40:19] paravoid: I think Rob H. was looking for a review of the SPF changes, though. There some confusion about what to include/exclude between the SPF record for wikimedia.org and lists.wm.o. [01:41:03] sorry, SPF doesn't pass the 5am threshold :) [01:41:13] :D [01:42:36] damn it smells like burned trees so much that my throat is itchy [01:47:57] so is everyone using AT&T unix here to run you own wikipedia? [01:49:23] what [01:49:30] AT&T developed Unix. [01:49:40] Well, Bell Labs or something. [01:50:21] my manual says "...appeared in Version 7 AT&T UNIX" [01:50:48] Sorry but what does this have to do with wikipedia? [01:51:34] iCandy is just casually asking if anyone replicates Wikipedia. [01:51:35] I think. [01:51:42] I don't. [01:51:48] I don't know what does it have to do with wikipedia? [01:53:10] You seem to know where global north is? can you help me setup the pictures from wikipedia? [01:54:10] the archive took forever to download [01:56:39] is there a faster way to get wikipedia with pictures from the archive?? [01:56:56] No. [01:57:14] oh [07:06:34] hey brion [07:06:40] heyyyy [07:06:56] in japan waiting for our final plane [07:07:47] how's the weather? yuvi-approved? :) [07:08:12] brion: where are you? [07:08:22] I'm in the lounge [07:08:29] Ryan_Lane: at the gate [07:08:31] 35 [07:08:37] ah. mine is at 53 [07:09:47] brion: indeed [07:09:58] brion: feels rather very familiar :) [07:10:00] hehe [07:14:11] brion: they maxed out my credit card though. so grr. need to wait till the evening before that is lifted. [07:14:19] fuuuuuuu [07:14:22] that sucks :( [07:14:36] brion: if they ask you for a card and you don't want to give them one, you're apparently supposed to poke leslie harms [07:14:37] just a fyi :) [07:14:45] heh [07:42:20] ugh github's down [07:42:48] work in gerrit :-P [07:42:57] :D [07:43:21] aren't you on a plane? or are you already in Hong Kong? [07:45:17] brion: gerrit worked fine for me :) [07:45:55] yuvipanda has this magical bot [16:22:09] apergos, parent5446: hello [16:22:15] hello [16:22:16] Hey [16:23:07] so, populateRevisionSha1 didn't work right away, i had to fix that https://gerrit.wikimedia.org/r/#/c/77903/ [16:23:29] parent5446: I won't be here Friday for the meeting, and it's possible I won't make it Monday (but you guys can ping me on Monday to see if I answer). So you guys should feel free to carry on without me and I'll read it in the logs. [16:24:12] OK [16:24:25] ok, you should probably add someone as a reviewer on that and gently nag them so it gets merged or they ell you what they prefer [16:25:13] I know for a fact that ^demon|away is not at wikimania, he might at least be able to point o a good reviewer (he's in pacific time zone though) [16:25:15] I'm not sure that's the proper way to fix that. You should call $rev->loadText() [16:25:30] parent5446: that's protected [16:25:47] Yeah I mean that the getSerializedData() function should be changed to call loadText(). [16:25:59] You'd think that'd be expected. [16:27:00] you would think [16:27:08] right, that was one possibility i thought of, but i wasn't sure about modifying a function in Revision that i don't know much about [16:27:13] there are a few other maint scripts that are going to be broken in the same way if that's the case [16:27:47] I think you can just go ahead and change the Revision::getSerializedData() function itself. I cannot think of any possible reason for why it would return unloaded data. [16:28:03] And if there is, the developer who implemented it should have left a comment. [16:28:17] that seems to be called almosst nowhere (though I haven't checked extensions yet) [16:28:32] so far two maintenance scripts and that's it [16:28:51] ok, i'll do that [16:29:52] so, after running all the populate scripts, there are still some differences between the dumps: [16:30:20] but they are all caused by not running it from the actual server [16:31:00] what sort of things? [16:31:14] e.g. is localhost instead of ten.wikipedia.org [16:31:37] or is incorrectly capitalized, because i don't have interwikis set up [16:32:25] so what should be is [16:32:59] also, a few redirectes seem to be imported incorrectly, but they don't work on the actual wiki either [16:33:17] so it looks like my code works correctly [16:33:22] great [16:34:03] now i'm going to verify that it works on Linux, so i'm setting up MW there and importing it [16:34:31] I would volunteer to test it for you but you'll want to test a pile more things so it makes sense [16:36:02] yeah, i think this is something i should do, especially since i sometimes get errors under gcc that are fine under VS [16:36:09] k [16:36:33] Pro-tip: always compile with -Wall ;) [16:37:03] oh, that's not the default? [16:37:26] I'm afraid not (usually). [16:38:23] thanks, then i will enable that [16:38:34] aprgos: also, i tried mwxml2sql, but it seems to produce nonsensical SQL [16:39:29] hmm not for me, but you should not spend time being a guineau pig for that until you have actual free time (and interest) [16:41:12] yeah, i didn't look into it further, since i have a setup that works now (though it's quite complicated) [16:42:04] at some point in the not too distant future if you want to put the dump you used and the exact command line you tried, I can look into it [16:42:14] put them somewhere, I mean :-D [16:43:18] well, i can tell you that right now: ./mwxml2sql -m 1.21 -s ~/tenwiki-20130622-stub-meta-history.xml -t ~/tenwiki-20130622-pages-meta-history.xml > output.sql [16:44:14] are the xml files from our server? [16:44:19] yes [16:44:21] sweet [16:44:25] ok I will check it out [16:44:26] thanks! [16:44:35] yw [16:48:55] I guess you are using the two pass approach for this btw? [16:51:45] kind of, i am using dumpBackup to create a full history dump and then fetchText to get the text of revisions; but it's not really 2-pass, it's both done at the same time [16:52:11] oh, I mean as far as dealing with deleted/etc stuff [16:54:52] right, i haven't worked on updating the dump after some changes yet, that's next; that means that deleted pages don't work yet [16:55:03] ok [16:55:39] (i will need to set up two different versions of the same wiki for that) [16:57:24] one more thing: i would like to vist a lecture on thursday, which would mean i wouldn't make the meeting at the usual time; how does 15:00 CEST (16:00 Greek time, 9:00 Staten Island time) sound? i assume you can't make it then, parent5446 , right? [16:57:29] I dunno why... just mysql dump the one, then hide / delete/ restore / move some stuff and save a mysqldump of that too, restore whichever one you need (for testing at least) [16:57:42] 4pm is fine for me [16:58:11] Yeah I can't make it [16:58:37] next year you need to mve to europe [16:58:52] fix the timezone issue [16:59:00] ;) [16:59:58] hmm, that could work; i wanted to test it on real data, i guess there is not much reason for that [17:00:23] well another option you have is to take an earlier stub for which we know a few pages have been deleted/moved since then [17:00:54] and use that to drive the history dump [17:01:08] where the history will pull from your installation of the wiki as it is now [17:02:13] except then i wouldn't have the revisions that are new in that stub [17:02:23] no, you won't [17:02:32] you won't get a full dump of the current wiki, but that's ok [17:02:44] right [17:04:38] i think that's it for today, see you tomorrow at the usual time [17:04:59] See you tomorrow [17:05:02] see ya [18:25:55] who do I need to poke to look at https://bugzilla.wikimedia.org/show_bug.cgi?id=50948 [18:36:23] Betacommand: it's probably a function of lua templates/modules, I'm guessing (from the title). I'll look more closely [18:36:53] Betacommand: oh, is John of Reading correct? do you only see it missing the "fair use" images? [18:37:30] Betacommand: I'll reply to that bug. [20:02:00] greg-g: thats where I noticed it [20:03:02] Betacommand: as in "only with fair use images"? [20:06:15] yeah, I didnt notice it with free images [20:06:29] Betacommand: then that is probably some feature of the collections extension [20:06:58] greg-g: its not a feature [20:07:05] its a pain in the ass [20:07:17] one man's feature is another man's bug [20:11:12] Betacommand: well, it's obviously intentional. Feel free to find the offending code and submit a patch (by removing it, probably). [20:11:42] greg-g: I dont know php [20:12:10] greg-g: there should be at least a way to override the removals [20:12:15] do you know any programming language? It's probably an if statement that checks for the images' category [20:12:45] Betacommand: maybe, yeah, that'd be a new feature though. It'd probably be a site-wide setting (and I'm unsure if WMF wikis would enable that) [20:13:12] greg-g: why wouldnt they? [20:14:30] well, as this takes more code to do what it is doing now than allowing all images through, it was obviously a conscious choice, I don't know the history, but it was done for a reason (probably). [20:18:35] does the resulting pdf have license information in it for the images? [20:20:38] me neither [20:20:43] wrong window [20:20:46] :) [20:20:57] " Nemo_bis doesn't have license info? huh" [20:21:05] weird huh [20:23:43] so that was an actual question, I was going to speculate about reasons for leaving out fair use images but only after finding out the answer [20:23:53] Betacommand: https://en.wikipedia.org/wiki/Category:Exclude_in_print and https://en.wikipedia.org/wiki/MediaWiki:PDF_Template_Blacklist are relevant [20:24:18] apergos: it does not, like it never existed [20:24:24] * greg-g just tested after you asked [20:24:32] no I mean of the images that *are* included [20:24:38] oh [20:25:37] * greg-g tests again [20:28:44] apergos: it does include license info for included images [20:28:59] huh [20:29:21] well if it puts that in there I don't see why the fair use ones can't go in then [20:29:35] eg: File:Djaoeh Di Mata ad.jpg Source: https://en.wikipedia.org/w/index.php?title=File:Djaoeh_Di_Mata_ad.jpg License: Public Domain Contributors: Uncredited [20:30:13] over cautious choice, probably, based on the categories/templates excluded I linkd above [20:31:22] * greg-g goes to get food [20:51:19] apergos: those PDFs are meant to be transformed in books and printed by PediaPress, that's not fair use [20:51:38] it makes sense to exclude unfree files, you can never know when it's acceptable to use them [20:51:58] if someone wants to generate a pdf and put it on their computer, that's fair use [20:52:09] in theory us serving them to millions of readers is fair use too [20:52:22] it may be fair use or not :) [20:52:34] sometimes personal use is less allowed than educational use on a website [20:52:44] anyway, Collection is made by and for pediapress [20:52:59] it's complicate enough as it is to use [20:53:32] people sell printed works that contain fair use material all the time [20:54:13] sure but they have to invest a lot of time in ensuring they can [20:54:46] fair use stuff is always a cost [20:54:47] if this is them taking a short cut, I can understand it, but then they should just tag those pdfs as 'contains fair use material' and not print them [20:54:58] hmmm? [20:55:03] pretty sure that could be managed automatically [20:55:08] no [20:55:21] no? [20:55:25] absolutely not [20:55:31] we have all sorts of unfree materials [20:56:16] whatever unfree they know t exclude now, they could instead mark the pdf [20:56:39] that would make two options, , free and unfree pdf [20:56:53] no reason to make stuff even more complex IMHO [20:56:53] right. the free ones they could print, since that's apparently the concern [20:57:11] the user could not be sure to be able to print the unfree one either [20:57:15] the stuff that has nufree material they wouldn't print [20:57:18] with their own printer [20:57:36] the user can print a pdf right now from any given article [20:57:40] ^P and save as pdf [20:57:49] doesn't mean that it's allowed [20:57:53] and then print it to a printer [20:58:13] for personal use? [20:58:21] yep, sometimes forbidden [20:58:39] or at least some try to claim so [20:58:48] how is it ok to read our materials on the screen and not in hard copy? I don't think we should be encouraging people to make that claim [20:59:01] ture [20:59:09] that's why unfree material is highly discouraged [20:59:20] and also why it's very good to exclude it from PDfs [20:59:49] it's always a nightmare to handle our unfree files for offline usage [21:00:20] even worse if you have to explain it to a publisher [21:00:55] I think we should be fighting to expand fair use rather than giving up on it [21:01:18] (which is what the copyright industry wants of course; if fair use were to die the death tomorrow they would be overjoyed) [21:05:14] well, my chapter did so [21:05:23] but it's a perennial controversy [21:05:59] and there's no reason to make stuff less usable just for the sake of showing we can use fair use [21:12:26] we want it to be more usable [21:12:36] having all the images in it is more usable than having them missing [21:12:57] tbh it should be user-selectable [21:17:14] IMO we shouldn't be exposing our re-users to the nightmare of fair use at all [21:17:50] well that's all our users because they are potentially re-users [21:17:58] exactly [21:18:06] so you're saying, no fair use materials [21:18:20] I think that's giving up on fair use as a right, and that's a mistake [21:18:51] it's difficult. In some cases, non-free media that can't possibly be replaced by free media really help to understand a subject [21:19:17] in those cases I can understand the choice to make fair use of non free images [21:19:53] well if we're going to allow it then we're going to allow it [21:19:55] but book covers and stuff like that, I have no idea how an image of a book cover helps the understanding of any subject [21:20:31] if the question is whether some stuff is actually fair use or not, that is a question to be settled on the wiki with that community [21:20:40] it's certainly not a technical matter [21:20:43] oh, no, it's easily fair use [21:20:58] it's just that we shouldn't use fair use in most cases [21:23:39] well again that seems like a policy issue for the community and not a technical one [21:25:18] sounds like I should un-cc myself from tha tbug now then ;) [21:25:32] why's that? :-P [21:26:13] oh you know, been around these parts long enough to know the joys of copyright debates :) [21:27:36] well I'm not on whatever the pdf bug is so [21:27:41] I won't be commenting there :-D [21:29:44] haha, not a comment on you, just, the topic as a whole. I've done the copyright thing as a day job for so long I'm about burnt out debating about things [21:33:26] ah that will do it to you [22:22:36] gn8 folks