[02:24:07] !log LocalisationUpdate completed (1.20wmf4) at Mon Jun 11 02:24:06 UTC 2012 [02:24:16] Logged the message, Master [07:31:10] urgh! http://seclists.org/oss-sec/2012/q2/493 [07:45:25] Daniel_WMDE: ? [07:46:29] Daniel_WMDE: I don't think we have any mysql which is remotely accessible [07:47:00] i hope so [07:48:40] I kinda lol'd [08:01:30] Damianz: yea. this is nearly as bad as debian's ssh key screwup some years back. [08:27:54] well, it is build specific [12:45:16] mark: around? [12:45:26] yes [12:46:09] mark: talked to our backbone guy again over the routing [12:46:21] it seems you announce the /48 in the US as well [12:46:29] is that mend to be? [12:47:43] no we don't [12:48:00] I we see the announcements via he.net [12:48:09] no you don't :) [12:49:40] hmmm [12:49:53] I did on friday, asking eric [12:51:21] nope, not on friday either [12:51:52] I said via he.net, might there be a peering from eu to he.net as well? [12:52:04] yes there is [12:52:23] ok [12:52:35] so they get the /48 via that peering [12:52:50] we get it in the US via them [12:53:07] more likely you get the /46 via them [12:53:12] and our internal routing is such that we prefer that one atm [12:53:13] both [12:53:18] ok [12:53:27] well then your internal routing needs fixing [12:53:57] can you add a community tag to your announcement, and do you mind doing that? [12:54:06] why would I do that? [12:54:18] because it helps us fixing [12:54:33] why do I need to do something while your network is broken? [12:55:10] well, what is broken, you know as well as I that all this routing can be quite tricky [12:55:30] I honestly don't think it's very hard [12:55:40] but if you think your proposal is reasonable, feel free to make a case for it in private [12:55:41] well [12:55:49] 'layer 8' [12:55:55] and I'll decide based on that [12:56:49] routing is easy, 'route add default 192.168.0.1' [15:13:40] !log Copying checkout of 1.20wmf5 onto NFS [15:13:45] Logged the message, Master [15:20:47] !log running sync-dir php-1.20wmf5 [15:20:52] Logged the message, Master [15:28:26] !log reedy synchronized php-1.20wmf5/ [15:28:30] Logged the message, Master [15:30:11] !log reedy rebuilt wikiversions.cdb and synchronized wikiversions files: test2wiki to 1.20wmf5 [15:30:16] Logged the message, Master [15:30:49] I know test2wiki is currently brokened [15:35:22] !log reedy Started syncing Wikimedia installation... : Rebuild messagecache for 1.20wmf5 [15:35:27] Logged the message, Master [15:54:20] !log reedy Finished syncing Wikimedia installation... : Rebuild messagecache for 1.20wmf5 [15:54:24] Logged the message, Master [15:55:10] !log reedy Started syncing Wikimedia installation... : Rebuild messagecache for 1.20wmf5 [15:55:15] Logged the message, Master [16:02:18] !log reedy Started syncing Wikimedia installation... : Rebuild messagecache for 1.20wmf5 [16:02:22] Logged the message, Master [16:17:47] !log reedy synchronized wmf-config/ExtensionMessages-1.20wmf5.php [16:17:52] Logged the message, Master [16:19:54] !log reedy Started syncing Wikimedia installation... : Rebuild messagecache for 1.20wmf5 [16:19:58] Logged the message, Master [16:53:13] !log reedy synchronized php-1.20wmf5/cache/l10n/ [16:53:17] Logged the message, Master [16:53:20] \o/ [16:58:23] !log reedy Started syncing Wikimedia installation... : Running scap to ensure consistency [16:58:28] Logged the message, Master [17:57:04] Why is scap not completing within an hour? :/ [17:58:17] RoanKattouw: how many checkouts do we need to keep around? [18:00:15] * robla doesn't see Roan....must be in a meeting or something [18:03:41] * Reedy tries scap again [18:03:51] !log reedy Started syncing Wikimedia installation... : Consistency [18:03:55] Logged the message, Master [18:16:51] Reedy: how's the scapping? [18:29:32] Reedy: how's the scap going? [18:29:49] It's scap numero somethingimnotsurenow [18:30:08] test2 is on 1.20wmf5 [18:36:11] Reedy: how long do you think it'll take to complete the 1.20wmf5 installation? [18:36:20] (so wikis can be switched to it) [18:36:34] That question doesn't really make any sense [18:36:41] How long to do what? [18:36:48] put the files in the right place [18:37:25] They've been there for a few hours [18:38:03] assuming that mw.org will be the first production wiki to get it, right? [18:38:08] yeah [18:38:15] test2 is already on it [18:38:19] testwiki also will be moved [18:38:32] https://www.mediawiki.org/wiki/MediaWiki_1.20/Roadmap [18:38:45] thanks for clarifying [18:40:48] I was hoping for a clean scap run to complete [18:41:01] 36 minutes or so now [18:41:02] what is a scap? [18:41:17] sync-common apache [18:41:37] ugh, that boggles my mind [18:41:37] it should mean all the apaches grab all of the needed files from the nfs server [18:41:46] oh the servers themselves [18:41:48] it also does some prep work, such as rebuilding localisation caches first [18:41:57] and then does some other config tasks [18:41:58] sounds pretty reasonable [18:42:13] I'm not the one managing hundreds of servers for the sixth-most-visited website on Earth, after all. [18:44:22] !log reedy synchronized php-1.20wmf5/ 'Scap is taking an age, just ensure deployment files are in sync' [18:44:27] Logged the message, Master [18:45:57] !log reedy synchronized php-1.20wmf2/cache/l10n/ 'Kill l10ncache for php-1.20wmf2 as its not needed' [18:46:02] Logged the message, Master [18:50:10] !log reedy synchronized php-1.20wmf3/cache/l10n/ 'Kill l10ncache for php-1.20wmf3 as its not needed' [18:50:16] Logged the message, Master [18:52:40] !log reedy synchronized php-1.20wmf5/ 'Now we have some more space...' [18:52:44] Logged the message, Master [18:54:49] Reedy: We need five [18:54:55] !log reedy rebuilt wikiversions.cdb and synchronized wikiversions files: testwiki to 1.20wmf5 also [18:55:00] Logged the message, Master [18:55:00] Reedy: I'll look into the l10nupdate-runs-for-old-versions issue [18:55:21] robla: that's test and test2 on 1.20wmf5 [18:55:54] I'm not sure what's up with scap, but that's another issue [18:56:36] * robla pestered RoanKattouw irl, and he's at his keyboard now [18:56:54] Yeah sorry for not /away-ing earlier [18:58:54] Reedy: Could you describe the issues you're having with cache updates running for old versions in more detail? Reading through the code it all appears to be using mwversionsinuse as it should [18:59:18] I'm not [18:59:31] I was wondering how many versions of php-X we need to keep the files around for at a time [18:59:40] ok, good [19:00:03] aaron@hume:~/common/multiversion$ ./activeMWVersions [19:00:05] 1.20wmf4 1.20wmf5 [19:00:21] I just killed the wmf2/wmf3 cache/l10n directories as dead files we didn't need. Even more so when we're low on disk space on some servers [19:16:51] uh, Reedy, the IPv6 rangeblock limit seems to be /19 [19:16:55] on testwiki [19:16:57] Indeed [19:17:14] https://gerrit.wikimedia.org/r/#/c/10856/ [19:20:35] !log reedy rebuilt wikiversions.cdb and synchronized wikiversions files: mediawikiwiki to 1.20wmf5 [19:20:40] Logged the message, Master [19:28:13] robla: all looks good to me [19:28:23] coolio [19:36:36] * YuviPanda wonders if he can coax anyone into looking at https://rt.wikimedia.org/Ticket/Display.html?id=2121 [19:42:53] I'm just actually replying to an email asking about that [19:44:03] Reedy: where? [19:44:10] on my desktop [19:44:16] of course [19:44:19] :p [19:44:28] have you a wm.o email? I'll cc you in on the reply [19:44:36] a desktop? dont you mean a floortop? [19:44:38] Reedy: yes. username@ [19:44:45] yuvipanda@wikimedia.org [19:44:50] Reedy: ^ [19:44:55] yup, ta [20:07:46] !log reedy synchronized php-1.20wmf5/ [20:07:51] Logged the message, Master [20:07:59] 1.20wmf5 already? [20:08:00] o.O [20:11:38] Reedy: I was AFK and my connection was reset so I didn't see the answer to why the limit is set at /19 [20:12:30] Jasper_Deng: kudos for writing up IPv6 stuff, great dox [20:12:49] saper: but the wikitech pages leave much to be desired [20:15:08] https://gerrit.wikimedia.org/r/#/c/10856/ [20:15:34] that's actually not really correct. AT&T only has a /24 [20:15:49] definitely much greater in magnitude than the IPv4 limit [20:20:00] if we want to block an entire ISP the IPv4 limit should also be upped to /8 [20:22:36] but /19 [20:22:51] is greater than most RIR allocations! [20:30:42] Jasper_Deng: Where did you put it? [20:30:57] put what? [20:31:07] 22:12:32] Jasper_Deng: kudos for writing up IPv6 stuff, great dox [20:31:44] mutichill: en:User:Jasper Deng/IPv6 and on MediaWiki.org [20:33:25] Cool. Did you also right something about the 6to4 stuff? That might be confusing too [20:39:05] not really [20:39:10] the WP article is pretty straightforward [20:39:25] Jasper_Deng: I made a short writeup for checkusers how SixXS works, maybe should add to it [20:39:33] Hi all. I need to ask for several posts to be removed from the archives of a mailing list that's hosted by the WMF - could anyone recommend who's best to contact to ask to do this? [20:39:51] saper: Sixxs uses regular ranges from ISPs so be aware of that [20:39:56] saper: where? and that would be greatly appreciated since no expert seems to want to write the WP article [20:40:12] multichill: luckily Sixxs has its own WHOIS [20:40:14] multichill: I am talking to you via SixXS now :) [20:40:34] Jasper_Deng: I can fwd to you it's just an email right now [20:40:35] They used to, I remember them moving to RIPE some years ago [20:40:48] it's all in RIPE now, replicated [20:40:52] you can whois me [20:41:07] saper: you have a cloak [20:41:13] * multichill just uses NCC1-RIPE [20:42:10] Jasper_Deng: let me fix that :) [20:43:52] I'm 2001:6a0:200:122::1 [20:44:11] saper: Me too, ipv6 is just broken on this laptop. I can still logon to SIXXS. "You currently have 1.669 ISK" (whatever that means) [20:44:40] multichill: check your routing table [20:45:06] multichill: I have only 600+ after 4 years [20:46:04] multichill: you need ISK to do things like request /48 [20:46:22] saper: it shows your specific /64 [20:46:24] in WHOIS [20:47:43] Jasper_Deng: Yeah, I should probably debug this laptop some day. Right I seem to have signed up in 2005 and it has been running ever since (altough I do remember rebooting the pop some years ago) [20:48:00] what OS? [20:48:32] Windows7 [20:50:37] show me results of route print -6 and ipconfig commands [20:52:03] (don't spam this channel) [20:52:25] I'll fix it some day ;-) [20:52:46] maybe you won't ever have to [20:54:21] !log reedy Started syncing Wikimedia installation... : [20:54:26] Logged the message, Master [20:56:17] Jasper_Deng: I think this is /127, since it's the tunnel endpoint [20:56:47] multichill: v6 works surprisingly good on my vista [20:58:11] saper: ironically the recommendation is an entire /64 [20:58:42] multichill: what do you get trying to ping wikipedia? [20:59:25] Jasper_Deng: well you are reading ietf lists probably :) [21:01:05] saper: no, Wikipedia articles [21:02:10] seldom accurate :) [21:11:43] Jasper_Deng: It's just broken router advertisements so it doesn't get a global address [21:12:03] multichill: common error [21:12:13] say "aye" if you aren't connecting through a tunenl [21:13:36] It works for the other machines in the lan, just not this one ;-) [21:49:10] !log Applied PageTriage schema updates to testwiki and enwiki [21:49:15] Logged the message, Master [22:15:16] scap is still hanging [23:11:24] Tim-away: did we have issues with scap just hanging before? It seems familiar [23:12:11] !log reedy Finished syncing Wikimedia installation... : [23:12:16] Logged the message, Master [23:12:26] .... [23:12:34] srv232: Timeout, server not responding. [23:12:34] srv232: Copying to srv232...Finished [23:15:45] I wonder why the setup timeout didn't seem to take effect [23:31:12] !log reedy synchronized wmf-config/InitialiseSettings.php 'Bug 27706 - enable RSS extension on uawikimedia' [23:31:16] Logged the message, Master [23:40:17] (Cannot contact the database server: Unknown error (10.0.0.227)) [23:40:23] this is becomeing to be common [23:40:37] can server kitties be told the ToAruShiroiNeko isnt impressed? :p [23:45:30] es3... binasher ^ [23:46:02] wee [23:46:08] =) [23:46:54] Is there any issue with editing?. Some users in spanish wikipedia can't save the articles... [23:46:55] he is a server kitty? [23:46:59] * ToAruShiroiNeko pets [23:47:05] I experienced it on en.wikt [23:48:26] wikidown! [23:48:37] abandon wiki! [23:48:41] LOL [23:48:55] '(Cannot contact the database server: Unknown error (10.0.0.227))' [23:48:55] sysops and b'crats last! :p [23:49:10] on commons [23:49:12] hoo how is it an unknown error when you know it is an error [23:49:26] what is this? rumsfelds speech? [23:50:11] Try waiting a few minutes and reloading. [23:50:12] mmmm Recent Changes doesn't have any activities (except New accounts and deleted articles) :B [23:50:14] * ToAruShiroiNeko tries [23:50:18] but fails at waiting [23:50:42] this makes rc patrolling easy [23:51:09] most effective anti-vandalism method [23:52:09] ToAruShiroiNeko: Did you reach any op, yet? [23:52:21] ToAruShiroiNeko for? [23:52:31] two people are aware [23:52:34] to get the wikis working, maybe [23:52:35] http://commons.wikimedia.org/wiki/Special:RecentChanges <-- wheee! [23:52:46] well Reedy and binasher know or something [23:52:46] !log asher synchronized wmf-config/db.php 'making es1 the master for blobs cluster 23' [23:52:51] Logged the message, Master [23:52:56] should be fixed now [23:52:58] It's back! [23:54:32] yup looks fixed [23:54:36] may I ask what died? [23:54:50] !log asher synchronized wmf-config/db.php 'fully commenting out es3' [23:54:54] Logged the message, Master [23:55:40] es3 [23:55:45] as I said originally ;) [23:58:44] what is es3? [23:58:57] ToAruShiroiNeko: External Storage 3 [23:59:06] that's where the actual page text is stored