[01:30:25] <jackmcbarn>	 why's the beta cluster 404ing?
[01:31:57] <p858snake|l>	 why? it's probably broken
[01:32:05] <p858snake|l>	 why its broken is another question
[01:33:41] <Reedy>	 hashar: ^^
[01:33:53] <hashar>	 URL ?
[01:34:04] <Reedy>	 jackmcbarn: where? link?
[01:34:27] <jackmcbarn>	 Reedy: http://deployment.wikimedia.beta.wmflabs.org/
[01:35:13] <Reedy>	 deployment.wikimedia?
[01:35:35] <Reedy>	 I don't think that is supposed to exist
[01:35:35] <Reedy>	 http://en.wikipedia.beta.wmflabs.org/wiki/Main_Page
[01:36:57] <hashar>	 jackmcbarn: I think that host has been deleted a while ago
[01:37:13] <hashar>	 hmmm
[01:37:14] <hashar>	 maybe not
[01:37:24] <hashar>	 we used to have http://beta.wmflabs.org  redirecting to it
[01:37:38] <hashar>	 Reedy: I guess some docroot has been removed ? 
[01:38:11] <Reedy>	 I didn't think I'd deleted any yet..
[01:38:33] <Reedy>	 Oh
[01:38:37] <Reedy>	 Are/were you using the "labs" docroot?
[01:39:09] <hashar>	 no clue
[01:39:36] <hashar>	 I think that part of the conf terribly confusing, I got to reverse engineer it every time I look at it
[01:40:06] <Reedy>	 lol
[01:40:15] <hashar>	 http://paste.openstack.org/show/47249/
[01:40:17] <Reedy>	 my intention is to remove most of those docroots this week hopefully
[01:40:22] <hashar>	 #0 /data/project/apache/common-local/multiversion/MWMultiVersion.php(351): trigger_error('Invalid host na...', 256)
[01:40:22] <hashar>	 :(
[01:40:35] <hashar>	 ah hmm on URL: http://deployment.en.wikipedia.beta.wmflabs.org/
[01:40:37] <hashar>	 grr
[01:40:47] <Reedy>	 Daniel and I had issues simplifying wikimania docroots yesterday... But I think we've a fix
[01:40:51] <Reedy>	 Need to test it again first
[01:41:17] <hashar>	 [Thu Sep 19 01:39:49 2013] [error] [client 10.4.1.133] File does not exist: /usr/local/apache/common/docroot/labs
[01:41:34] <hashar>	 wondering if that is defined in the apache conf
[01:41:35] <Reedy>	 lol
[01:41:47] <Reedy>	 It wasn't in operations/apache-config
[01:41:49] <Reedy>	 Hence deleting it
[01:41:56] <hashar>	 ah
[01:42:03] <hashar>	 beta still does not use the apache-config repo :D
[01:42:06] <hashar>	 we have a local copy there
[01:42:06] <Reedy>	 heh
[01:42:14] <hashar>	 anyway
[01:42:15] <Reedy>	 if you look in any of the subfolders, there is only a w folder now
[01:42:17] <hashar>	 since that points to http://deployment.wikimedia.beta.wmflabs.org
[01:42:17] <Reedy>	 :D
[01:42:28] <hashar>	 I guess I could hack the apache conf to use the wikimedia docroot instead of the labs one
[01:42:40] <Reedy>	 I moved and removed robot.txt and favicon.ico etc
[01:42:48] <hashar>	 niiiceee
[01:43:00] <Reedy>	 The next step being to remove most of the docroots
[01:43:12] <Reedy>	 bar noc and a docroot/wiki or something :D
[01:43:30] <Reedy>	 I wanted to remove them all and replace with symlinks earlier today
[01:43:52] <Reedy>	 But yeah
[01:43:55] <Reedy>	 Sorrt for breaking it :p
[01:44:03] <Reedy>	 s/Sorrt/Sorry/
[01:44:29] <hashar>	 Reedy: the apache conf http://paste.openstack.org/show/47250/
[01:44:39] <hashar>	 I guess I can replace all labs occurrences :D
[01:46:09] <hashar>	 Warning: DocumentRoot [/usr/local/apache/common/docroot/wikimedia] does not exist
[01:46:10] <hashar>	 moahah
[01:46:38] <Reedy>	 wikimedia.org
[01:46:56] <Reedy>	 or default
[01:47:15] <hashar>	 default is missing the w  / 404 etc
[01:47:38] <Reedy>	 it is!
[01:48:03] <Reedy>	 Shall I just create you a labs symlink? ;)
[01:48:07] <hashar>	 :D
[01:48:12] <hashar>	 will use the wikimedia.org one
[01:48:21] <hashar>	 http://deployment.wikimedia.beta.wmflabs.org
[01:48:24] <hashar>	 now I get a 404 :-D
[01:48:31] <hashar>	 err wiki not found
[01:48:32] <hashar>	 bah
[01:48:56] <hashar>	 the lame router might use the document root path to find out whether the wiki exists
[01:49:28] <Reedy>	 it does
[01:49:30] <Reedy>	 it's scary code :(
[01:49:42] <Reedy>	 part of the fail we had yesterday
[01:49:44] <hashar>	 where is it?
[01:49:53] <Reedy>	 multiversion/MWMultiVersion.php
[01:50:03] <Reedy>	 not sure on the capitalisation
[01:50:38] <hashar>	 ah yeah
[01:50:41] <hashar>	 that piece of crap grgb
[01:51:11] <hashar>	 setSiteInfoForWiki oh my god
[01:51:52] <Reedy>	 I noticed something last night
[01:58:16] <Reedy>	 Yeah
[01:58:26] <Reedy>	 } elseif ( preg_match( '/^\/usr\/local\/apache\/(?:htdocs|common\/docroot)\/([a-z]+)\.org/', $docRoot, $matches ) ) {
[01:58:35] <jackmcbarn>	 i get the feeling a lot happeend in the 4 minutes i was disconnected
[01:58:45] <hashar>	 jackmcbarn: yeah looking at it
[01:58:47] <hashar>	  :-D
[01:59:02] <Reedy>	 self::error( "Invalid host name ($serverName), can't determine language.\n" );
[01:59:04] <Reedy>	 ^ That one?
[01:59:14] <hashar>	 yeah that throw a fatal
[01:59:21] <hashar>	 but now we have a different error "wiki not found"
[02:09:07] <Hahc21>	 oh ok
[02:09:10] <jackmcbarn>	 hashar: https://bugzilla.wikimedia.org/show_bug.cgi?id=54302
[02:09:53] <Hahc21>	 thanks :)
[02:11:09] <jackmcbarn>	 while i'm here, what's the deployment schedule for extensions in general?
[02:11:55] <hashar>	 Reedy: honestly no clue how to fix that vhost issue :/
[02:12:06] <Reedy>	 symlink is the easiest
[02:12:09] <Reedy>	 jackmcbarn: In what sense?
[02:12:20] <hashar>	 Reedy: symlink would work, but it seems lame :d
[02:12:35] <Reedy>	 Like I say, I'm wanting to get this fixed up pretty soon
[02:13:04] <jackmcbarn>	 i just got a change merged at https://gerrit.wikimedia.org/r/#/c/83389/ . how long will it be until that hits wikipedia? i see the deployment schedule, but it only talks about core and a few extensions
[02:15:01] <Reedy>	 jackmcbarn: A week on thursday then
[02:15:09] <jackmcbarn>	 do extension changes go with core?
[02:15:12] <Reedy>	 yes
[02:15:14] <Reedy>	 It will be on testwiki and mediawikiwiki as of this thursday
[02:15:23] <Reedy>	 it'll then be on all projects that aren't wikipedia on monday
[02:15:38] <jackmcbarn>	 and it would be on the beta cluster now if the beta cluster were working, right?
[02:15:38] <Reedy>	 hashar: 
[02:15:40] <Reedy>	  3 files changed, 56 insertions(+), 455 deletions(-)
[02:15:56] <Reedy>	 Yeah, at worst a couple of hours later
[02:16:18] <jackmcbarn>	 ok, thanks
[02:17:05] <hashar>	 Reedy: oh so you get a patch pending ? :)
[02:17:18] <Reedy>	 hashar: Need to steal mutante tomorrow ideally ;)
[02:17:33] <Reedy>	 hashar: That change is just for wikimania wikis..
[11:08:17] <apergos>	 ηευ
[11:08:20] <apergos>	 hey
[11:08:37] <apergos>	 I started a new fiwiki run this morning, it's still going
[11:08:41] <apergos>	 no exceptions yet
[11:10:32] <svick>	 ok, good
[11:11:41] <svick>	 i was thinking about what format should the documentation comments be in
[11:11:51] <apergos>	 wio is a little bad but not as bad as it was with the read over nfs
[11:12:18] <svick>	 that's reading locally and writing to nfs?
[11:13:16] <apergos>	 that's what I'm doing now, right
[11:13:25] <svick>	 ok
[12:22:09] <snitch>	 [[Tech]]; Nicholasjf21; /* Scaling images with links in mobile view */ new section; https://meta.wikimedia.org/w/index.php?diff=5819420&oldid=5675311&rcid=4555615
[16:02:00] <Romaine>	 ok
[17:00:41] <greg-g>	 man, Fx, er IceWeasel, sure chokes on BZ tabs