[00:00:31] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:02:21] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK: Status line output matched 400 - 336 bytes in 0.136 second response time [00:14:22] Reedy: around?] [00:14:29] Yup [00:14:49] evidently meta is sending $site as Wikipedia rather then Wikimedia when they try to vote and so screwing up the authentication [00:15:21] this wasn't an issue in past years because in past years they used the secure site which DID use /wikipedia/meta rather then https://meta.wikimedia [00:16:21] $url = wfAppendQuery( $url, array( 'site' => $site, 'lang' => $lang ) ); [00:16:22] ? [00:16:39] yeah, I wonder if the easiest thing is to hack there to make an exception for meta? [00:16:42] to pass wikimedia? [00:16:58] aren't there going to be other sites showing as wikipedia? [00:17:09] assuming that changing meta to squeak $site=wikimedia would fuck with things [00:17:18] possibly, what does commons do? [00:17:33] I don't think there will be many [00:17:38] species should do wikimedia ... [00:17:45] the others aren't really content sites .... [00:18:13] reedy@fenari:~$ mwscript eval.php commonswiki [00:18:13] > var_dump( $lang, $site ); [00:18:13] string(7) "commons" [00:18:13] string(9) "wikipedia" [00:18:18] offs [00:18:26] then yes, that will be an issue too [00:18:32] plus many more.. [00:18:41] it's only really like chapter wikis that identify as wikimedia iirc [00:18:45] god dammit [00:19:18] this has never come up because the old auth url pointed to secure.wikimedia.org/$site/$lang [00:19:41] hmm [00:19:44] dbname? :D [00:19:58] I changed it to https://$lang.$site.org which was fine for testWiki and test2 sigh [00:21:17] would it work? I think the only variables it understands on the remote-mw side is site and lang but we could split it up however we like to send it there [00:21:30] What's it actually used for? [00:22:07] SiteConfiguration should be able to deal with/translate this stuff between one and another... [00:22:25] it's used to authenticate the voter [00:22:35] RECOVERY - Puppet freshness on wtp1 is OK: puppet ran at Sat Jun 8 00:22:27 UTC 2013 [00:22:52] so it's given a script path like https://$lang.$site.org/w and then it queries for the secure poll script path and queries auth-api [00:22:58] then it knows if the user can vote [00:23:02] RECOVERY - Puppet freshness on mexia is OK: puppet ran at Sat Jun 8 00:22:55 UTC 2013 [00:23:04] Right [00:23:22] PROBLEM - Puppet freshness on mexia is CRITICAL: No successful Puppet run in the last 10 hours [00:23:22] PROBLEM - Puppet freshness on wtp1 is CRITICAL: No successful Puppet run in the last 10 hours [00:24:12] RECOVERY - Puppet freshness on lardner is OK: puppet ran at Sat Jun 8 00:24:02 UTC 2013 [00:24:22] PROBLEM - Puppet freshness on lardner is CRITICAL: No successful Puppet run in the last 10 hours [00:25:02] RECOVERY - Puppet freshness on tola is OK: puppet ran at Sat Jun 8 00:24:59 UTC 2013 [00:25:32] PROBLEM - Puppet freshness on tola is CRITICAL: No successful Puppet run in the last 10 hours [00:25:53] Reedy: Should probably use what SiteMatrix uses to determine the URL based on the DB name? [00:26:05] hmm [00:26:12] RECOVERY - Puppet freshness on kuo is OK: puppet ran at Sat Jun 8 00:26:04 UTC 2013 [00:26:12] PROBLEM - Puppet freshness on kuo is CRITICAL: No successful Puppet run in the last 10 hours [00:26:12] RECOVERY - Puppet freshness on kuo is OK: puppet ran at Sat Jun 8 00:26:07 UTC 2013 [00:26:37] I was thinking something along those lines [00:26:37] > $wgConf->loadFullData(); [00:26:37] > var_dump( $wgConf->get( 'wgServer', 'commonswiki' ) ); [00:26:37] string(23) "//commons.wikimedia.org" [00:27:12] Yeah exactly [00:27:12] PROBLEM - Puppet freshness on kuo is CRITICAL: No successful Puppet run in the last 10 hours [00:27:39] LeslieCarr: Isn't it supposed to stop whining about that ---^^ if the box is marked as decommissioned? [00:28:47] Though, with lang and site... [00:28:48] $matrix->getCanonicalUrl( $lang, $site ) [00:37:28] ahh, yup, that looks like it would work. It does essentially doing exactly what we want in the matrixapi using that [00:38:09] brb, keep coughing grabbing water [00:39:09] We have hacks to deal with our hacks [00:39:11] * Reedy grins [00:39:56] LOL [00:39:59] just another day [00:40:29] we're at the point that the software is designed to assume the hacks are always there ;) [00:42:48] Reedy, would it be really difficult to ditch the 'wiki' suffix completely and move to 'wikipedia' and 'wikimedia'? [00:43:02] We should really do that at some point [00:43:08] But it's reasonably difficult to rename existing wikis [00:43:25] We can't rename dbs currently [00:43:38] so the long list of bugs going back years tells me [00:43:47] what blocks renaming DBs? [00:43:50] Which is why Daniel and I gave up trying to do really different renames [00:43:58] external storage cluster is readonly [00:44:36] read-only? umm [00:44:52] Yes [00:44:57] Obviously stuff is written to it [00:45:33] https://bugzilla.wikimedia.org/show_bug.cgi?id=19986#c12 [00:45:47] But you can't change stuff once it goes in? [00:45:56] "talked to Asher, it is possible but not easy. a DBA would have to do some scripting to fix the external storage issue" [00:46:07] Noting it's nearly 3 months since I touched any of this [00:46:25] I'm not sure how external storage can be readonly but written to [00:47:29] It's append-only [00:47:40] right, that makes sense [00:53:52] !log Restarted Apache on gallium AGAIN. Patch for the underlying issue is https://gerrit.wikimedia.org/r/#/c/67551/ [00:54:00] Logged the message, Mr. Obvious [00:54:21] Reedy: any time frame do you think for getting the fix out? [00:54:36] RECOVERY - Parsoid on wtp1015 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.003 second response time [00:54:39] Who's writing it? [00:54:40] ;) [00:55:57] I had assumed you were ;) sorry about that, if I need to I can but you're likely to be able to do it much faster [00:56:01] and the election is currently going [00:56:05] PROBLEM - Parsoid on wtp1009 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:56:05] PROBLEM - Parsoid on wtp1006 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:56:05] PROBLEM - Parsoid on wtp1002 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:56:11] I've stopped banners but can't really stop the electoin [00:56:16] *election [00:56:35] PROBLEM - Parsoid on wtp1020 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:56:36] PROBLEM - Parsoid on wtp1019 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:56:36] PROBLEM - Parsoid on wtp1021 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:56:36] PROBLEM - Parsoid on wtp1017 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:56:42] ^ [00:56:45] PROBLEM - Parsoid on wtp1011 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:56:45] PROBLEM - Parsoid on wtp1018 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:56:45] PROBLEM - Parsoid on wtp1024 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:56:55] PROBLEM - Parsoid on wtp1001 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:56:55] PROBLEM - Parsoid on wtp1023 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:56:55] PROBLEM - Parsoid on wtp1007 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:56:55] PROBLEM - Parsoid on wtp1003 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:56:55] PROBLEM - Parsoid on wtp1010 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:56:56] PROBLEM - Parsoid on wtp1004 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:56:56] PROBLEM - Parsoid on wtp1005 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:56:56] PROBLEM - Parsoid on wtp1022 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:57:01] ohoh [00:57:17] Restarts? [00:57:32] wtp1011 looks fine on cursory inspection [00:57:34] Checking LVS [00:57:47] LVS believes they're down [00:57:57] yeah, I just pushed the little debug print patch [00:58:16] Does it print before sending headers? [00:58:23] Did you drop Connection: Close again? [00:58:38] wtp1005 seems to have failed to restart [00:58:45] there are still hanging workers [00:58:48] wtp1011 just doesn't respond at all [00:58:52] I'll restart them harder [00:58:58] kill -9 all the workers [00:59:03] that should help [00:59:21] Jamesofur: Where am I looking in the code? Where does it build the reverse url back up? [00:59:28] !log Restarting Parsoid on all backends [00:59:36] Logged the message, Mr. Obvious [01:00:25] RECOVERY - Parsoid on wtp1020 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.008 second response time [01:00:25] RECOVERY - Parsoid on wtp1019 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.003 second response time [01:00:25] RECOVERY - Parsoid on wtp1017 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.004 second response time [01:00:25] RECOVERY - Parsoid on wtp1021 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.004 second response time [01:00:35] RECOVERY - Parsoid on wtp1011 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.004 second response time [01:00:36] RECOVERY - Parsoid on wtp1018 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.003 second response time [01:00:36] RECOVERY - Parsoid on wtp1024 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.005 second response time [01:00:45] RECOVERY - Parsoid on wtp1001 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.003 second response time [01:00:45] RECOVERY - Parsoid on wtp1010 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.003 second response time [01:00:45] RECOVERY - Parsoid on wtp1007 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.005 second response time [01:00:45] RECOVERY - Parsoid on wtp1023 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.005 second response time [01:00:45] RECOVERY - Parsoid on wtp1003 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.007 second response time [01:00:46] RECOVERY - Parsoid on wtp1004 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.009 second response time [01:00:46] RECOVERY - Parsoid on wtp1022 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.004 second response time [01:00:47] RECOVERY - Parsoid on wtp1005 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.007 second response time [01:00:48] that looks better [01:00:56] RECOVERY - Parsoid on wtp1009 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.017 second response time [01:00:56] RECOVERY - Parsoid on wtp1006 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.005 second response time [01:00:56] RECOVERY - Parsoid on wtp1002 is OK: HTTP OK: HTTP/1.1 200 OK - 1373 bytes in 0.048 second response time [01:01:41] $url = $election->getProperty( 'remote-mw-script-path' ); [01:01:42] $url = strtr( $url, $vars ); [01:02:42] I need to fix the init script [01:03:04] New patchset: Catrope; "Fix it for ganglia_new too" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/67569 [01:03:34] Reedy: ahh, hmm so it builds the reverse url back up on votewiki. It's given a remote-mw-script-path variable that is currently $https://$lang.$site.org/w/ and is used as the script path to check for the originating wiki. That's built in Auth.php to actually do the call back when it gets a call [01:04:10] reading auth.php it looks like I can give it arbitrary vars [01:04:25] so if we could just pass it canonical url it may even work ..... [01:04:32] well $url/w/ or whatever [01:04:50] $url = $election->getProperty( 'remote-mw-script-path' ); [01:04:50] https://$lang.$site.org/w [01:04:51] $url = strtr( $url, $vars ); [01:04:52] if ( substr( $url, -1 ) != '/' ) { [01:04:53] $url .= '/'; [01:04:54] } [01:04:58] right, I can change that easily [01:05:10] it's the building and knowing what to change it too [01:06:03] So we want to transform site and lang, rather than replacing them verbatim [01:06:49] yeah, currently it gets passed the two variables through the common settings script [01:06:58] and then uses them to find the appropriate local script path [01:11:12] New patchset: Catrope; "Deal with stuck Parsoid workers when restarting the service" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/67571 [01:11:39] Any idea what's passed as domain? [01:12:44] New patchset: Catrope; "Fix port for Parsoid health check" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/67336 [01:12:59] New review: Catrope; "Bumped to get Jenkins to run; Leslie, could you re-+2 this?" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/67336 [01:13:01] Reedy: as far as I can tell home wiki such as meta.wikimedia.org or en.wikipedia.org [01:13:08] (but home rather then the wiki you're actually on) [01:13:52] I'm wondering if we should say swap https://$lang.$site.org/w for like https:$website [01:14:14] Can we pass them $website and get what we need? [01:14:33] it would probably want to be https:$website/w (it has to be script path) [01:15:02] An define $website from $matrix->getUrl( $lang, $site ); [01:15:07] New review: Catrope; "This still isn't working though; Parsoid caches still don't show up in Ganglia" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/67569 [01:15:36] Reedy: We should be able to do it, it does not look like it's forcing us to use any specific variables as long as it received them [01:16:02] New patchset: Lcarr; "Fix port for Parsoid health check" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/67336 [01:17:43] Reedy: why don't we set it up, for now, to do that. So that we have less of a downtime we can set it up to, initially, send $website, $site & $lang (so that it keeps going until I make the property change and if it breaks we can quick revert with just a property change rather then a new config push) [01:17:43] Change merged: Lcarr; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/67336 [01:18:03] Thanks LeslieCarr [01:18:06] yw [01:18:29] $website = new SiteMatrix()->getUrl( $params['lang'], $params['site'] ); [01:18:29] $params['website'] = $value; [01:18:29] $vars["\$website"] = $value; [01:18:36] LeslieCarr: Do you have time for a couple more small ones? [01:18:37] blarfgh [01:18:42] First should be value (obviously) [01:18:53] you have 15 minutes :) [01:19:00] :) [01:19:03] OK, well https://gerrit.wikimedia.org/r/67569 is one line :) [01:19:16] And relatedly, it's still not working, but it's Ganglia and it's Friday so I've given up for now [01:19:16] http$website/w [01:19:29] * Jamesofur nods [01:19:38] https://gerrit.wikimedia.org/r/67520 is two lines [01:19:48] though https [01:20:00] And https://gerrit.wikimedia.org/r/67571 is short too [01:20:00] uh, yeah [01:20:03] and with a colon too [01:20:09] * Jamesofur nods [01:20:09] $wgServer is //en.wikipedia.org [01:20:23] ok, that works perfectly [01:20:25] I wonder if we should just call it $wgServer in the path for clarity [01:20:37] $website is just more vagueness [01:20:55] Reedy: You may be interested in $wgCanonicalServer [01:20:55] the only question is if it will get confused with it's own local variables but it shouldn't because it never calls the globals [01:21:08] RoanKattouw: no, because I need to force https [01:21:11] canonical does not always [01:21:12] RoanKattouw: It's usually http [01:21:16] :D [01:21:18] Yeah canonical is always http [01:21:22] OK, then yes you do want $wgServer [01:21:22] Sadly. [01:21:27] well except for one wiki I believe atm :) [01:21:29] hrm, i wonder why we have two modules .. i should figure that out :) [01:21:46] using str_replace http -> https on $wgCanonicalServer just feels even more silly [01:21:53] Change merged: Lcarr; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/67569 [01:21:53] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:21:59] LeslieCarr: _new appears unused [01:22:13] Probably someone working on rewriting it [01:22:36] New patchset: Lcarr; "Disable HTCP daemon for Parsoid caches" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/67520 [01:22:37] New patchset: Lcarr; "Deal with stuck Parsoid workers when restarting the service" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/67571 [01:22:43] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK: Status line output matched 400 - 336 bytes in 0.126 second response time [01:24:09] hrm, i am unsure if the pkill should be in the first bit or the case request [01:24:50] It needs to be there for both stop and restart [01:25:00] So it seemed to make sense to me to put it in the function that's called by both [01:25:18] ah yeah [01:25:30] Change merged: Lcarr; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/67571 [01:25:59] Whee [01:26:01] Thanks [01:26:01] Change merged: Lcarr; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/67520 [01:26:20] I'll go and stop the HTCP daemon manually [01:26:32] I couldn't be bothered to figure out the ensure => absent equivalent for that [01:26:44] it's for service [01:26:48] https://gerrit.wikimedia.org/r/67575 [01:26:49] and you do ensure => stopped [01:28:12] New patchset: Demon; "Tweak access to gitblit so it's readonly as intended" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/67576 [01:28:28] RoanKattouw: Thanks [01:28:33] <^demon|dinner> Someone got a second for a 1-line patch ^? [01:28:43] LeslieCarr: The class doesn't let me do that directly though [01:29:19] thanks both [01:30:03] Thanks much LeslieCarr [01:30:18] yw RoanKattouw [01:30:35] demon - i'll look [01:30:39] also i hate your ^ ! [01:31:38] hrm, that config file is so difficult to understand [01:31:53] RECOVERY - NTP on ssl3003 is OK: NTP OK: Offset -0.001225352287 secs [01:32:03] ;) [01:32:25] Change merged: Lcarr; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/67576 [01:33:11] Reedy: let me know when synced, I'll change the url variable [01:33:54] PROBLEM - Varnish HTCP daemon on titanium is CRITICAL: PROCS CRITICAL: 0 processes with UID = 110 (vhtcpd), args vhtcpd [01:33:54] RECOVERY - NTP on ssl3002 is OK: NTP OK: Offset -0.003519415855 secs [01:34:13] PROBLEM - Varnish HTCP daemon on cerium is CRITICAL: PROCS CRITICAL: 0 processes with UID = 110 (vhtcpd), args vhtcpd [01:34:45] Gee thanks icinga [01:34:52] 'domain' => preg_replace( '!.*/(.*)$!', '$1', $wgServer ), [01:35:21] merged in sockpuppet [01:35:24] which machine is it on? [01:35:36] gitblit, that is [01:36:22] !log reedy synchronized php-1.22wmf5/extensions/SecurePoll/ [01:36:32] Logged the message, Master [01:36:59] !log reedy synchronized php-1.22wmf6/extensions/SecurePoll/ [01:37:06] Logged the message, Master [01:38:26] <^demon> LeslieCarr: antimony, I'll run puppet, already logged in [01:40:23] ok everyone, free merge time has ended [01:42:35] Jamesofur: Should be good to go... [01:42:41] yeah .... [01:42:50] to didn't break on the wikis it already worked on .... [01:42:59] but for some reason it still isn't working for meta ... [01:43:29] | 290 | remote-mw-script-path | https:$wgServer/w | [01:44:57] *it didn't [01:50:13] > $matrix = new SiteMatrix(); [01:50:13] > var_dump( $matrix->getUrl( $lang, $site ) ); [01:50:13] string(20) "//meta.wikipedia.org" [01:50:16] :/ [01:50:23] god damnit [01:51:41] could we grab $wgServer directly and pass it as a variable ?server= when it goes through wmfSecurePollJumpUrl on CommonSettings.php ? (on the origination side)? [01:51:57] because that should be right? It will see the overrides we have for places like meta/commons [01:52:53] function wmfSecurePollJumpUrl( $page, &$url ) { [01:52:54] global $site, $lang, $wgServer; [01:52:56] $url = wfAppendQuery( $url, array( 'site' => $site, 'lang' => $lang, 'server' => $wgServer) ); [01:52:56] return true; [01:52:57] } [01:53:09] we'd have to escape the // [01:53:19] hmm yeah [01:56:04] So what are we missing [01:57:31] Almost like something is cached [01:58:47] what would it be? if getURL (lang,site) gets meta.wikipedia Securepoll is currently acting as intended right? … it's just not how we want it …… [01:59:42] Oh shit [01:59:46] I didn't notice that [02:00:08] yeah :-/ [02:00:26] wrtf [02:02:55] Jamesofur: wgDBname? [02:02:56] > echo $wgConf->get( 'wgServer', 'metawiki' ); [02:02:56] /meta.wikimedia.org [02:03:06] irc client obv ate the first / [02:03:24] err [02:04:46] since we're on our servers that could work, would just want to wrap it around something that we knew would only be here I guess (though at this point 'meh I can keep a note and fix that later' ) [02:05:35] How does meta (et al) display correctly elswhere then? [02:05:57] !log LocalisationUpdate completed (1.22wmf5) at Sat Jun 8 02:05:56 UTC 2013 [02:06:06] Logged the message, Master [02:06:18] elsewhere where? Around the projects it has an override in initializesettings I think so that $wgServer is always going to be right [02:07:09] [02:08:16] !log LocalisationUpdate completed (1.22wmf6) at Sat Jun 8 02:08:16 UTC 2013 [02:08:19] > echo $matrix->getUrl( 'meta', 'wiki' ); [02:08:19] /meta.wikimedia.org [02:08:24] Logged the message, Master [02:08:35] etf [02:08:37] wtf [02:08:39] OH [02:08:39] > var_dump ($lang, $site ); [02:08:39] string(4) "meta" [02:08:39] string(9) "wikipedia" [02:08:41] wiki [02:08:42] wikipedia [02:08:46] fESAFTRMJOIREWMFOMFR|OIM£FRw3f [02:08:52] * Jamesofur facepalms [02:09:24] > echo $matrix->getUrl( 'en', 'wiki' ); [02:09:24] /en.wikipedia.org [02:09:24] > echo $matrix->getUrl( 'en', 'wikipedia' ); [02:09:24] /en.wikipedia.org [02:09:24] > echo $matrix->getUrl( 'meta', 'wiki' ); [02:09:25] /meta.wikimedia.org [02:09:27] > echo $matrix->getUrl( 'meta', 'wikipedia' ); [02:09:30] /meta.wikipedia.org [02:10:57] SiteConfiguration normalises this in sitefromdb [02:12:40] str_replace( 'wikipedia', 'wiki', $params['site'] ) [02:14:02] Jamesofur: works for you? :p [02:14:10] * Jamesofur coughs [02:14:12] works for me LOL [02:14:33] dbname would've been easier ;) [02:14:41] yeah lol [02:15:48] welcome back [02:15:56] right click fail [02:18:15] I wonder if I should just add in wgDBname too [02:19:14] Meh, not now [02:19:43] !log LocalisationUpdate ResourceLoader cache refresh completed at Sat Jun 8 02:19:43 UTC 2013 [02:19:51] Logged the message, Master [02:21:52] yeahhh, though now that we've said it we'll need it [02:24:18] !log reedy synchronized php-1.22wmf6/extensions/SecurePoll/ [02:24:26] Logged the message, Master [02:24:49] And it's apparently getting light outside.. [02:25:01] !log reedy synchronized php-1.22wmf5/extensions/SecurePoll/ [02:25:08] Logged the message, Master [02:25:13] as it's getting dark outside here :( [02:25:33] AHA! [02:25:34] \o/ [02:25:38] I has vote from meta [02:25:54] Actually, 'wiki' would already be the dbname [02:26:22] but of course, that doesn't transfer to a domain or anything [02:26:35] Risker: we're working from commons/meta now, we should be good though I'm trying a couple different random sites to make sure [02:26:46] you said you had a vote from meta? [02:26:53] it's not in the list [02:26:58] okay [02:27:04] I just got to the vote page meaning that I got through login ;) [02:27:06] you had me slightly freaking out there :) [02:27:09] sorry [02:27:25] see if you can get someone to do a vote [02:30:06] Risker: Want me to? [02:30:18] sure, James._F [02:30:24] if you could that would be great [02:30:34] banners are ready just want to make sure we're good on all ends [02:30:43] I've tried authentication on wikivoyage/commons/meta/en so far [02:30:59] Just voted. [02:31:10] Want me to paste the SPID? [02:32:47] nah [02:32:51] thanks [02:33:05] Did it work? [02:33:10] it did [02:33:14] Yay? [02:33:18] yay! ;) [02:33:23] Yay. :-) [02:33:43] the big thing before was an authentication issue, so no votes were lost just people annoyed (they were told it was broken, they weren't able to actually get to a vote screen) [02:33:46] ok [02:37:33] thanks, James and Reedy [02:42:18] Is it bed time now? [02:43:02] <^demon> Still never got those redirects working :\ [02:44:34] Reedy, I owe you a couple of pitchers of whatever you're drinking whenever our paths cross [02:46:27] Reedy: you say that like you are allowed to sleep? [03:57:44] PROBLEM - Puppetmaster HTTPS on stafford is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:58:34] RECOVERY - Puppetmaster HTTPS on stafford is OK: HTTP OK: Status line output matched 400 - 336 bytes in 0.136 second response time [04:11:58] RECOVERY - Puppet freshness on wtp1 is OK: puppet ran at Sat Jun 8 04:11:53 UTC 2013 [04:11:58] PROBLEM - Puppet freshness on wtp1 is CRITICAL: No successful Puppet run in the last 10 hours [04:11:58] RECOVERY - Puppet freshness on wtp1 is OK: puppet ran at Sat Jun 8 04:11:55 UTC 2013 [04:11:58] RECOVERY - Puppet freshness on mexia is OK: puppet ran at Sat Jun 8 04:11:57 UTC 2013 [04:12:38] PROBLEM - Puppet freshness on mexia is CRITICAL: No successful Puppet run in the last 10 hours [04:12:48] RECOVERY - Puppet freshness on lardner is OK: puppet ran at Sat Jun 8 04:12:45 UTC 2013 [04:12:58] PROBLEM - Puppet freshness on wtp1 is CRITICAL: No successful Puppet run in the last 10 hours [04:13:19] PROBLEM - Puppet freshness on lardner is CRITICAL: No successful Puppet run in the last 10 hours [04:14:48] RECOVERY - Puppet freshness on tola is OK: puppet ran at Sat Jun 8 04:14:38 UTC 2013 [04:14:48] PROBLEM - Puppet freshness on tola is CRITICAL: No successful Puppet run in the last 10 hours [04:15:38] RECOVERY - Puppet freshness on kuo is OK: puppet ran at Sat Jun 8 04:15:32 UTC 2013 [04:16:19] PROBLEM - Puppet freshness on kuo is CRITICAL: No successful Puppet run in the last 10 hours [04:29:38] RECOVERY - Puppet freshness on wtp1 is OK: puppet ran at Sat Jun 8 04:29:35 UTC 2013 [04:29:58] PROBLEM - Puppet freshness on wtp1 is CRITICAL: No successful Puppet run in the last 10 hours [04:30:08] RECOVERY - Puppet freshness on mexia is OK: puppet ran at Sat Jun 8 04:29:59 UTC 2013 [04:30:28] RECOVERY - Puppet freshness on lardner is OK: puppet ran at Sat Jun 8 04:30:17 UTC 2013 [04:30:38] PROBLEM - Puppet freshness on mexia is CRITICAL: No successful Puppet run in the last 10 hours [04:31:18] PROBLEM - Puppet freshness on lardner is CRITICAL: No successful Puppet run in the last 10 hours [04:32:28] RECOVERY - Puppet freshness on tola is OK: puppet ran at Sat Jun 8 04:32:16 UTC 2013 [04:32:49] PROBLEM - Puppet freshness on tola is CRITICAL: No successful Puppet run in the last 10 hours [04:34:58] RECOVERY - Puppet freshness on kuo is OK: puppet ran at Sat Jun 8 04:34:53 UTC 2013 [04:35:18] PROBLEM - Puppet freshness on kuo is CRITICAL: No successful Puppet run in the last 10 hours [05:27:32] PROBLEM - NTP on ssl3002 is CRITICAL: NTP CRITICAL: No response from NTP server [05:29:30] PROBLEM - NTP on ssl3003 is CRITICAL: NTP CRITICAL: No response from NTP server [06:23:14] New review: Tim Landscheidt; "This (and the bastion's section) should go in exec_environ, Also, the script should be installed in..." [operations/puppet] (production) C: -1; - https://gerrit.wikimedia.org/r/66266 [07:39:58]