[00:00:05] <legoktm>	 maybe someone can fix that? ^^
[00:00:16] <Ryan_Lane>	 wait...
[00:00:21] <Ryan_Lane>	 this is pointing at the wrong thing
[00:00:22] <YuviPanda>	 Ryan_Lane: i'm adding addresses for people, and stuff. I'll try to not steal the certs :)
[00:00:26] <YuviPanda>	 Ryan_Lane: what is?
[00:00:29] <Ryan_Lane>	 YuviPanda: you had a passphrase on *your* key
[00:00:34] <YuviPanda>	 Ryan_Lane: oh, yes
[00:00:36] <Ryan_Lane>	 and it's still pointing at it
[00:00:39] <YuviPanda>	 what
[00:00:42] <YuviPanda>	 Ryan_Lane: what instance are you on
[00:00:44] <Ryan_Lane>	 no idea why
[00:00:46] <Ryan_Lane>	 proxy-dammit
[00:01:05] <YuviPanda>	 Ryan_Lane: that's... weird.
[00:01:09] <Ryan_Lane>	 ooohhhh
[00:01:18] <Ryan_Lane>	 sites-enabled isn't a link
[00:01:40] <Ryan_Lane>	 removed the default file
[00:01:43] <Ryan_Lane>	 it should add a link
[00:02:13] <YuviPanda>	 Ryan_Lane: running puppet again?
[00:02:16] <Ryan_Lane>	 I'm still getting a self-signed
[00:02:17] <YuviPanda>	 Ryan_Lane: also can i haz projectadmin back?
[00:02:32] <YuviPanda>	 legoktm: add a puppet patch? ;)
[00:02:32] <Ryan_Lane>	 I put the private key there
[00:02:41] <legoktm>	 YuviPanda: I don't know puppet....
[00:02:49] <Ryan_Lane>	 I have to go, the proxy is back up and running like it was before
[00:02:55] <Ryan_Lane>	 I'll look at this in a bit
[00:03:11] <YuviPanda>	 Ryan_Lane: can you give me projectadmin?
[00:03:16] <YuviPanda>	 Ryan_Lane: have to add domains and stuff for people
[00:03:23] <Ryan_Lane>	 YuviPanda: you can give yourself root again if I do that
[00:03:44] <YuviPanda>	 Ryan_Lane: yeah, I guess that's okay with me now.
[00:03:58] <Ryan_Lane>	 I wouldn't add anyone new until we have everything working right ;)
[00:04:12] <YuviPanda>	 Ryan_Lane: oh come on :P
[00:04:17] <Ryan_Lane>	 heh
[00:04:18] <Ryan_Lane>	 one sec
[00:04:29] <Ryan_Lane>	 you have projectadmin again
[00:04:30] <Ryan_Lane>	 back in a while
[00:04:31] <YuviPanda>	 thanks
[01:39:00] <wm-bot>	 [bz] (8NEW - created by: 2Krinkle, priority: 4Low - 6enhancement) [Bug 45829] Labs: Let mail (from cron and perhaps other defaults) go to project forwarder by default - https://bugzilla.wikimedia.org/show_bug.cgi?id=45829
[01:41:49] <Betacommand>	 does tools have its own section in bugzilla?
[01:47:27] <wm-bot>	 [bz] (8NEW - created by: 2Betacommand, priority: 4Unprioritized - 6normal) [Bug 54052] tools.wmflabs.org inaccessible via labs instances - https://bugzilla.wikimedia.org/show_bug.cgi?id=54052
[01:54:41] <wm-bot>	 [bz] (8NEW - created by: 2Betacommand, priority: 4Unprioritized - 6normal) [Bug 54053] webserver treats script headers flaky - https://bugzilla.wikimedia.org/show_bug.cgi?id=54053
[01:57:34] <wm-bot>	 [bz] (8NEW - created by: 2Betacommand, priority: 4Unprioritized - 6normal) [Bug 54054] grid default output location - https://bugzilla.wikimedia.org/show_bug.cgi?id=54054
[02:00:13] <wm-bot>	 [bz] (8NEW - created by: 2Betacommand, priority: 4Unprioritized - 6normal) [Bug 54055] tool account cron emails - https://bugzilla.wikimedia.org/show_bug.cgi?id=54055
[02:03:06] <wm-bot>	 [bz] (8NEW - created by: 2Betacommand, priority: 4Unprioritized - 6normal) [Bug 54056] symlinks not always followed - https://bugzilla.wikimedia.org/show_bug.cgi?id=54056
[02:03:22] <Betacommand>	 You guys asked me to file bugs
[03:48:02] <wm-bot>	 [bz] (8UNCONFIRMED - created by: 2Nicolas Raoul, priority: 4Unprioritized - 6critical) [Bug 54059] LAMP instance becomes 404 a few hours after spawn (reproducible) - https://bugzilla.wikimedia.org/show_bug.cgi?id=54059
[06:20:59] <Nicolas>	 How to make a LAMP instance accessible via HTTPS too?
[06:39:41] <Nicolas>	 What is the difference between certificates::star_wmflabs and certificates::star_wmflabs_org, which one should I use?
[06:55:21] <wm-bot>	 [bz] (8UNCONFIRMED - created by: 2Nicolas Raoul, priority: 4Unprioritized - 6normal) [Bug 54065] Enable HTTPS on a Labs instance - https://bugzilla.wikimedia.org/show_bug.cgi?id=54065
[09:17:48] <wm-bot>	 [bz] (8NEW - created by: 2Betacommand, priority: 4High - 6major) [Bug 54052] tools.wmflabs.org inaccessible via labs instances - https://bugzilla.wikimedia.org/show_bug.cgi?id=54052
[10:17:15] <wm-bot>	 [bz] (8NEW - created by: 2Betacommand, priority: 4Unprioritized - 6normal) [Bug 54053] webserver treats script headers flaky - https://bugzilla.wikimedia.org/show_bug.cgi?id=54053
[10:21:08] <wm-bot>	 [bz] (8NEW - created by: 2Betacommand, priority: 4Unprioritized - 6normal) [Bug 54056] symlinks not always followed - https://bugzilla.wikimedia.org/show_bug.cgi?id=54056
[12:31:05] * Cyberpower678  pokes Coren 

[12:41:39] <wm-bot>	 [bz] (8NEW - created by: 2Betacommand, priority: 4Unprioritized - 6normal) [Bug 54074] $HOME not abbreviated - https://bugzilla.wikimedia.org/show_bug.cgi?id=54074
[13:00:45] <Betacommand>	 what is the difference between revision and revision_complete
[13:00:58] <Cyberpower678>	 ?
[14:28:12] <wm-bot>	 [bz] (8NEW - created by: 2Betacommand, priority: 4Unprioritized - 6normal) [Bug 54077] Webservers mis-configured - https://bugzilla.wikimedia.org/show_bug.cgi?id=54077
[14:43:32] <alchimista>	 Coren: who manages  /shared/pywikipedia ? It's quite outdated
[14:52:48] <harshkothari>	 ssh: Could not resolve hostname tools-login.wmflabs.org: No address associated with hostname 
[14:53:03] <harshkothari>	 what is the remedy of this ?
[15:01:44] <harshkothari>	 ssh: Could not resolve hostname tools-login.wmflabs.org: No address associated with hostname
[15:01:54] <harshkothari>	 how can I solve it 
[15:38:20] <harshkothari>	 ssh: Could not resolve hostname tools-login.wmflabs.org: No address associated with hostname
[15:38:28] <harshkothari>	 any remedy for this ?
[15:52:22] <Cyberpower678>	 Coren, poke
[16:03:17] <petan>	 harshkot_: around?
[16:03:22] <petan>	 !bastion | harshkot_
[16:03:22] <wm-bot>	 harshkot_: http://en.wikipedia.org/wiki/Bastion_host; lab's specific bastion host is: bastion.wmflabs.org which should resolve to 208.80.153.194; see !access
[16:03:32] <petan>	 try that IP
[16:03:52] <harshkot_>	 ok ok yes I am just finding IP of this petan
[16:04:07] <harshkot_>	 thanks petan working this is issue with ubuntu 12.04
[16:04:14] <petan>	 tbh it changed
[16:04:20] <petan>	 !bastio del
[16:04:21] <wm-bot>	 Unable to find the specified key in db
[16:04:25] <petan>	 !bastion del
[16:04:25] <wm-bot>	 Successfully removed bastion
[16:04:41] <petan>	 !bastion is http://en.wikipedia.org/wiki/Bastion_host; lab's specific bastion host is: bastion.wmflabs.org which should resolve to 208.80.153.207; see !access
[16:04:41] <wm-bot>	 Key was added
[16:05:32] <Cyberpower678>	 petan, can you program wm-bot to page Coren when I use @page Coren
[16:05:40] <petan>	 yes
[16:05:53] <petan>	 but now I am on my way :P
[16:06:02] <Cyberpower678>	 When can you do it?
[16:06:03] <petan>	 but wait
[16:06:12] <petan>	 @page user bla bla bla
[16:06:15] <petan>	 or jjust @page user
[16:06:17] <petan>	 I don'
[16:06:25] <petan>	 t understand what such a thing would be useful for
[16:06:31] <Cyberpower678>	 @page Coren
[16:06:46] <petan>	 so that it would just do sometehing like hey Coren Cyberpower678 was looking for you?
[16:06:49] <Cyberpower678>	 petan, I mean page him in RL.  On his pager.
[16:06:55] <petan>	 oh LOL
[16:06:57] <petan>	 no
[16:07:00] <petan>	 I won't do that
[16:07:27] <Cyberpower678>	 And then also program @alarm
[16:07:32] <petan>	 what for
[16:07:47] <Cyberpower678>	 Which sends an alarm to Coren when ever he is sleeping. :p
[16:08:59] <Cyberpower678>	 I need him to add another CPU to cyberbot and maybe 1 gig of additional RAM.
[16:24:54] <T13|needsCoffee>	 Ha...
[16:26:45] <T13|needsCoffee>	 petan: I could go for an @alarm that does something like ping you at a certain time so if you're plugging along oblivious to time and you have a hot date at 8 it breaks your concentration so she doesn't break your neck for standing her up...
[17:07:50] <ireas>	 hi there! 
[17:08:33] <ireas>	 I’m looking for a Tools admin. My tool (pb) does not have a replica.my.cnf in its home directory. :/
[17:19:06] <wm-bot>	 [bz] (8NEW - created by: 2Ryan Lane, priority: 4Unprioritized - 6normal) [Bug 54080] man package needs to be added to default image - https://bugzilla.wikimedia.org/show_bug.cgi?id=54080
[17:21:13] <Ryan_Lane>	 YuviPanda: proxy is working!
[17:21:33] <Ryan_Lane>	 Coren: ireas is having some replica issues ^^
[17:25:58] <YuviPanda>	 Ryan_Lane: woohoo!
[17:26:00] * YuviPanda  checks

[17:26:10] <YuviPanda>	 Ryan_Lane: it does!
[17:26:13] <YuviPanda>	 Ryan_Lane: what was the problem?
[17:26:32] <Ryan_Lane>	 you aren't linking default in sites-enabled to sites-available
[17:26:42] <Ryan_Lane>	 I'm actually surprised anything was working at all without that
[17:27:09] <YuviPanda>	 Ryan_Lane: oh, hmm
[17:27:41] <YuviPanda>	 Ryan_Lane: good question. I've no idea how that worked
[17:27:47] <YuviPanda>	 Ryan_Lane: but yeah, now all cool! yay! :)
[17:28:08] <YuviPanda>	 Ryan_Lane: wikitech interface and we can then start knocking on doors and getting back IPs :)
[17:28:16] <Ryan_Lane>	 yep
[17:37:57] <YuviPanda>	 Ryan_Lane: did the bug with 'can not delete domains' got fixed?
[17:40:08] <Betacommand>	 Coren: you around?
[17:42:37] <YuviPanda>	 Ryan_Lane: heh, https://blue-dragon.wmflabs.org/ gives me self-signed cert, because that domain has been added to two instances - the real proxy and my test proxy, and I can't remove it from the test proxy
[17:42:50] <Ryan_Lane>	 it doesn't for me on new browsers
[17:43:40] <YuviPanda>	 Ryan_Lane: yeah, because if you do a dig you see you get two IPs back. I'm on the bad one, you're on the good one
[17:44:11] <Cyberpower678>	 !Coren
[17:44:11] <wm-bot>	 Coren is dead.  petan killed him.  He now roams about as a zombie.
[17:44:19] <Cyberpower678>	 :O
[17:46:37] <Ryan_Lane>	 YuviPanda: ahhhhh
[17:46:41] <Ryan_Lane>	 stupid broken dns :)
[17:46:46] <Ryan_Lane>	 thankfully we're fixing that
[17:47:24] <YuviPanda>	 Ryan_Lane: ah, okay. How long off would that be?
[17:47:45] <YuviPanda>	 Ryan_Lane: also is there any way you can get rid of one of the IPs for me, in an internal something way? :D
[17:47:49] <YuviPanda>	 so that it can consistently work?
[17:48:42] <YuviPanda>	 mutante: https://wikistats.wmflabs.org/ works fine now :)
[17:51:59] <Ryan_Lane>	 fixing DNS is necessary for the frontend
[17:52:06] <Ryan_Lane>	 andrewbogott_afk: is working on that now
[17:52:11] <paravoid>	 DNS?
[17:52:33] <YuviPanda>	 Ryan_Lane: alright
[17:55:25] <Ryan_Lane>	 paravoid: the way we write entries into LDAP isn't good
[17:55:25] <Ryan_Lane>	 we're fixing that
[18:20:16] <Betacommand>	 Ryan_Lane: the tools webservers are being strange again
[18:20:25] <Ryan_Lane>	 Coren: ^^
[18:22:06] <Betacommand>	 OK whos been playing with the web servers?
[18:29:17] <Coren>	 I'm at the all hands but will try to see what's up. Don't have good backs roll, can someone give me the summary?
[18:30:23] <Coren>	 Scrollback*
[18:30:33] <YuviPanda>	 !logs
[18:30:34] <wm-bot>	 raw text: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-labs/ cute html: http://tools.wmflabs.org/wm-bot/logs/index.php?display=%23wikimedia-labs
[18:30:36] <YuviPanda>	 :D
[18:31:00] <ireas>	 Coren: my problem is: my tool pb has no .replica.my.cnf, so I cannot access the database
[18:31:55] <Coren>	 Ireas: that is easily fixed if you give me a minute. But there was something about the web servers?
[18:33:07] <ireas>	 Coren: okay, I’ll wait :)
[18:35:57] <Coren>	 ireas: You should be okay. 
[18:36:21] <Coren>	 Betacommand: What issue do you have with the web servers?
[18:36:38] <ireas>	 Coren: great, thank you :)
[18:37:16] <Cyberpower678>	 Coren, Betacommand's issue is that the scripts don't run in the cgi-bin
[18:37:37] <Cyberpower678>	 Coren, Can I get another gig and CPU added to Cyberbot?
[18:37:46] <wm-bot>	 [bz] (8RESOLVED - created by: 2Liangent, priority: 4Low - 6normal) [Bug 49454] Install mosh-server on bastion - https://bugzilla.wikimedia.org/show_bug.cgi?id=49454
[18:37:47] <wm-bot>	 [bz] (8ASSIGNED - created by: 2Daniel Kinzler, priority: 4Unprioritized - 6enhancement) [Bug 52693] Allow login using mosh as an alternative to plain ssh on bastion - https://bugzilla.wikimedia.org/show_bug.cgi?id=52693
[18:38:24] <Coren>	 Cyberpower678: Not now, but once I'm back at my office we can look at it. 
[18:38:34] <Cyberpower678>	 Coren, thanks.
[18:39:46] <Cyberpower678>	 @seen riley
[18:39:46] <wm-bot>	 Cyberpower678: Last time I saw riley they were changing the nickname to , but  is no longer in channel #wikipedia-en-accounts-unreg at 7/23/2013 8:37:42 AM (51d10h2m3s ago)
[18:39:59] <Cyberpower678>	 @seen Riley
[18:40:00] <wm-bot>	 Cyberpower678: Last time I saw Riley they were changing the nickname to , but  is no longer in channel #wikipedia-en-accounts-unreg at 7/23/2013 8:37:42 AM (51d10h2m17s ago)
[18:43:50] <YuviPanda>	 legoktm: I got a patch in for mosh on bastions
[18:44:39] <Betacommand>	 Coren: its started to work now but see https://bugzilla.wikimedia.org/show_bug.cgi?id=54077
[18:53:49] <wm-bot>	 [bz] (8NEW - created by: 2Betacommand, priority: 4Unprioritized - 6normal) [Bug 54077] Webservers mis-configured - https://bugzilla.wikimedia.org/show_bug.cgi?id=54077
[18:59:21] <alchimista>	 hello, a cron question, since i'm having some dificulties. Does jsub acepts arguments to set the directory here the script will run?
[19:01:02] <YuviPanda>	 Coren: ^
[19:01:52] <Coren>	 alchimista: Not directly, but if you use -cwd it will stay in the directory it was s tarted in. 
[19:02:55] <alchimista>	 that i know, and it really works, but now i'm tryng to migrate my scripts wich will run from cron Coren, and the problem is that some of them *must* run on a specific dir
[19:03:39] <Coren>	 You can do "cd someplace; jsub -cwd ..." In the crontab.
[19:04:47] <alchimista>	 without a bash wrapper?
[19:05:20] <Coren>	 That'd also do, but isn't necessary. 
[19:14:02] <alchimista>	 it's working Coren, thanks! 
[19:21:06] <Betacommand>	 Coren: I responded to your comment on bugzilla
[20:30:49] <Nettrom>	 Coren (or someone else): I posted to labs-l about non-tool accounts not having access to tools-db.  Is that by design and something I should just work my way around?  Never saw any of the admins comment on the thread, figured I'd ask here.
[21:31:04] <wm-bot>	 Change on 12mediawiki a page Wikimedia Labs/Tool Labs/Roadmap de was modified, changed by Wiegels link https://www.mediawiki.org/w/index.php?diff=781818 edit summary: Rechtschreibung
[22:00:18] <YuviPanda>	 andrewbogott_afk: any word on DNS on labs getting fixed?
[22:11:46] <Coren>	 Jett
[22:11:57] <Coren>	 &&$ auto fill
[22:12:26] <YuviPanda>	 it's okay, Coren. You can say 'fuck' on IRC :)
[22:12:34] <Coren>	 Nettrom: it was by design, but I've been reconsidering it
[22:13:25] <YuviPanda>	 Coren: can non-tools submit jobs?
[22:13:41] <YuviPanda>	 (I've a use case for both)
[22:14:32] <Coren>	 YuviPanda: Yes, but not continuous jobs
[22:33:01] <YuviPanda>	 !logs
[22:33:01] <wm-bot>	 raw text: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-labs/ cute html: http://tools.wmflabs.org/wm-bot/logs/index.php?display=%23wikimedia-labs
[22:33:52] <YuviPanda>	 Coren: oh, that's fine. one off ones are good enough for me.
[22:33:59] <YuviPanda>	 Coren: also why not continuous jobs?
[22:46:51] <legoktm>	 YuviPanda: yay :D ty!
[22:55:03] <legoktm>	 YuviPanda: is it live?
[23:01:21] <Coren>	 YuviPanda:  Because we don
[23:01:39] <Coren>	 don't want bot or bot-like things that cannot have maintainers switched.
[23:03:45] <Betacommand>	 Coren: those times are when I was debugging the script
[23:05:23] <Coren>	 When it failed, you mean?  Is there some instrumentation you turned on only for those runs?
[23:06:25] <Betacommand>	 Coren: no
[23:06:40] <Technical_13>	 Cyberpower678: did Coren give you your extra cpu and ram yet?
[23:06:41] <YuviPanda>	 legoktm: not merged yet
[23:06:45] <legoktm>	 ok
[23:07:02] <YuviPanda>	 legoktm: will need to merge that
[23:07:21] <Cyberpower678>	 Coren said he didn't have time at the moment.  That he will take a look later.
[23:07:22] <Betacommand>	 Coren: I was just getting the new script setup
[23:07:42] <Betacommand>	 Cyberpower678 suggested moving it out of cgi-bin and it worked
[23:07:59] <Coren>	 Technical_13: I'm at the all0staff, much like basically all of ops; so most things will be delayed slightly that are below "machine room is on fire" level.  :-)
[23:08:01] <Betacommand>	 I came back a few hours later and both locations where working
[23:08:31] <Technical_13>	 Fair enough. :)
[23:09:45] <Coren>	 Betacommand: That strays in "dafu" territory.
[23:09:54] <Betacommand>	 dafu?
[23:10:14] <Coren>	 "What dafu is going on?"
[23:10:37] <Betacommand>	 Coren: I know, really pissed me off when I was trying to get the code working
[23:11:21] <Betacommand>	 Coren: my Blood pressure almost doubled
[23:11:54] <Coren>	 Understandably so.  I really wish it had happened while I could catch it in the act.  Hopefully, there is some trace in the logs so that I can figure out what happened so that it doesn
[23:12:01] <Coren>	 doesn't again.
[23:12:57] <Betacommand>	 Coren: did you see the half dozen bugs I filed?
[23:13:09] <Coren>	 I did.  I
[23:13:52] <Coren>	 I'm going to be able to concentrate on bug squishing once all-staff is done.  Right now we are "team building"
[23:14:55] <Betacommand>	 Coren: Ill continue to file bugs until things work like they should :)
[23:15:35] <Coren>	 That's what they pay me the big^H^H^H medi^H^H^H^H somewhat adequate bucks for.  :-)
[23:17:40] <YuviPanda>	 Coren: thoughts on https://gerrit.wikimedia.org/r/#/c/84024/? :D
[23:17:53] <Coren>	 Although sometimes our respective definitions of "should" diverge.
[23:18:31] <Coren>	 YuviPanda: Please to use => present, we don
[23:18:47] <YuviPanda>	 Coren: there's a ensure => latest right above it
[23:18:59] <Coren>	 we don't want to have something logins depends on upgraded behind our backs.
[23:19:17] <YuviPanda>	 Coren: fixing
[23:20:22] <YuviPanda>	 Coren: updated.
[23:23:29] <wm-bot>	 [bz] (8PATCH_TO_REVIEW - created by: 2Daniel Kinzler, priority: 4Unprioritized - 6enhancement) [Bug 52693] Allow login using mosh as an alternative to plain ssh on bastion - https://bugzilla.wikimedia.org/show_bug.cgi?id=52693
[23:24:52] <Coren>	 YuviPanda: pushed
[23:24:59] <YuviPanda>	 Coren: sweeet!
[23:25:01] <YuviPanda>	 legoktm: ^
[23:25:44] <YuviPanda>	 Coren: do you know if the labs bastion has the misc::bastionhost role applied?
[23:33:26] <Coren>	 YuviPanda: it uses ssh::bastion
[23:43:57] <YuviPanda>	 Coren!
[23:44:06] <YuviPanda>	 Coren: https://gerrit.wikimedia.org/r/#/c/84105/ and https://gerrit.wikimedia.org/r/#/c/84104/2?
[23:44:11] <YuviPanda>	 Coren: same things but on different class
[23:45:08] <Coren>	 Man, Yuvi, can this wait until a slightly better time than on my ipad at the corner of a table during all-hands?  :-)
[23:45:12] <YuviPanda>	 legoktm: well, these two should help 
[23:45:16] <YuviPanda>	 Coren: pfffffft :P
[23:45:27] <YuviPanda>	 Coren: there are comfy coucheeessss! 
[23:46:29] <Coren>	 YuviPanda: You know Ryan has +2 too, right?  :-)
[23:46:38] <YuviPanda>	 Coren: sure, but I don't find hihm
[23:46:41] <YuviPanda>	 Ryan_Lane: ^
[23:46:49] * Ryan_Lane  hides

[23:46:51] <YuviPanda>	 Coren: he's in a different set of couches
[23:47:01] <YuviPanda>	 Ryan_Lane: DNS is still broken :(
[23:48:25] <Ryan_Lane>	 YuviPanda: yeah, we need to modify the code
[23:48:26] <Ryan_Lane>	 oh
[23:48:29] <Ryan_Lane>	 you mean for blue-dragon?
[23:48:33] <YuviPanda>	 Ryan_Lane: yeah
[23:48:36] <Ryan_Lane>	 I can manually fix some things...
[23:48:38] <YuviPanda>	 Ryan_Lane: and also for gingle.wmflabs.org now
[23:48:47] <YuviPanda>	 Ryan_Lane: woo, that'll be nice
[23:48:56] <YuviPanda>	 Ryan_Lane: because it works half the time and doesn't half the other time
[23:49:09] <Ryan_Lane>	 heh
[23:49:24] <Ryan_Lane>	 YuviPanda: what's broken with it?