[00:49:25] <wikibugs>	 06Labs, 10Tool-Labs: Can't login to tts service group in tools - https://phabricator.wikimedia.org/T149505#2754654 (10Ladsgroup)
[01:01:19] <wikibugs>	 06Labs, 10Tool-Labs: New user home directory creation broken - https://phabricator.wikimedia.org/T149505#2754707 (10Krenair)
[01:52:51] <Zppix>	 8:43 PM <Zppix> Anyone have any idea why my bot will run on bastion but not on the grid even with trusty?? Ive asked in labs but havent really got a response
[01:53:15] <Krenair>	 what was the error from the grid?
[01:57:42] <Zppix>	 T148944
[01:57:43] <stashbot>	 T148944: Complete config - https://phabricator.wikimedia.org/T148944
[01:58:55] <Krenair>	 what command are you running to submit it to the grid engine Zppix?
[01:59:43] <Krenair>	 because to me it looks like you tried to provide -l release=trusty as an argument to your program rather than jsub
[01:59:46] <Zppix>	 Ive tried jstart ive tried jsub with -l release trusty ive tried jstart with the release arg
[01:59:54] <Krenair>	 the full command, Zppix 
[02:00:03] <Zppix>	 What should i donthen
[02:00:05] <Zppix>	 Do
[02:00:12] <Krenair>	 paste the full command
[02:01:08] <Zppix>	 I tried jstart -l release=trusty sopel
[02:01:20] <Krenair>	 and this resulted in?
[02:01:22] <Zppix>	 Jsub -l release=trusty sopel
[02:01:25] <Zppix>	 Same
[02:01:29] <Zppix>	 As it shows on task
[02:02:30] <Krenair>	 krenair@tools-bastion-03:~$ jstart -l release=trusty lsb_release -c
[02:02:31] <Krenair>	 Your job 234671 ("lsb_release") has been submitted
[02:02:31] <Krenair>	 krenair@tools-bastion-03:~$ cat lsb_release.out
[02:02:31] <Krenair>	 Codename:	trusty
[02:03:03] <Zppix>	 ??
[02:03:39] <Krenair>	 jsub does the same
[02:03:52] <Zppix>	 Does the codename mean it uses trusty
[02:03:55] <Zppix>	 Im confused
[02:04:00] <Krenair>	 yes
[02:04:03] <Zppix>	 Ok
[02:04:05] <Zppix>	 Let me see
[02:04:56] <Krenair>	 What is this 'sopel' anyway?
[02:05:39] <Krenair>	 I just get jstart: error: argument program: Program 'sopel' not found.
[02:06:44] <Zppix>	 Its in envs/bin/
[02:06:48] <Zppix>	 Its irc bt
[02:06:49] <Zppix>	 Bot
[02:07:05] <Zppix>	 See wikitech.wikimedia.org/wiki/tool:Zppixbot
[02:09:12] <Zppix>	 still doesnt work
[02:22:30] <Krenair>	 Zppix, I didn't change anything
[02:22:50] <Krenair>	 so it's not surprising that doing the same command still produces the same response
[02:25:42] <yuvipanda>	 !log tools restarted maintain-kubeusers
[02:25:47] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[02:27:33] <wikibugs>	 06Labs, 10Tool-Labs: New user home directory creation broken - https://phabricator.wikimedia.org/T149505#2754785 (10Krenair) 05Open>03Resolved a:03yuvipanda @yuvipanda restarted maintain-kubeusers on tools-k8s-master-01, it works again now: ```krenair@tools-bastion-03:~$ sudo become tts tools.tts@tools-b...
[02:28:45] <wikibugs>	 06Labs, 10Tool-Labs: become should have a better error message when homedir doesn't exist - https://phabricator.wikimedia.org/T149511#2754788 (10yuvipanda)
[03:39:41] <wikibugs>	 06Labs, 07Tracking: Existing Labs project quota increase requests (Tracking) - https://phabricator.wikimedia.org/T140904#2754813 (10Andrew)
[04:18:50] <wikibugs>	 10Labs-project-Wikistats: W3C wiki updates broken - https://phabricator.wikimedia.org/T149000#2754818 (10Dzahn) @hashar Thank you for this, but there is a difference between different tables. The "fixit" thing is for wikis in the "mediawikis" table, while it does not apply to "farm" tables likw W3C where we dont...
[04:37:37] <wikibugs>	 10Labs-project-Wikistats: W3C wiki updates broken - https://phabricator.wikimedia.org/T149000#2754824 (10Dzahn) So in this case we actually just had to switch http to https here:  https://gerrit.wikimedia.org/r/#/c/318776/1/usr/lib/wikistats/update.php  and that switches LXDE and Editthis Wikis to https
[04:39:48] <wikibugs>	 10Labs-project-Wikistats, 13Patch-For-Review: LXDE wiki updates broken - https://phabricator.wikimedia.org/T149396#2754825 (10Dzahn) @RobiH   yep, it's the protocol change. http -> https (did not change the format part)  https://gerrit.wikimedia.org/r/#/c/318776  https://gerrit.wikimedia.org/r/#/c/318776/2/usr...
[04:40:45] <wikibugs>	 10Labs-project-Wikistats: W3C wiki updates broken - https://phabricator.wikimedia.org/T149000#2754828 (10Dzahn)
[04:41:05] <wikibugs>	 10Labs-project-Wikistats, 13Patch-For-Review: LXDE wiki updates broken - https://phabricator.wikimedia.org/T149396#2750958 (10Dzahn) http://wikistats.wmflabs.org/display.php?t=lx
[06:46:52] <shinken-wm>	 PROBLEM - Puppet run on tools-services-01 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]
[06:51:20] <shinken-wm>	 PROBLEM - Puppet run on tools-webgrid-lighttpd-1411 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]
[07:26:52] <shinken-wm>	 RECOVERY - Puppet run on tools-services-01 is OK: OK: Less than 1.00% above the threshold [0.0]
[07:31:21] <shinken-wm>	 RECOVERY - Puppet run on tools-webgrid-lighttpd-1411 is OK: OK: Less than 1.00% above the threshold [0.0]
[12:24:17] <wm-bot>	 Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/GNegar was created, changed by GNegar link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/GNegar edit summary: Created page with "{{Tools Access Request |Justification=Persian auto correction tool |Completed=false |User Name=GNegar }}"
[12:28:14] <CreepyZombie>	 does anybode here have a working piece of code to connect a java-programm to a mysql/maria-db database?
[12:31:55] <paladox>	 CreepyZombie https://dev.mysql.com/downloads/connector/j/5.1.html
[12:32:35] <CreepyZombie>	 I'll check it out later, thx
[12:32:42] <paladox>	 Your welcome :)
[12:53:36] <shinken-wm>	 PROBLEM - Host tools-secgroup-test-102 is DOWN: CRITICAL - Host Unreachable (10.68.21.170)
[12:55:13] <wm-bot>	 Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/GNegar was modified, changed by Tim Landscheidt link https://wikitech.wikimedia.org/w/index.php?diff=939604 edit summary: 
[14:14:22] <wikibugs>	 10Striker, 05WMF-deploy-2016-10-11_(1.28.0-wmf.22), 05WMF-deploy-2016-10-25_(1.28.0-wmf.23): Check for 2FA protection and enforce validation of 2FA tokens - https://phabricator.wikimedia.org/T144712#2755023 (10Reedy)
[14:17:43] <shinken-wm>	 PROBLEM - Host tools-secgroup-test-103 is DOWN: CRITICAL - Host Unreachable (10.68.21.22)
[14:28:30] <shinken-wm>	 PROBLEM - Host secgroup-lag-102 is DOWN: CRITICAL - Host Unreachable (10.68.17.218)
[15:48:16] <andrewbogott>	 halfak, can you clean up your homedir on tools-bastion-02?  It's getting a bit crowded in there.  (Not an emergency, but, next few days would be good.)
[15:48:50] <andrewbogott>	 halfak: oh, wait, nevermind, I think I'm mis-identifying the problem
[15:54:28] <andrewbogott>	 (I briefly forgot that homedirs are on NFS and so not relevant to disk warnings :) )
[16:13:13] <paladox>	 !log tools.lolrrit-wm testing ssl connection to irc on grrrit-wm
[16:13:15] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lolrrit-wm/SAL
[16:24:34] <grrrit-wm>	 (03CR) 10Glaisher: "Only had a quick look and it looks okay code-wise but I'm not sure this the right approach. This seems like the sort of thing which should" [labs/tools/stewardbots] - 10https://gerrit.wikimedia.org/r/318229 (https://phabricator.wikimedia.org/T149265) (owner: 10Platonides)
[16:43:01] <grrrit-wm>	 (03Draft1) 10Paladox: Make grrrit-wm connection to freenode secure using ssl [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/318790 
[16:43:04] <grrrit-wm>	 (03Draft2) 10Paladox: Make grrrit-wm connection to freenode secure using ssl [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/318790 
[16:44:08] <paladox>	 !log tools.lolrrit-wm test is a success switching grrrit-wm to ssl and deploying https://gerrit.wikimedia.org/r/#/c/318790/ to switch it
[16:44:10] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lolrrit-wm/SAL
[16:44:20] <grrrit-wm>	 (03CR) 10Paladox: [C: 032] Make grrrit-wm connection to freenode secure using ssl [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/318790 (owner: 10Paladox)
[16:44:50] <grrrit-wm>	 (03Merged) 10jenkins-bot: Make grrrit-wm connection to freenode secure using ssl [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/318790 (owner: 10Paladox)
[17:46:18] <paladox>	 !log lolrrit-wm testing trying for a specifyied name instead of it comming up with random ones.
[17:46:19] <stashbot>	 Unknown project "lolrrit-wm"
[17:46:19] <stashbot>	 Did you mean to say "tools.lolrrit-wm" instead?
[17:46:29] <paladox>	 !log tools.lolrrit-wm testing trying for a specifyied name instead of it comming up with random ones.
[17:46:31] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lolrrit-wm/SAL
[18:10:33] <paladox>	 !nick
[18:10:56] <Luke_>	 paladox: ?
[18:11:25] <paladox>	 Luke_ just trying something, i found this https://github.com/matrix-org/matrix-appservice-irc/blob/master/config.sample.yaml#L196
[18:11:33] <paladox>	 I am deciding weather to try the libary :)
[18:13:26] <paladox>	 That's actually a good idea to support !nick.
[18:14:00] <paladox>	 I also want to get !restart-grrrit-wm supported to :)
[18:14:05] <paladox>	 Luke_ ^^
[18:14:43] <Luke_>	 why don't do it via shell? IRC-Commands make it easier to abuse, and you need more access management
[18:17:20] <paladox>	 Luke_ it is already supported via shell
[18:17:48] <paladox>	 But we need it easyer to restart it
[18:17:54] <paladox>	 without ssh into the instance
[18:18:16] <paladox>	 Oh your talking about the nick
[18:20:14] <Luke_>	 about every command from IRC
[18:24:03] <paladox>	 Oh, i would only support !restart, but maybe we could do what wm-bot does and only support it for certain user
[18:24:05] <paladox>	 users
[18:29:53] <Zppix>	 paladox: add a userlist for admins and trusted (wmf related cloaks)
[18:29:58] <Zppix>	 ?
[18:31:43] <Luke_>	 Zppix: this always exluce me :P
[18:32:08] <Zppix>	 Become special then maybe just kidsin
[18:32:11] <Zppix>	 Kidding
[18:32:25] <Krenair>	 you're eligible for a wikimedia cloak aren't you?
[18:33:42] <Luke_>	 yep, but I don't want to change :P
[18:34:23] <Luke_>	 since this cloak gives me access to a bunch of channels too
[18:34:34] <paladox>	 Ok
[18:34:44] <paladox>	 Luke_ https://github.com/draggor/node-ircbot
[18:34:55] <paladox>	 permission section
[18:39:52] <Zppix>	 Luke you can technically have 2 cloaks you know right
[18:40:30] <Luke_>	 Zppix: you mean like @tnb/staff/wikimedia.Luke081515?
[18:41:08] <Zppix>	 I dont know how it works
[18:41:14] <Zppix>	 I just know you can
[19:18:01] <halfak>	 o/ madhuvishy 
[19:18:04] <halfak>	 Around?
[19:18:26] <madhuvishy>	 halfak: yeah!
[19:18:38] <madhuvishy>	 \o
[19:18:47] <halfak>	 I saw andrewboggot's ping earlier about my homedir size on tools
[19:18:56] <halfak>	 It's 650MB.  Is that too much?
[19:20:28] <madhuvishy>	 halfak: aah, yeah that's about 16% of all of tools
[19:21:20] <halfak>	 650MB is 16% of all available space? o.O
[19:21:45] <halfak>	 I see 4.1T of 8T in use
[19:21:47] <madhuvishy>	 halfak: no. 8% of available space. we have 8T
[19:21:48] <madhuvishy>	 yeah
[19:22:00] <halfak>	 "MB"
[19:22:16] <madhuvishy>	 oh
[19:22:18] <madhuvishy>	 sorry
[19:22:21] <madhuvishy>	 i misread
[19:22:29] <madhuvishy>	 halfak: not at all, you're fine :)
[19:23:38] <halfak>	 You know, back when I started grad school, we had a 250MB quota on our NFS-based home directories. 
[19:23:52] <halfak>	 That was bananas.  It was 2006. 
[19:23:59] <madhuvishy>	 i think andrew thought it was in the local disk of tools-bastion, which have less space. 
[19:24:10] <madhuvishy>	 oh wow
[19:24:14] <halfak>	 Oh!  That makes sense.  
[19:25:12] <halfak>	 Oh well.  andrewbogott said he was mistaken, but I thought I'd check in anyway.  Thanks :) 
[19:25:14] <halfak>	 o/
[19:25:31] <madhuvishy>	 yw :)
[19:25:37] * halfak considers puppetizing snuggle today. 
[19:30:25] <halfak>	 madhuvishy, sorry one more thing.  What is your puppet development workflow when you are setting up a new project?
[19:31:44] <madhuvishy>	 halfak: mostly using self hosted puppet master on a labs instance, pushing to gerrit and testing on the labs instance. 
[19:32:30] <madhuvishy>	 you could also develop on the instance with vim, but have to setup gerrit there with ssh keys to push directly
[19:32:47] <halfak>	 Is there a good wiki page documenting how to do something like that?
[19:33:05] * halfak isn't clear on self-hosting a puppet master
[19:35:43] <madhuvishy>	 halfak: I don't think so. The way to set up own puppetmaster has changed a bit with yuvi's recent work on it too  - I ran into some trouble doing this yesterday and he said he would fix the docs. https://wikitech.wikimedia.org/wiki/Standalone_puppetmaster. 
[19:36:12] <madhuvishy>	 I think at the moment, make a new instance, apply role puppetmaster::standalone through Horizon (I can help with this), and then poke yuvipanda 
[19:36:31] <halfak>	 Bummer.  OK, we'll, maybe today I'll just take detailed notes on my migration and use that to build up puppet some other time. 
[19:36:58] <madhuvishy>	 okay, i'll try to put some docs together meanwhile
[19:37:47] <halfak>	 madhuvishy, was thinking that a mini-puppetizing-workshop might be a fun thing to do at a hackathon
[19:38:40] <madhuvishy>	 halfak: yeah for sure
[19:38:59] <madhuvishy>	 halfak: is there a reason why you aren't running snuggle on tools-kubernetes?
[19:39:18] <halfak>	 madhuvishy, snuggle predates tools-kubernetes quite a bit 
[19:40:15] <madhuvishy>	 ah, I guess my question is, have you considered running it on k8s, and made a conscious decision not to, and what the reasoning was
[19:41:02] <halfak>	 Oh.  Yeah.  Have not yet seriously considered.  I use MongoDB -- is that going to work out?
[19:41:17] <halfak>	 If I could have all the time, I'd probably spend some time on it. 
[19:41:19] <Zppix>	 Yes
[19:41:44] <Zppix>	 Wait
[19:41:47] <Zppix>	 No
[19:41:52] <Zppix>	 Sql i believe
[19:42:10] <Zppix>	 I get tools and mw confused
[19:42:12] <Zppix>	 Lol
[19:42:37] <halfak>	 :P 
[19:43:06] <madhuvishy>	 halfak: ah hmmm mongodb might be tricky
[19:43:16] <halfak>	 I use denormalization and Mongo's efficient update patterns to make a live-update problem tractable. 
[19:43:25] <halfak>	 So it might be a trick to move to something else. 
[19:43:32] <halfak>	 Honestly, I think redis would work fine. 
[19:43:38] <halfak>	 But it would require a lot of coding time :( 
[19:43:51] <halfak>	 redis would be far superior, to be honest :) 
[19:44:31] * halfak starts up a Jessie instance to replace an Ubuntu 12.04 instance. 
[19:44:32] <halfak>	 :D
[20:01:46] <madhuvishy>	 halfak: you could also run mongodb from a labs instance (mongo is already puppetized i think), and have snuggle talk to it from tools
[20:02:11] <madhuvishy>	 would require fairly less effort
[20:03:30] <Urbanecm>	 Hi, why is loading python interpreter at toollabs taking about 4 seconds? I tried to load it at another computer, it took only 20 miliseconds. 
[20:06:21] <madhuvishy>	 Urbanecm: tools runs on top of nfs - therefore while it looks like simple disk IO, there's a a bunch of talking over the network happening underneath
[20:06:53] <Urbanecm>	 Stats: dev.tools.wmflabs.org: 4 seconds; bastion.wmflabs.org: 20 miliseconds, masaryk.wikimedia.cz (this computers is under control of Wikimedia Czech Republic) 20 ms, Raspberry Pi: 191 miliseconds, login.tools.wmflabs.org: 2 seconds
[20:07:15] <madhuvishy>	 so performance on tools is not really comparable to local disk
[20:07:20] <Urbanecm>	 I think bastion.wmflabs.org use NFS too. Am I right?
[20:08:06] <madhuvishy>	 Urbanecm: no, that would be local too i think
[20:09:13] <madhuvishy>	 home directories on tools comes from nfs. not for the rest of labs 
[20:09:20] <Urbanecm>	 Okay, thanks. 
[20:09:28] <Urbanecm>	 So let's call it normal. 
[20:09:40] <Urbanecm>	 But is it okay too that jsub pwd runs for almost 11 seconds?
[20:12:13] <madhuvishy>	 Urbanecm: it's possible - a bunch of things need to happen for a job submitted to the grid to complete. It needs to gets scheduled on an instance first and then run. Plus, all of this is happening over nfs
[20:12:33] <Urbanecm>	 Okay. Could you tell me which things exactly?
[20:13:32] <madhuvishy>	 Urbanecm: https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Grid
[20:13:37] <Urbanecm>	 Thans. 
[20:13:39] <Urbanecm>	 *Thanks
[20:15:36] <halfak>	 I'm getting a weird error when setting up the /srv/ on a brand new instance (snuggle-enwiki-01.eqiad.wmflabs).  See https://gist.github.com/halfak/3b132a307445990295b99dd71ccc0e5f
[20:15:58] <halfak>	  /usr/local/sbin/nfs-mount-manager mount /home returned 32 instead of one of [0]
[20:16:29] <madhuvishy>	 halfak: does it persist if you rerun puppet?
[20:17:17] <halfak>	 Will try again in a moment and report
[20:21:03] <halfak>	 madhuvishy, error persists
[20:24:36] <halfak>	 Looks like I was using apache for snuggle O.O
[20:24:41] <halfak>	 yuck
[20:24:54] <madhuvishy>	 halfak: looking
[20:25:38] <halfak>	 madhuvishy, looks like I have something mounted there anyway. 
[20:25:41] <halfak>	 "/dev/mapper/vd-second--local--disk   21G   44M   19G   1% /srv"
[20:25:51] <halfak>	 ^ from "df -h"
[20:26:07] <madhuvishy>	 halfak: that's just your local disk
[20:26:21] <halfak>	 Hmm... I kind of just want local disk :) 
[20:26:36] <madhuvishy>	 ah, yeah - but the snuggle project has nfs mounted
[20:26:39] <halfak>	 was only worried about the /srv mount having some space for me to put stuff :D
[20:26:47] <madhuvishy>	 the nfs server hasn't picked up on the new instance yet
[20:26:49] <halfak>	 I want to turn that off. 
[20:26:53] <madhuvishy>	 so puppet is failing
[20:27:12] <madhuvishy>	 ah you don't want it at all? there are 2 instances i think
[20:27:59] <madhuvishy>	 halfak: if you'd like it turned off, that's easy to do - can you file a task I can take care of it
[20:28:05] <halfak>	 Yeah.  I don't use it at all. 
[20:28:05] <madhuvishy>	 probably tomorrow though
[20:28:30] <halfak>	 madhuvishy, gotcha.  I'll do that as soon as I have the migration working so that we can make sure I don't accidentally break anythign first :D
[20:34:05] <madhuvishy>	 halfak: run puppet now?
[20:34:11] <madhuvishy>	 should be fixed.
[20:34:32] <halfak>	 Thanks madhuvishy 
[20:39:59] <halfak>	 madhuvishy, I can't "sudo -u www-data ..."
[20:40:10] <halfak>	 I know there's a security setting to change for this, but I can't find it. 
[20:50:49] <madhuvishy>	 halfak: lojjik 
[20:50:52] <madhuvishy>	 uhh
[20:50:54] <madhuvishy>	 looking
[20:51:01] <madhuvishy>	 sorry for the wrong ping!
[20:51:49] <halfak>	 ha!
[20:52:04] <halfak>	 Thanks for helping me today :) 
[20:52:52] * halfak is trying to cram some volunteer hours in and appreciates madhuvishy doing the same. 
[20:53:54] <madhuvishy>	 halfak: iirc - it's supposed to be here https://wikitech.wikimedia.org/wiki/Special:NovaSudoer
[20:54:04] <madhuvishy>	 the page is really slow though
[20:54:11] <halfak>	 \o/ 
[20:54:14] <halfak>	 That looks familiar
[20:55:34] <madhuvishy>	 halfak: :) ya it doesn't load for me for some reason
[20:55:52] <madhuvishy>	 but wikilabels/wikimetrics should have settings for www-data if you wanna check
[21:02:03] <halfak>	 Got it working \o/
[21:09:52] <madhuvishy>	 yay
[21:22:11] <Terminator>	 is an admin online?
[21:22:26] <halfak>	 madhuvishy, ^ 
[21:22:40] <madhuvishy>	 Terminator: sup?
[21:27:49] <Terminator>	 madhuvishy: handling with krenair now, thanks :)
[21:28:23] <madhuvishy>	 okay :)
[21:43:31] <multichill>	 yuvipanda  / valhallasw`vecto : Mind giving http://wdq.wmflabs.org/ a nudge?
[21:47:51] <shinken-wm>	 PROBLEM - Puppet run on tools-services-02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]
[21:48:10] <yuvipanda>	 !log wdq-mm restart wdq-01.wdq-mm.eqiad.wmflabs, instance was unsshable
[21:48:10] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wdq-mm/SAL
[21:49:27] <yuvipanda>	 multichill: it seems to be performing a lot of IO, I guess it's trying to start up?
[21:55:04] <yuvipanda>	 multichill: do you know how long it's been down?
[21:55:38] <yuvipanda>	 multichill: logs say it might've been down since... june?!
[21:55:46] <yuvipanda>	 that sounds wrong
[21:55:49] <yuvipanda>	 multichill: anyway, is back up now
[21:56:16] <multichill>	 yuvipanda: I would say since last round of reboots?
[21:57:48] <yuvipanda>	 multichill: yeah, not entirely surprised I guess
[21:57:58] <yuvipanda>	 multichill: do you have any idea what % of tools are still using it vs wdqs?
[21:58:03] <multichill>	 I just noticed because I forgot one of my bots still used it
[21:58:16] <multichill>	 Switched it to the other system right away
[21:58:48] <yuvipanda>	 ah nice :D
[22:03:59] <halfak>	 I'm not sure why www-data seems to not have permission to bind to port 80 on jessie. 
[22:04:07] <halfak>	 Anyone know what might be up with that. 
[22:05:20] <yuvipanda>	 hey halfak 
[22:05:29] <halfak>	 Hey yuvipanda 
[22:05:37] <yuvipanda>	 www-data usually doesn't have permission to bind to port 80 anywhere. usually nginx / apache spawn as root, bind, then fork
[22:06:11] <halfak>	 Gotcha.  Hmm.  For this system, I don't have nginx or apache.  I was hoping to just use uwsgi directly. 
[22:06:35] <halfak>	 Maybe I could do my routing to 8080 in the dns
[22:06:51] <yuvipanda>	 halfak: yeah
[22:06:58] <yuvipanda>	 uwsgi has the --http thing
[22:07:05] <yuvipanda>	 easiest is to make it listen on 8080 yes
[22:07:36] <yuvipanda>	 I've fiddled with it before and didn't find an easy way for uwsgi to bind to 80 as root and then fork
[22:07:42] <yuvipanda>	 but I didn't spend more than 30min looking
[22:08:45] <halfak>	 Yeah.  That's what a quick googling showed me too
[22:08:49] <halfak>	 Thanks this will work
[22:10:34] <yuvipanda>	 halfak: cool. there are ways to hack around it, but not worth it imo
[22:27:49] <shinken-wm>	 RECOVERY - Puppet run on tools-services-02 is OK: OK: Less than 1.00% above the threshold [0.0]
[23:09:39] <halfak>	 OK.  Half-way through the migration.  Calling it quits for today.  Sync server was started and that will be a good test. 
[23:10:01] <halfak>	 !log snuggle started up snuggle-enwiki-01.  syncd running
[23:10:03] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Snuggle/SAL