[13:03:58] <halfak|Mobile>	 Just saw this on quarry.  Running a demo now so I don't have time to file a task:
[13:04:01] <halfak|Mobile>	 Can't connect to MySQL server on 'enwiki.analytics.db.svc.eqiad.wmflabs' ((1226, u"User 's52788' has exceeded the 'max_user_connections' resource (current value: 10)"))
[14:00:37] <wikibugs>	 (03PS1) 10Dzahn: passwords: update labs root key for Daniel [labs/private] - 10https://gerrit.wikimedia.org/r/390240
[14:01:49] <wikibugs>	 (03CR) 10Dzahn: [C: 04-1] passwords: update labs root key for Daniel [labs/private] - 10https://gerrit.wikimedia.org/r/390240 (owner: 10Dzahn)
[14:06:45] <wikibugs>	 (03PS2) 10Dzahn: passwords: update labs root key for Daniel [labs/private] - 10https://gerrit.wikimedia.org/r/390240
[15:08:01] <wikibugs>	 (03CR) 10BryanDavis: passwords: add labs key for arturo (031 comment) [labs/private] - 10https://gerrit.wikimedia.org/r/390027 (owner: 10Arturo Borrero Gonzalez)
[15:14:26] <bd808>	 halfak|Mobile: I filed T180141 based on your message. Thanks for letting us know.
[15:14:27] <stashbot>	 T180141: Raise concurrent mysql connection limit for Quarry (or throttle application concurrency) - https://phabricator.wikimedia.org/T180141
[15:18:51] <bd808>	 !log tools.himo Killed `python pwb.py c18/scat -MakeNewCat noprint t:test -new2018 test:Guayaquil test:Hamburg` job running directly on tools-bastion-03. See https://wikitech.wikimedia.org/wiki/Help:Toolforge/Grid for instructions on running jobs on the grid.
[15:18:53] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.himo/SAL
[15:19:37] <bd808>	 !log tools.himo Killed `python pwb.py c18/scat -addlabel noprint -ssubcat:Category:Treaties_by_country` job running directly on tools-bastion-03. See https://wikitech.wikimedia.org/wiki/Help:Toolforge/Grid for instructions on running jobs on the grid.
[15:19:38] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.himo/SAL
[15:29:37] <wikibugs>	 (03PS2) 10Arturo Borrero Gonzalez: passwords: add labs key for arturo [labs/private] - 10https://gerrit.wikimedia.org/r/390027
[15:37:05] <wikibugs>	 (03CR) 10BryanDavis: [C: 031] passwords: add labs key for arturo [labs/private] - 10https://gerrit.wikimedia.org/r/390027 (owner: 10Arturo Borrero Gonzalez)
[16:16:52] <madhuvishy>	 halfak|Mobile: thanks, I'll follow up in a bit
[17:18:24] <wikibugs>	 (03PS1) 10Chad: git.wm.o -> phab.wm.o [labs/toollabs] - 10https://gerrit.wikimedia.org/r/390276
[17:29:28] <wikibugs>	 (03CR) 10Madhuvishy: [V: 032 C: 032] passwords: add labs key for arturo [labs/private] - 10https://gerrit.wikimedia.org/r/390027 (owner: 10Arturo Borrero Gonzalez)
[18:40:27] <DMaza>	 what's the usual wait time 'til someone reviews your request to access toolforge? :)
[18:41:07] <Zppix>	 DMaza: depends if someone has the time or not but i know my request albeit some time ago was approx 24hrs
[18:41:32] <Krenair>	 https://toolsadmin.wikimedia.org/tools/membership/status/154
[18:41:35] <Krenair>	 two others open
[18:41:43] <Zppix>	 Im sure if you ask nicely someones willing to take some looks
[18:41:45] <Krenair>	 I think someone in the tools.admin group has to approve it
[18:41:56] <Zppix>	 Krenair: this is correct
[18:43:08] <DMaza>	 Zppix: I figured if it hasn't been reviewed is 'cause there are other priorities :). Who can I very poletely ask?
[18:43:21] <DMaza>	 *politely 
[18:43:58] <madhuvishy>	 DMaza: I'm looking now
[18:44:18] <DMaza>	 madhuvishy: thank you very much
[18:44:49] <Zppix>	 DMaza: the wonders of irc :)
[18:45:02] <DMaza>	 hehe
[18:45:18] <Zppix>	 DMaza: what do you plan on working on?
[18:46:21] <DMaza>	 https://tools.wmflabs.org/interaction-timeline/
[18:46:39] <madhuvishy>	 DMaza: I approved you
[18:46:46] <DMaza>	 already helped on it a bit.. and we'll be needing an api for it to query the DB
[18:46:54] <Zppix>	 DMaza: welcome to the toolforge family :)
[18:47:00] <DMaza>	 thank you 
[18:47:22] <Zppix>	 Feel free to bug us if you run into issues
[18:47:44] <DMaza>	 Zppix: thanks a lot :)
[18:48:48] <Zppix>	 Np
[18:58:58] <wikibugs>	 (03CR) 10BryanDavis: [C: 032] git.wm.o -> phab.wm.o [labs/toollabs] - 10https://gerrit.wikimedia.org/r/390276 (owner: 10Chad)
[18:59:50] <wikibugs>	 (03Merged) 10jenkins-bot: git.wm.o -> phab.wm.o [labs/toollabs] - 10https://gerrit.wikimedia.org/r/390276 (owner: 10Chad)
[19:59:18] <legoktm>	 valhallasw`cloud: any opinions on hiding User-XX projects from wikibugs output? (unless they're the reason for the notification I suppose)
[19:59:36] <valhallasw`cloud>	 legoktm: sounds reasonable to me
[20:00:06] <valhallasw`cloud>	 legoktm: isn't there already a limit to the number of projects (or the total length of that part?). In that case, we could just sort them as last
[20:01:15] <legoktm>	 I only noticed because it ends up causing the comment to be truncated more than if it the project didn't exist
[20:01:15] <legoktm>	 e.g.
[20:01:16] <legoktm>	 [11:17:20] <wikibugs> Cloud-Services, DBA, User-Jayprakash12345, cloud-services-team (Kanban): Prepare and check storage layer for hif.wiktionary - https://phabricator.wikimedia.org/T173647#3748834 (Marostegui) Multiples, it has been commented many times when it arises but we are not actively troubleshooting it as it...
[20:01:44] <valhallasw`cloud>	 yeah, that's not great
[20:02:10] <valhallasw`cloud>	 and I don't think the User-X really adds much to the notification
[20:02:11] <Zppix>	 I am okay with that (i use a person user- project)
[20:02:26] <Zppix>	 Personal*
[20:11:00] <e64b1>	 :-/
[20:23:16] <matanya>	 madhuvishy: x tools seems down, can you check please ?
[20:23:41] <madhuvishy>	 musikanimal: ^ do you maintain xtools?
[20:24:05] <Zppix>	 Xtools i am able to access
[20:24:13] <Zppix>	 Online
[20:24:17] <Zppix>	 Not as in maintain
[20:24:28] <bd808>	 matanya: any particular tool in that pile of stuff?
[20:24:34] <Zppix>	 Let me just rephrase i am able to get onto xtools.wmflabs
[20:24:38] <musikanimal>	 I do
[20:24:59] <musikanimal>	 it seems to be working for me?
[20:25:03] <matanya>	 https://tools.wmflabs.org/xtools/adminstats
[20:25:23] <Zppix>	 musikanimal: adminstats not working
[20:25:23] <matanya>	 i just noticed it is a pile :)
[20:25:26] <musikanimal>	 oh, so the old XTools is currently freezing up and not redirecting
[20:25:37] <musikanimal>	 check out https://xtools.wmflabs.org/adminstats
[20:25:54] <matanya>	 thanks madhuvishy
[20:26:03] <matanya>	 and musikanimal  and bd808
[20:26:16] <musikanimal>	 I'll restart the old one. All it does is forward the request, so not sure why it's down
[20:26:17] <matanya>	 and Zppix ;)
[20:26:34] <Zppix>	 matanya: thanks for helping me figure out my english i speak natively apparently needs fixed
[20:26:34] <matanya>	 i will fix the redirecting link
[20:26:53] <musikanimal>	 yeah please do :)
[20:27:07] <matanya>	 Zppix: :D
[20:28:01] <bd808>	 musikanimal: is the redirect that you are running at the tool account "simple"? Like just some lighttpd rewrite rules?
[20:28:08] <musikanimal>	 yeah
[20:28:36] <musikanimal>	 there's actually 2 tools still running on that account, so maybe they messed up and brought the whole thing down
[20:28:43] <bd808>	 maybe you and I should work up a patch for the toolforge proxy itself that handles that redirect
[20:28:57] <bd808>	 ah. still some bits that haven't moved?
[20:29:03] <musikanimal>	 yeah that sounds good!
[20:29:04] <musikanimal>	 http://tools.wmflabs.org/xtools/rfap/
[20:29:11] <musikanimal>	 and https://tools.wmflabs.org/xtools/rfa/
[20:29:17] <musikanimal>	 which I'm not sure we even want to keep
[20:29:20] <musikanimal>	 they're very enwiki
[20:29:47] <musikanimal>	 and dewiki, I guess
[20:29:57] <matanya>	 kill them all, global is what we want
[20:30:39] <musikanimal>	 well, they've been partialling rewritten, but to me XTools is supposed to work on every wiki (MediaWiki, that is), out of the box
[20:31:15] <musikanimal>	 maybe we'll finish them at some point, but probably won't expose them in the nav
[20:31:42] <musikanimal>	 for now we should keep the old ones running
[20:32:02] <musikanimal>	 or maybe move them to a different, dedicated tools
[20:32:54] <bd808>	 the modules/dynamicproxy/templates/urlproxy.conf file in operations/puppet.git is where we could put rewrites
[20:33:43] <bd808>	 those sub tools could get moved to their own tool projects too in theory
[20:34:04] <bd808>	 I really dislike the "giant pile of tools" model
[20:34:31] <bd808>	 but I know it makes some lkinds of code sharing easier
[20:37:03] <tom29739>	 Hi, is it possible to get a newer version of Ubuntu on cloud VPS?
[20:37:05] <musikanimal>	 I gotta be honest I don't understand how urlproxy.conf works, but if we could add the XTools redirects that'd be great
[20:37:23] <musikanimal>	 as for rfa and rfap, I hope we do move them to separate tool projects
[20:40:37] <valhallasw`cloud>	 tom29739: no -- it's trusty, or a debian distribution.
[20:41:19] <valhallasw`cloud>	 tom29739: it's likely that the thing you need is available on debian as well, though
[20:43:16] <tom29739>	 valhallasw`cloud, thanks. which is the newest debian image on there?
[20:44:41] <valhallasw`cloud>	 tom29739: debian-9.1-stretch
[21:14:03] <Zppix>	 Theres no way to connect to cloud vos instances without in someway modifying your ssh config is there i am asking cause i am having to use an ios shell application and i obviosusly cannot modify a config file
[21:17:44] <bd808>	 instances with static ips are exposed (like login.tools.wmflabs.org), but generally yes you need to have an ssh client that can use a jump host
[21:18:32] <Zppix>	 I cannot go on bastion and connect from there similar to login.tools? bd808
[21:18:57] <bd808>	 yes, if you have agent forwarding
[21:19:24] <Zppix>	 That would require config modify?
[21:19:56] <bd808>	 *shrug* it requires agent forwarding and that depends on your ssh client
[21:20:38] <bd808>	 Zppix: http://www.unixwiz.net/techtips/ssh-agent-forwarding.html
[21:20:55] <Zppix>	 I found a client for ios that does that
[21:20:56] <Zppix>	 Ty
[21:24:25] <Zppix>	 Well i thought i did oh well thanks anyway
[21:41:12] <wikibugs>	 (03Draft2) 10Zppix: setting up a new puppetdb host [labs/icinga2] - 10https://gerrit.wikimedia.org/r/390320
[21:44:06] <wikibugs>	 (03CR) 10Paladox: [V: 032 C: 032] setting up a new puppetdb host [labs/icinga2] - 10https://gerrit.wikimedia.org/r/390320 (owner: 10Zppix)
[21:49:16] <paladox>	 Nov  9 21:48:52 gerrit-mysql icinga2[28129]: [2017-11-09 21:48:52 +0000] critical/config: Error: Validation failed for object 'puppetdb-phab!ping4' of type 'Service'; Attribute 'host_name': Object 'puppetdb-phab' of type 'Host' does not exist.
[21:49:19] <paladox>	 uh woops
[21:49:22] <paladox>	 wrong channel
[21:50:33] <wikibugs>	 (03Draft1) 10Paladox: Fix syntax error [labs/icinga2] - 10https://gerrit.wikimedia.org/r/390327
[21:50:35] <wikibugs>	 (03Draft2) 10Paladox: Fix syntax error [labs/icinga2] - 10https://gerrit.wikimedia.org/r/390327
[21:50:54] <wikibugs>	 (03CR) 10Zppix: [V: 032 C: 032] Fix syntax error [labs/icinga2] - 10https://gerrit.wikimedia.org/r/390327 (owner: 10Paladox)
[21:57:38] <Zppix>	 !log git added a new host, puppetdb-phabricator.phabricator.eqiad.wmflabs, as it is apart of a currently moderated project, and per maintainer request. Done 3:55 CST UTC-6
[21:57:40] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL
[21:57:48] <Zppix>	 paladox: ^ fyi
[21:58:11] <paladox>	 thanks
[21:58:17] <paladox>	 wrong project
[21:58:23] <Zppix>	 No its not
[21:58:35] <Zppix>	 That was for the project icinga2 is on
[21:58:51] <Zppix>	 I dont know how phab project perfers there log msgs
[21:59:12] <paladox>	 ah ok
[22:05:36] <wikibugs>	 (03PS1) 10TerraCodes: git.wikimedia.org -> phab [labs/tools/lists] - 10https://gerrit.wikimedia.org/r/390329 (https://phabricator.wikimedia.org/T139089)
[22:20:15] <XioNoX>	 is there a policy/process to have a new image in openstack/horizon?
[22:21:58] <wikibugs>	 (03Draft2) 10Zppix: Config [labs/icinga2] - 10https://gerrit.wikimedia.org/r/390333
[22:22:34] <wikibugs>	 (03PS3) 10Paladox: Config [labs/icinga2] - 10https://gerrit.wikimedia.org/r/390333 (owner: 10Zppix)
[22:22:36] <XioNoX>	 and more instances in my project? :)
[22:22:37] <wikibugs>	 (03CR) 10Paladox: [V: 032 C: 032] Config [labs/icinga2] - 10https://gerrit.wikimedia.org/r/390333 (owner: 10Zppix)
[22:23:13] <Zppix>	 If you met your quota  you can request a limit increase
[22:24:32] <bd808>	 XioNoX: are you wanting a new "flavor" (CPU/RAM/Disk) or a different base image for an instance?
[22:25:12] <XioNoX>	 bd808: different base image, Juniper vSRX and/or vMX
[22:25:25] <bd808>	 oh interesting...
[22:25:49] <XioNoX>	 vSRX looks easy, vMX looks more difficult (based on the lenght of the doc)
[22:25:50] <bd808>	 make a phab task and we can try to work out if it is possible in our setup
[22:26:08] <XioNoX>	 https://www.juniper.net/documentation/en_US/vsrx/topics/task/multi-task/security-vsrx-contrail-installing.html
[22:26:08] <XioNoX>	 https://www.juniper.net/documentation/en_US/vmx17.2/information-products/pathway-pages/getting-started/vmx-gsg-openstack.pdf
[22:26:09] <bd808>	 Our OpenStack is a bit weird
[22:26:14] <XioNoX>	 cool, will do
[22:26:55] <XioNoX>	 where do I request a limit increase for a project?
[22:26:58] <XioNoX>	 phab as well?
[22:27:11] <bd808>	 yeah, let me give you a link
[22:27:28] <bd808>	 the process is described at https://phabricator.wikimedia.org/project/view/2880/
[22:27:45] <bd808>	 with a handy link that gives you a template task to fill out
[22:28:58] <XioNoX>	 great!
[22:43:50] <paladox>	 Hi, im wondering how can i use puppetdb in labs please?
[22:44:13] <paladox>	 I've applied the role, which applied successfully
[22:44:14] <paladox>	 but not sure what to do from there
[22:47:33] <wikibugs>	 (03CR) 10Chad: [C: 031] git.wikimedia.org -> phab [labs/tools/lists] - 10https://gerrit.wikimedia.org/r/390329 (https://phabricator.wikimedia.org/T139089) (owner: 10TerraCodes)
[23:14:21] <madhuvishy>	 paladox: we don't really use puppetdb in VPS, not super sure how exactly it is set up in prod, you might have better luck on #operations on how they actually set it up!
[23:14:32] <paladox>	 ok thanks
[23:46:13] <XioNoX>	 bd808: https://phabricator.wikimedia.org/T180179
[23:46:50] <XioNoX>	 and https://phabricator.wikimedia.org/T180178
[23:52:05] <madhuvishy>	 XioNoX: thanks for filing, the quota requests usually get discussed and approved on our Tuesday morning meetings, and since half the team is out this week, the other one too probably at the same time