[00:30:25] <bd808>	 !log tools.meetbot Killing bot to see if both irc attachments disappear
[00:30:27] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.meetbot/SAL
[00:32:24] <bd808>	 !log tools.meetbot Restarted bot according to instructions at [[Tool:Meetbot]] (T186040)
[00:32:26] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.meetbot/SAL
[00:32:26] <stashbot>	 T186040: Meetbot is frequently logged in twice - https://phabricator.wikimedia.org/T186040
[11:42:08] <Adityaj>	 !help
[11:42:08] <wm-bot5>	 Adityaj: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team
[11:47:47] <Adityaj>	 I am not able to run this command "ssh login.tools.wmflabs.org" on ubuntu 16.04. Getting timed out. Can't figure out what is going wrong. I am getting the following error message:  "ssh: connect to host login.tools.wmflabs.org port 22: Connection timed out"
[11:52:31] <chicocvenancio>	 Adityaj: hello
[11:52:56] <chicocvenancio>	 what is your shell user?
[11:54:29] <Adityaj>	 thanks for the reply
[11:54:39] <Adityaj>	 my shell username is AdityaJ
[12:00:05] <chicocvenancio>	 ok, I don't see attempts from you in the log
[12:00:16] <chicocvenancio>	 how is your ssh configured?
[12:01:28] <chicocvenancio>	 do you have something in your  `~/.ssh/config´
[12:01:39] <chicocvenancio>	  `~/.ssh/config`
[12:02:47] <chicocvenancio>	 Adityaj: ^?
[12:09:08] <Adityaj>	 how can I know that I have something in `~/.ssh/config´? I am actually new to this so please help me.
[12:09:49] <chicocvenancio>	 Sure, no problem 
[12:10:33] <chicocvenancio>	 You can use any editor or viewer on that file and see if it returns something 
[12:11:15] <chicocvenancio>	 Such as `less ~/.ssh/config`
[12:12:05] <chicocvenancio>	 did you see this? https://wikitech.wikimedia.org/wiki/Help:Access#Accessing_Toolforge_instances
[12:12:29] <chicocvenancio>	 you probably only have to send the username and the private key in the command line itself
[12:12:34] <chicocvenancio>	 Adityaj: ^
[12:21:00] <AdityaJ>	 !help
[12:21:00] <wm-bot5>	 AdityaJ: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team
[12:22:26] <AdityaJ>	 I guess it means I don't have any config file in .ssh?
[13:03:18] <wm-bot5>	 I will let you know when I see AdityaJ and I will deliver that message to them
[13:03:18] <chicocvenancio>	 @notify AdityaJ You probably are not setting your shell username and your private key to connect. You can do that by setting the `-i` flag to ssh and prepending your username  and an `@` to the address like: `ssh -i /PATH/TO/YOUR/KEY adityaj@login.tools.wmflabs.org`
[14:02:37] <arturo>	 !log tools T188994 upgrading trusty-tools packages in all the cluster, this includes jobutils, openssh-server and openssh-sftp-server
[14:02:43] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[14:02:43] <stashbot>	 T188994: toolforge: package upgrades as part of the new workflow - https://phabricator.wikimedia.org/T188994
[14:41:54] <bd808>	  "ssh: connect to host login.tools.wmflabs.org port 22: Connection timed out" -- that pretty obviously is a network routing issue of some kind. Maybe a restrictive egress firewall on the origin network?
[14:57:14] <arturo>	 bd808:  is that for me?
[14:58:04] <bd808>	 arturo: no it was a drive-by on something way back in the backscroll between chicocvenancio and Adityaj
[14:58:47] <chicocvenancio>	 bd808: I think ssh gives you that for a lot of errors
[14:59:01] <bd808>	 not in my experience, but ok
[14:59:49] <chicocvenancio>	 There were a few attempts by the same ip at the time, could be port scanning
[15:11:11] <chicocvenancio>	 But you're probably right 
[15:14:06] <wm-bot5>	 You've already asked me to watch this user
[15:14:06] <chicocvenancio>	 @notify AdityaJ it may be that your network is blocking these connections. Ping me when you get back so we can test that. 
[16:01:38] <mmecor>	 hello! quick question. I can't seem to find the wikireplica of be-tarask
[16:01:54] <mmecor>	 _mysql_exceptions.OperationalError: (2005, "Unknown MySQL server host 'be_taraskwiki.analytics.db.svc.eqiad.wmflabs' (0)")
[16:02:02] <mmecor>	 i thought it was the hyphen but it isn't
[16:03:36] <chasemp>	 mmecor: underscores are not valid in DNS I believe
[16:03:47] * chicocvenancio looking it up, mmecor 
[16:04:25] <mmecor>	 i also tried to connect to it manually with no success
[16:04:26] <mmecor>	 sql be_taraskwiki_p;
[16:04:37] <zhuyifei1999_>	 mmecor: sql be[tab][tab]
[16:04:59] <zhuyifei1999_>	 you'll see all the wikis starting with 'be'
[16:05:01] <bd808>	 fancy tab completion is fancy :)
[16:05:13] <chicocvenancio>	 :) 
[16:05:25] <chasemp>	 !log tools tools-clushmaster-01:~$ clush -g all 'sudo puppet agent --test'
[16:05:30] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[16:05:37] <mmecor>	 it does not exist then
[16:05:41] <paladox>	 !help hi, puppet seems to be failing again 
[16:05:41] <wm-bot5>	 paladox: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team
[16:05:44] <mmecor>	 how come?
[16:05:44] <paladox>	 15:38 icinga2-wm: PROBLEM - puppet on puppet-paladox.git.eqiad.wmflabs is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues
[16:05:54] <andrewbogott>	 paladox: it was transient, should be back by now
[16:06:00] <paladox>	 Ah thanks 
[16:06:00] <bd808>	 paladox: actual details are helpful
[16:06:04] <zhuyifei1999_>	 mmecor: no, be-tarask was originally be-x-old
[16:06:06] <andrewbogott>	 wikitech was off for a bit
[16:06:07] <chicocvenancio>	 mmecor: what is the wiki address? 
[16:06:16] <zhuyifei1999_>	 before be-tarask was a code
[16:06:23] <paladox>	 bd808: not sure what you mean?
[16:06:28] <zhuyifei1999_>	 there was a ticket about this
[16:06:55] <bd808>	 paladox: "puppet seems to be failing again" does not tell us what project, what instance, what operating system, or what error.
[16:06:57] <chicocvenancio>	 zhuyifei1999_: thanks I would have probably taken a long time to learn that
[16:07:03] <zhuyifei1999_>	 https://phabricator.wikimedia.org/T11823
[16:07:06] <mmecor>	 there is be_x_oldwiki database chicocvenancio
[16:07:12] <mmecor>	 but i'm not sure it is be-tarask
[16:07:24] <mmecor>	 i extract the wikicodes from wikidata
[16:07:27] <paladox>	 Oh I see
[16:07:41] <mmecor>	 https://www.wikidata.org/wiki/Q2087886
[16:07:58] <mmecor>	 where the code for that wiki is be-tarask
[16:08:00] <paladox>	 bd808: the error message had the instance that I gave :),
[16:08:01] <chicocvenancio>	 mmecor: yeah, it is the correct db for now
[16:08:08] <mmecor>	 which is the one i use to connect to wikireplicas
[16:08:18] <mmecor>	 chicocvenancio: so, i set an equivalence for that particular wiki, ok :)
[16:08:19] <mmecor>	 thanks
[16:08:20] <chicocvenancio>	 It should be renamed, but hasn't yet
[16:08:23] <zhuyifei1999_>	 I'm not sure why the database isn't renamed, but you might want to file a ticket about that
[16:08:23] <paladox>	 bd808: I just needed to include the os, but I also left an error message
[16:10:09] <bd808>	 paladox: heh. my eyes filtered out your past of an icinga alert because they are 90% noise
[16:10:14] <bd808>	 *paste
[16:10:56] <paladox>	 Oh
[18:28:43] <Ester>	 chicocvenancio: reguarding the node app, etytree, it works now but I have problems managing the history - sorry not sure how to explain - so for instance I cannot seem to be able to go to page https://tools.wmflabs.org/etytree/label=door&lang=eng&ety=1 
[18:29:06] <Ester>	 while in development I run "npm start" and it works
[18:29:31] <chicocvenancio>	 Ester: atm it gives me 502 for any url
[18:30:01] <chicocvenancio>	 Ester: your tool doesn't need node outside development, as bd80.8 later pointed out to me
[18:30:29] <chicocvenancio>	 (it can use it, to make changes appear quicker, but we can disable that)
[18:30:56] <chicocvenancio>	 Ester: never mind not needing node
[18:31:06] <chicocvenancio>	 that was another tool, and I need more coffee
[18:32:11] <Ester>	 no worries
[18:32:17] <Ester>	 not sure what's going on
[18:32:44] <Ester>	 sorry first try
[18:33:18] <chicocvenancio>	 its not running now, right?
[18:33:28] <Ester>	 ok
[18:33:31] <Ester>	 now it's running
[18:33:33] <Ester>	 and you can see
[18:33:44] <Ester>	 if you use it
[18:33:50] <chicocvenancio>	 how are you running it?
[18:33:52] <Ester>	 and enter word door in the search bar
[18:34:13] <Ester>	 an press ok
[18:34:16] <Ester>	 and press ok
[18:34:23] <Ester>	 you get something
[18:34:29] <Ester>	 and the link is https://tools.wmflabs.org/etytree/label=door
[18:34:54] <Ester>	 but if you go to page  https://tools.wmflabs.org/etytree/label=door it gives 404 
[18:35:07] <Ester>	 while it should work
[18:36:08] <chicocvenancio>	 Ester: it is not running
[18:37:21] <chicocvenancio>	 you need to run `webservice --backend=kubernetes php5.6 start`
[18:37:37] <chicocvenancio>	 so it will serve the index.html and bundle.js files
[18:54:52] <chicocvenancio>	 Ester: did you understand how your tool should run
[18:55:51] <Ester>	 I tried with webservice --backend=kubernetes php5.6 start
[18:56:04] <Ester>	 not sure why it's not working
[18:56:16] <chicocvenancio>	 ok, now I get the 404
[18:56:25] <chicocvenancio>	 but the main page is working
[18:56:29] <Ester>	 yes
[18:58:19] <chicocvenancio>	 Ester: this is the problem I was mentioning to you the last time
[18:58:57] <chicocvenancio>	 there must be some for lighttpd to know what it should serve
[19:00:10] <chicocvenancio>	 there are several options for this
[19:19:56] <Ester>	 chicocvenancio: should I go back and try the different options you guys suggested?
[19:20:23] <Ester>	 I thought that putting all files in the public html folder would work
[19:21:23] <chicocvenancio>	 it does, until you go to a page beyond it
[19:21:42] <chicocvenancio>	 I think the best solution is to use url parameters instead of pages
[19:21:54] <bd808>	 Ester: how is this supposed to work? Is there a central dispatch script that /label=door should route through?
[19:22:13] <chicocvenancio>	 bd808: its using page.js for clientside routing
[19:22:34] <Ester>	 it's not using page.js anymore
[19:22:40] <bd808>	 it should be something like https://tools.wmflabs.org/etytree/#!label=door
[19:22:48] <chicocvenancio>	 bd808:  but lighttpd doesn't see a label=door file and fails
[19:22:58] <chicocvenancio>	 bd808: or https://tools.wmflabs.org/etytree/?label=door&lang=eng&ety=1 
[19:23:35] <bd808>	 sure. the point being that the URLs it puts in the browser search bar should actually be valid
[19:24:07] <bd808>	 would it work if all URLs responded with the index.html or whatever the default page it?
[19:24:09] <bd808>	 *is?
[19:24:13] <chicocvenancio>	 yes
[19:24:29] <bd808>	 ok. let me find an example of that...
[19:24:43] <chicocvenancio>	 responding 404 with index.html
[19:26:48] <bd808>	 tools.sal has this in .lighttpd.conf -- url.rewrite-if-not-file += ( "(.*)" => "/sal/index.php/$0" )
[19:27:14] <bd808>	 that makes everything route through the central dispatch script
[19:27:28] <chicocvenancio>	 but that redirects, doesn't it?
[19:27:38] <chicocvenancio>	 js would not have the original url, I think
[19:28:02] <bd808>	 it's not a redirect, its a server side rewrite
[19:28:27] <bd808>	 it makes lighttpd respond like that was the orginal request
[19:28:41] <chicocvenancio>	 cool, better than my alternative
[19:29:09] <bd808>	 I think this would work: url.rewrite-if-not-file += ( "(.*)" => "/etytree/index.html" )
[19:29:41] <bd808>	 that pretty much makes the 404 response into serving the index file
[19:29:57] <Ester>	 and the js reads the input string in the url?
[19:29:58] <chicocvenancio>	 it actually becomes a 200 response
[19:30:09] <bd808>	 chicocvenancio: correct
[19:30:20] <bd808>	 Ester: that's up to your js
[19:30:55] <bd808>	 but yes, it would be in the URL as seen by the javascript once the browser laoded it
[19:32:13] <Ester>	 so should I edit the .lighttpd.conf file?
[19:32:18] <chicocvenancio>	 yes
[19:33:14] <Ester>	 do I need to restart something?
[19:34:16] <Ester>	 great working
[19:34:20] <Ester>	 thank you guys
[19:34:34] <chicocvenancio>	 yes
[19:34:47] <chicocvenancio>	 `webservice restart`should be enough
[19:35:33] <Ester>	 chicocvenancio: thanks, great
[19:35:45] <Ester>	 bd808: thanks, great :)
[19:36:23] <bd808>	 (javascript routers using the path and not the fragment always seems wrong to me)
[19:37:01] <Ester>	 I see, can you point me to some documentation?
[19:37:25] <bd808>	 https://en.wikipedia.org/wiki/URL#Syntax
[19:38:10] <bd808>	 what I mean is that your javascript is preserving state in the URL (which is great) but doing it in a way that requires special handling from the backing server
[19:38:48] <bd808>	 if your url looked like https://tools.wmflabs.org/etytree/#label=door or https://tools.wmflabs.org/etytree/#!label=door or https://tools.wmflabs.org/etytree/?abel=door then this would have just worked
[19:39:25] <bd808>	 the # (fragment) and ? (query) components would not have changed the path lookup on the server side
[19:39:51] <Ester>	  do you mean https://tools.wmflabs.org/etytree/?label=door
[19:40:09] <Ester>	 I see, so I just put a question mark
[19:41:35] <bd808>	 yes, and then in your javascript you would read and write the query string. That's exposed as window.location.search
[19:41:48] <Ester>	 I'm using URLSearchParams
[19:41:59] <bd808>	 new URLSearchParams(window.location.search) ?
[19:42:08] <Ester>	 correct
[19:42:19] <Ester>	 it works withough question mark too
[19:42:29] <bd808>	 I think that should "just work" with the question mark, yeah
[20:50:30] <RoanKattouw>	 I'm trying to run puppet on a new instance and it's failing, something with "certificate verify failed"
[20:50:42] <RoanKattouw>	 Error: /File[/var/lib/puppet/facts.d]: Failed to generate additional resources using 'eval_generate': SSL_connect returned=1 errno=0 state=error: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA: deployment-puppetmaster02.deployment-prep.eqiad.wmflabs]
[20:50:50] <RoanKattouw>	 (This is on deployment-maps01)
[20:51:05] <paladox>	 RoanKattouw rm -rf /var/lib/puppet/ssl && puppet agent -tv
[20:51:17] <paladox>	 (it's because this has a local puppetmaster)
[20:51:28] <paladox>	 so it needs to generate the cert on the puppetmaster
[20:53:00] <RoanKattouw>	 Why do I have a local puppetmaster? I didn't enable that role
[20:54:06] <paladox>	 RoanKattouw in the deployment project, it is probaly enforced globally
[20:54:20] <RoanKattouw>	 OK now that I've deleted that directory (and backed it up first), it's now failing with: Exiting; no certificate found and waitforcert is disabled
[20:54:42] <paladox>	 RoanKattouw you need to ssh into deployment-puppetmaster02.deployment-prep.eqiad.wmflabs
[20:54:43] <paladox>	 and type
[20:54:46] <paladox>	 puppet cert list
[20:54:57] <paladox>	 which should show the name of the instance you created in there
[20:55:15] <paladox>	 use the full name and type puppet cert sign <name-of-instance>
[20:55:30] <paladox>	 deployment-maps01.deployment-prep.eqiad.wmflabs
[20:55:54] <RoanKattouw>	 All that "puppet cert list" printed was Notice: Signed certificate request for ca
[20:56:18] <RoanKattouw>	 The second time I ran it it printed nothing
[20:56:29] <paladox>	 RoanKattouw hmm, on deployment-puppetmaster02 ?
[20:56:39] <RoanKattouw>	 Yes
[20:57:07] <paladox>	 RoanKattouw did you do that as root?
[20:57:29] <RoanKattouw>	 No, will try with sudo
[20:57:46] <RoanKattouw>	 OK yes I see it now
[20:58:01] <paladox>	 puppet cert sign deployment-maps01.deployment-prep.eqiad.wmflabs
[20:58:59] <RoanKattouw>	 OK yay thanks
[20:59:01] <RoanKattouw>	 It's working now
[20:59:07] <RoanKattouw>	 Well, it's still failing, but for other reasons
[20:59:20] <RoanKattouw>	 Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Function Call, Could not find data item profile::cassandra::single_instance::seeds in any Hiera data file and no default supplied at /etc/puppet/modules/profile/manifests/maps/apps.pp:2:24 on node
[20:59:20] <RoanKattouw>	 deployment-maps01.deployment-prep.eqiad.wmflabs
[20:59:25] <RoanKattouw>	 But that's a problem with the role that I'm trying to enable
[21:00:22] <Krenair>	 usually with those it's been defined only in prod, e.g. as hieradata for a role which puppet won't take into account in labs
[21:02:09] <RoanKattouw>	 I can tell it's been too long since I did puppet because I don't know anything about hiera
[21:02:19] <RoanKattouw>	 Where is labs hiera data defined?
[21:02:48] <RoanKattouw>	 paladox: Thanks for helping me fix the cert error!
[21:02:56] <chicocvenancio>	 RoanKattouw: wikitech, ops puppet repo and horizon
[21:03:23] * chicocvenancio goes check what is the hierarchy of those backends
[21:03:46] <paladox>	 RoanKattouw your welcome :)
[21:04:02] <RoanKattouw>	 Oh maybe it's because the maps-test stuff is in codfw
[21:04:12] <RoanKattouw>	 hieradata/role/codfw/maps/test/master.yaml
[21:06:28] <Krenair>	 RoanKattouw, that plus the role stuff isn't applied in labs
[21:06:48] <RoanKattouw>	 Ha I see there is no role/labs
[21:07:02] <RoanKattouw>	 So how do I configure this?
[21:07:15] <RoanKattouw>	 It's good that it's not pulling the value from production because that value is wrong (it's an array of server names)
[21:20:22] <RoanKattouw>	 Found it: https://wikitech.wikimedia.org/wiki/Puppet_Hiera#In_Labs
[23:43:06] <wm-bot5>	 I trust:  bd808!.*@wikimedia/BDavis-WMF (2admin), .*@wikimedia/andrew-bogott (2admin), .*@wikimedia/mviswanathan-wmf (2admin), .*@wikimedia/.* (2trusted), .*@mediawiki/.* (2trusted), .*@wikipedia/.* (2trusted),
[23:43:06] <MacFan4000>	 @trusted
[23:43:28] <MacFan4000>	 Ignore pings
[23:43:36] <bd808>	 MacFan4000: something you need to bot to do?
[23:44:02] <MacFan4000>	 Was just looking at the list but thanks anyway