[05:23:59] bd808: fwiw, toolforge has this cert [05:24:03] https://www.irccloud.com/pastebin/I02tZKUD/ [05:24:22] so it's one that's trusted everywhere [05:24:42] I have no idea how the puppet master cert used to work [08:09:40] !log wikilabels staging wikilabels-wmflabs-deploy:6930d09 [08:09:41] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikilabels/SAL [08:56:05] !log wikilabels deploying wikilabels-wmflabs-deploy:6930d09 [08:56:07] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikilabels/SAL [14:28:46] Technical Advice IRC meeting starting in 30 minutes in channel #wikimedia-tech, hosts: @Thiemo_WMDE & @Lucas_WMDE - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting [14:38:02] !help I created a new tool https://toolsadmin.wikimedia.org/tools/id/genedb but "become" doesn't work. Could someone kick tool creation please? [14:38:02] Magnus___: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [14:38:37] Magnus___: you probably need to log out and login again [14:38:59] tried login/logout [14:39:04] did killall [14:39:09] no joy [14:40:57] Magnus___: you are in the tools.genedb group, what is become telling you? [14:41:48] thank you chicocvenancio, I have no idea how to troubleshoot this :) [14:44:13] chicocvenancio: magnus@tools-bastion-03:~$ become genedb become: no such tool 'genedb' [14:44:32] chicocvenancio: Also: magnus@tools-bastion-03:~$ ls -l /data/project/genedb ls: cannot access /data/project/genedb: No such file or directory [14:47:02] huh, seems somehow the folder was indeed not created [14:47:09] and become checks that... [14:48:04] andrewbogott: my guess is striker failed to create the folder for some reason [14:50:03] is that something I can just create by hand? and/or maybe we should file a striker task [14:50:13] both, I think [14:50:32] chicocvenancio: Also don't forget the replica file ;-) [14:51:06] that's a bit more difficult [14:51:13] hm, I wonder if nfs is overloaded. "/data/project# ls -ltrah" is hanging [14:51:36] lots of folders though [14:55:24] ok, I created that folder [14:55:32] no idea how to make the replica file though [14:55:57] Technical Advice IRC meeting starting in 5 minutes in channel #wikimedia-tech, hosts: @Thiemo_WMDE & @Lucas_WMDE - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting [14:56:22] huh [14:56:32] the replica got there andrewbogott [14:56:37] andrewbogott: there is a process on labstore1004 that does it after creating teh actual creds [14:56:45] it probably was failing too if no folder [14:56:49] oh, great [14:57:58] so Magnus___ you're probably all set [14:58:09] chicocvenancio: Yay, thanks! [15:13:52] !log wikilabels staging wikilabels-wmflabs-deploy:d7ddb5d [15:13:53] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikilabels/SAL [15:14:45] !log wikilabels deploying wikilabels-wmflabs-deploy:d7ddb5d [15:14:45] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikilabels/SAL [15:32:49] ^ andrewbogott [15:34:07] what's the diff? [15:34:10] oh, did I forget the /topic? [15:35:57] maybe? [15:41:55] xd [16:48:31] !help HI, I have problems with labels.wmflabs.org because I can not to request new work list [16:48:31] Zoranzoki21: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [16:50:59] Zoranoki21: we did reboot that service today, but things looked ok. Are you using it for the first time, or is this something that usually works? [16:51:22] bstorm_ No for the first time [16:51:43] I was able to request a work set on there [16:51:50] for srwiki [16:52:00] Hmm. I haven't tried that one :) [16:52:57] I get an error about images [16:52:58] hmm [16:53:09] same [16:53:30] to he can not to load damaging_and_goodfalth [16:53:36] Same here [16:53:41] Exactly that error [16:55:16] I think creating a phabricator task is a good idea here. I can add at least myself and halfak to the task if you do and send me the number [16:55:33] With the form above in the help message [16:55:37] I no know what in task should be included [16:56:09] I'd mention the error you are getting on the wiki labels tool. I can add notes as well [16:56:36] Can you create task. I no know how to create task with all details which will help to problem be resolved [16:58:07] I can. However, if you do, you'll be subscribed to it and will get updates in email. I can add to the task and edit the description and all that, since I've seen the error. [16:58:25] So don't worry about it leaving out a lot of info. I can fix it up [16:58:27] I can subscribe same if you create [16:58:32] Ok :) [16:58:42] Thank you [17:00:23] T194840 [17:00:23] T194840: wiki-labels issues on srwiki - https://phabricator.wikimedia.org/T194840 [17:00:44] There might be some re-routing of the ticket, since I'm not sure it belongs in our cloud-services queue, but that can be sorted out in time :) [17:00:56] I added the admin for wikilabels to the ticket [17:01:03] Thank you very much [17:01:06] I will now track task [17:01:37] You should get comfortable making those! It's a good thing for getting help during non-US hours, especially since most of my team is US. [17:01:42] :) [17:02:05] bstorm_ Ok. I am from Serbia [17:02:18] I know that the admin for that is in Spain at the moment, so he's likely not to see the ticket until morning there [17:02:25] I guessed by the wiki choice [17:02:28] :) [17:03:23] yeh, it's 7pm there :) [17:03:55] same [17:04:01] 7pm and in Serbia :) [17:04:36] 1 hour behind here [17:05:50] ok [20:58:15] andrewbogott, chicocvenancio: tool home dirs are supposed to be created by maintain-kubeusers IIRC. that /data/project/genedb still lacks .kube [20:58:49] * zhuyifei1999_ checks maintain-kubeusers [21:01:37] !log tools maintain-kubeusers on stuck in infinite sleeps of 10 seconds [21:02:02] stashbot: hello? [21:13:05] !log tools installing python3-dbg temporary on tools-k8s-master-01 for T194859 [22:53:27] !log tools.stashbot Checking !log function [22:53:28] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stashbot/SAL [23:01:56] zhuyifei1999_: I wonder if we could figure out how to get letsencrypt to make ssl certs for the toolsbeta kubernetes api? Other than than I guess we will need to dig into config and figure out how to make the self-signed puppet cert(s) trusted [23:02:51] would that require having the puppetmaster store the cert as secret? [23:04:12] the private part of the cert would need to be secret, yes [23:04:34] and the public part would change every 3? months as it was renewed [23:05:02] though, I'm not sure how widely used is the `webservice` command within the pod. like, is it worth the effort [23:05:30] oh wait... if we used letsencrypt then their signing root should already be trusted. [23:05:50] on the kubernetes side I kind of doubt it is used often [23:06:19] on the grid there are tools that make their own watchdog process using crontabs [23:06:59] I actually did not know that `webservice` worked inside a container until a couple of months ago. I don't think its documented [23:07:51] is it only running webservice from inside a container that is having problems now? [23:07:53] * zhuyifei1999_ neither [23:08:06] I think so, yeah [23:08:27] running it outside the container seems fine [23:08:45] let's call that a low priority bug then :) [23:08:47] although curl always complains about certs being insecure when I debug [23:09:30] we should just make sure that Neha knows about it so that she does not get confused [23:10:10] right [23:11:04] zhuyifei1999_: thanks for all your work on the toolsbeta setup! [23:11:09] np