[00:27:39] !log tools.coverme restarted web service, was returning HTTP 500 [00:27:42] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.coverme/SAL [08:19:13] !log toolsbeta rebuild toolsbeta-test-k8s-haproxy-[12] without nfs [08:19:18] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [09:44:21] !log tools `sudo wmcs-openstack --os-project-id=tools port create --network lan-flat-cloudinstances2b tools-k8s-haproxy-keepalived-vip` [09:44:24] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [09:44:37] !log toolsbeta `sudo wmcs-openstack --os-project-id=toolsbeta port create --network lan-flat-cloudinstances2b toolsbeta-k8s-haproxy-keepalived-vip` [09:44:38] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [13:51:41] !log admin add inherited 'admin' right to novaadmin user throughout eqiad1. I was trying to narrow down the rights here but lack of admin breaks some workflows, e.g. T281894 and T282235 [13:51:46] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [13:51:47] T281894: Cloud: designate: missing or misconfigured API policy for creating new domains - https://phabricator.wikimedia.org/T281894 [13:51:47] T282235: Configure Neutron VIP ports for tools and toolsbeta k8s haproxies - https://phabricator.wikimedia.org/T282235 [14:20:34] !log toolsbeta cherry pick https://gerrit.wikimedia.org/r/c/operations/puppet/+/686607/ [14:20:36] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [16:16:19] !log toolsbeta create record k8s.svc.toolsbeta.eqiad1.wikimedia.cloud. pointing to haproxy vip T282227 [16:16:24] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [16:16:24] T282227: Make toolforge k8s service names a CNAME to .svc. to allow editing without cloudinfra access - https://phabricator.wikimedia.org/T282227 [16:22:12] I just got a fingerprint error when trying to connect to login-stretch.tools.wmflabs.org [16:23:42] Fingerprint I'm getting: ab:e6:9c:4e:bc:26:2c:b9:43:5b:3d:9a:37:e8:cb:5b [16:25:02] Cyberpower678: that appears to be correct per https://wikitech.wikimedia.org/wiki/Help:SSH_Fingerprints/login.toolforge.org [16:25:18] Great. Did it change recently? [16:25:31] which fingerprints did you have previously? we haven't changed those I believe [16:26:07] Hmm. Strange. Maybe the software update to Datagrip wiped my known hosts. [16:28:01] the fingerprint on wikitech was last updated 2019-03-07T23:21:41‎ :) [16:30:29] !log toolsbeta recreated k8s.toolsbeta.eqiad1.wikimedia.cloud. as a CNAME to k8s.svc.toolsbeta.eqiad1.wikimedia.cloud. T282227 [16:30:33] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [16:30:33] T282227: Make toolforge k8s service names a CNAME to .svc. to allow editing without cloudinfra access - https://phabricator.wikimedia.org/T282227 [17:00:36] !log toolsbeta deleted "toolsbeta-test-k8s-haproxy-2", "toolsbeta-test-k8s-haproxy-1" when the dns caches finally dropped T282227 [17:00:40] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [17:00:40] T282227: Make toolforge k8s service names a CNAME to .svc. to allow editing without cloudinfra access - https://phabricator.wikimedia.org/T282227 [17:12:27] !log tools created A record of k8s.svc.tools.eqiad1.wikimedia.cloud pointing at current cluster with TTL of 300 for quick initial failover when the new set of haproxy nodes are ready T282227 [17:12:31] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [17:12:31] T282227: Make toolforge k8s service names a CNAME to .svc. to allow editing without cloudinfra access - https://phabricator.wikimedia.org/T282227 [17:15:40] !log tools recreated recordset of k8s.tools.eqiad1.wikimedia.cloud as CNAME to k8s.svc.tools.eqiad1.wikimedia.cloud T282227 [17:15:44] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [18:07:12] !log tools generate and add k8s haproxy keepalived password (profile::toolforge::k8s::haproxy::keepalived_password) to private puppet repo [18:07:15] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [19:46:06] hello, is this where I can ask for help about toolforge? [19:47:37] !help [19:47:37] If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-kanban [19:47:55] !ask | bennylin [19:47:55] bennylin: Hi, how can we help you? Just ask your question. [19:49:33] I want to create a new tool, I have Toolforge acc set up, followed the https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Quickstart, but stopped short at the "As soon as you get SSH access to bastion" part. There's no explanation about bastion or how to use it. [19:50:21] I'm familiar with how Github repositories work, but not toolforge's repo [19:50:58] where should I proceed / look for how to set up my repo? [19:51:01] thanks [19:52:21] bennylin: toolforge isn't bound to any particular code repo, so you can continue to store your code in github if you like. [19:52:54] Typically your next step would be to set up a login to login.toolforge.org [19:53:28] are you following this guide? https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Quickstart [19:53:34] Ah, so you said :) [19:53:45] let's say i have the repo in my computer and want to host it in toolforge's repo, how do i do that? [19:54:27] i did a search about "bastion" in the wiki, but no immediate result. googled it, but not really helpful either. Maybe the phrase need to be wiki-linked somewhere, for the uninitiated. [19:54:48] https://wikitech.wikimedia.org/wiki/Bastion [19:55:03] a "bastion host" is a server that gates access to other servers [19:55:07] bennylin: I assume that you are used to a system where you deploy code directly out of a git repo? That's not something that we support at the moment; you'll need to copy your code to a host, then log in there to start it. [19:55:19] For Toolforge, login.toolforge.org is a "bastion" [19:56:03] https://en.wikipedia.org/wiki/Bastion_host [19:56:36] https://wikitech.wikimedia.org/wiki/Help:Access_to_Toolforge_instances_with_PuTTY_and_WinSCP is linked on the quickstart checklist there, we should likely put the instructions as words instead of screenshots only [19:57:18] The cool kids want youtube videos these days, apparently [19:57:39] It was TLDR, but I watched a 5 hour video!!!! [19:58:01] I'm downloading WinSCP now. I think this is what I'm looking for. Will try and ask again if still failed. Thanks! [19:58:20] sounds good! [19:58:48] I guess the Telegram bridge stopped working for some 29 hours [20:00:12] Majavah. I see what you mean: The screenshots are 8+ years old, I think many things have changed, such as host name, etc. [20:00:34] bennylin: I tried to adjust that document a bit to avoid using 'bastion' without explanation :) [20:00:57] thanks andrew! [20:01:23] https://wikitech.wikimedia.org/wiki/Help:Access_to_Toolforge_instances_with_PuTTY_and_WinSCP so host name is login.toolforge.org? [20:01:51] bennylin: correct [20:03:15] bennylin: I think the screnshots are actually up to date, aren't they? [20:03:34] you could setup a new git repo and then git push to it from your existing rep [20:03:44] to move it with the history [20:05:46] chicocvenancio: hmmm... I wonder what is up with bridgebot. It looks like it's missing a lot of messages in both directions :/ [20:07:06] !log tools.bridgebot Restarting due to irc<->telegram message drops [20:07:10] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.bridgebot/SAL [20:07:10] chicocvenancio: I was looking at Putty screenshots, lol. For WinSCP, there's some minor difference in the interface, https://wikitech.wikimedia.org/wiki/File:WINSCP_screen_advanced_settings3.png the SFTP is no longer there (screenshot from 2018, I downloaded the latest). [20:07:25] bd808: when did it stop working? maybe it got mad after the cloudgw migration yesterday [20:08:02] Majavah: it looks like it probably lines up with that [20:08:13] it's back working now after a restart [20:08:22] next question: https://wikitech.wikimedia.org/wiki/File:WINSCP_screen_advanced_settings_authentication.png it ask for private key, does that mean my ssh id_rsa.pub or something? but the browse file section ask for Putty private key files (.ppk), [20:09:09] bennylin: it's the one that does not have an extension. id_rsa but without the .pub. pub stands for public. and it's always a pair consisting of a private and a public part [20:09:37] Yeah, but you might need to convert it with puttygen [20:10:25] * chicocvenancio hasn't used putty or winscp for a while [20:13:12] cool, it auto convert it. I'm in. Many thanks. [20:52:41] !log tools.jouncebot Restarted to pick up 560a22a (T243394) [20:52:44] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.jouncebot/SAL [21:38:39] hi, it's me again. when I tried to "become ", the cmd line froze: "Host is not communicating for more than 15 seconds. Still waiting..." [21:39:50] when I tried to start the tool with "webservice start" it also failed [21:39:52] https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Quickstart#Run_new_tool_-_Quickstart [21:40:26] 'Tool username should begin with " + Tool.PREFIX' [21:41:22] what should I do? [21:41:42] what should I do? (via WinSCP's command line) [21:57:03] @Bennylin: what is your tool name? It sounds like you may be having some networking problems with your ssh connection, but maybe there is more going wrong. [21:57:15] leksem-indonesia [21:57:42] dmin.wikimedia.org/tools/id/leksem-indonesia [21:57:51] https://toolsadmin.wikimedia.org/tools/id/leksem-indonesia [21:58:46] *nod* I can see that the tool account creation worked and that `become leksem-indonesia` works for me (I have to add `sudo` to that because I'm using toolforge admin rights) [21:59:36] "Host is not communicating for more than 15 seconds. Still waiting..." sounds very much like network issues between your computer and the Toolforge computer [22:02:14] Does anyone have time to work with @Bennylin on debugging their ssh issues? https://wikitech.wikimedia.org/wiki/Reporting_a_connectivity_issue would give some ideas on figuring out if it is a routing issue of some kind. [22:04:36] yes please, if anyone would be able to help, I still have many followup questions, such as OAuth, cloning, and connecting .py files to the web frontend. don't want to flood this channel. any help is greatly appreciated. [22:08:42] @Bennylin: asking questions here is good! But if you have that many questions and the tutorials are not helping you then maybe you do need someone to act as a tutor [22:09:45] @Bennylin: have you seen https://wikitech.wikimedia.org/wiki/Help:Toolforge/My_first_Flask_OAuth_tool ? It may cover some of your questions [22:10:07] not yet. reading it now, ty [22:16:16] ok, I switched to putty, and "become " worked. Previously was on WinSCP's cmdline. (re @wmtelegram_bot: *nod* I can see that the tool account creation worked and that `become leksem-indonesia` works for me (I have to add `sudo` to that because I'm using toolforge admin rights)) [22:17:38] alright. webservice started. will continue tomorrow. thanks @bd808