[06:55:24] <Majavah>	 !log tools clear error state from tools-sgeexec-0916
[06:55:27] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[10:53:06] <arturo>	 !log admin icinga-downtime cloudmetrics1002 for 3 months (T275605)
[10:53:09] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[10:53:09] <stashbot>	 T275605: cloudmetrics1002: mysterious issue - https://phabricator.wikimedia.org/T275605
[11:08:25] <Nemo_bis>	 https://grafana-labs.wikimedia.org/ "upstream connect error or disconnect/reset before headers. reset reason: connection failure"
[11:08:48] <Nemo_bis>	 same as bug above?
[11:10:18] <arturo>	 server having hardware issues
[11:17:53] <arturo>	 should be fixed now
[11:18:00] <arturo>	 (the URL)
[11:21:19] <Nemo_bis>	 front page loads, yes :) 
[11:21:26] <Nemo_bis>	 I didn't remember I was logged in previously, hmm
[13:50:10] <wm-bot>	 !log tools.lexeme-forms <lucaswerkmeister> deployed 5951b46450 (fix lang= and dir= on index)
[13:50:13] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL
[17:13:47] <Probateur>	 Mrb
[17:15:14] <Majavah>	 ?
[20:07:26] <wm-bb>	 <Ich66008834> Hi was macht ihr denn so 🧐
[20:08:14] <paladox>	 Hi, i'm wondering if anyone could help me ssh into an instance?
[20:08:38] <paladox>	 when trying to ssh into phabricator-prod-1001 it fails with "paladox@bastion.wmcloud.org: Permission denied (publickey)."
[20:08:53] <paladox>	 ```ssh phabricator-prod-1001.devtools.eqiad1.wikimedia.cloud``` is what i used
[20:09:08] <paladox>	 and my config:
[20:09:10] <paladox>	 ```
[20:09:25] <paladox>	 https://www.irccloud.com/pastebin/faz8rJ2S/
[20:10:12] <paladox>	 hmm i can ssh directly to bastion.wmcloud.org
[20:10:44] <andrewbogott>	 paladox: I'm checking...
[20:10:51] <paladox>	 thanks!
[20:11:16] <paladox>	 I think it could be the same issue i had a long while ago where the 22 port was shut for some reason and had to have the firewall added manually
[20:11:37] <andrewbogott>	 hm, it's working for me.  Can you reach other non-bastion VMs?
[20:12:32] <Majavah>	 paladox: do you have anything else in ssh config related to the bastion? with that config it would use a different key on the bastion and everything else
[20:12:53] <paladox>	 Nope (i cannot ssh into puppetmaster-1001 either)
[20:12:59] <paladox>	 and nope
[20:13:08] <andrewbogott>	 paladox: have you been able to, historically?
[20:13:38] <paladox>	 yes but before the switch to wmcloud i think
[20:14:19] <andrewbogott>	 ok, probably you need to duplicate your ssh config for *.wmcloud.org
[20:14:44] <andrewbogott>	 can you ssh to phabricator-prod-1001.devtools.eqiad.wmflabs ?
[20:14:46] <paladox>	 oh
[20:14:54] <paladox>	 i'll try the last one
[20:14:55] <Majavah>	 shouldn't the identity file config option point to the private key instead of the public one?
[20:15:01] <wm-bb>	 <Ich66008834> Hmm Unfortunately I'm having trouble with a persistent trojan or hacker...plus the amazonaws virus that I'm trying to get rid of.
[20:15:04] <paladox>	 ah right
[20:15:27] <Majavah>	 adding -v to the ssh command gives verbose output which would also be helpful
[20:15:31] <paladox>	 changing it didn't fix it
[20:15:45] <paladox>	 https://www.irccloud.com/pastebin/8HaIOjA5/
[20:16:37] <wm-bb>	 <Ich66008834> f anyone is familiar with Android, I would be very grateful for tips
[20:17:30] <Majavah>	 does the public key in the specified file match what you have on https://wikitech.wikimedia.org/wiki/Special:Preferences#mw-prefsection-openstack
[20:17:44] <paladox>	 yeh
[20:18:08] <andrewbogott>	 the latest docs no how to make this work are at https://wikitech.wikimedia.org/wiki/Help:Accessing_Cloud_VPS_instances#ProxyJump_%28recommended%29
[20:18:35] <andrewbogott>	 *on how
[20:18:36] <paladox>	 oh i try proxyjump then
[20:21:59] <paladox>	 oh
[20:22:02] <paladox>	 i got it working now
[20:22:05] <paladox>	 i had to do this:
[20:22:21] <paladox>	 https://www.irccloud.com/pastebin/9FGMuIgT/
[20:22:57] <andrewbogott>	 great!
[22:09:41] <wm-bb>	 <jhsoby> any issues with Toolforge at the moment? @wikilinksbot on Telegram is not working (but shows as running)
[22:11:20] <wm-bb>	 <jhsoby> ah nevermind, it works after a restart (took way longer than usual to restart, but still. no problem now)
[23:34:31] <wm-bb>	 <Ich66008834> @gifontenelle you here ah okay now its clear 🤭
[23:45:01] <wm-bb>	 <Ich66008834> ec2-52-34-13-77.us-west-2.compute.amazonaws.com  ?? What is this...always sucks my Batterie empty in 30 minutes