[10:52:04] <arturo>	 !log wmflabsdotorg make myself project admin to investigate T234836
[10:52:07] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wmflabsdotorg/SAL
[10:52:07] <stashbot>	 T234836: CloudVPS: update DNS record for eqiad1 routing_source_ip - https://phabricator.wikimedia.org/T234836
[10:55:17] <arturo>	 !log wmflabsdotorg (I was already member of the project, but horizon wont let me switch to it)
[10:55:18] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wmflabsdotorg/SAL
[10:57:25] <arturo>	 !log twl remove my own project membership (user & projectadmin)
[10:57:26] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Twl/SAL
[10:57:43] <arturo>	 !log wcdo remove my own project membership (user & projectadmin)
[10:57:43] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wcdo/SAL
[10:58:26] <arturo>	 !log orig remove my own project membership (user & projectadmin)
[10:58:27] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Orig/SAL
[10:58:59] <arturo>	 !log wikibase-registry remove my own project membership (user & projectadmin)
[10:59:00] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikibase-registry/SAL
[10:59:51] <arturo>	 !log wikicitevis remove my own project membership (user & projectadmin)
[10:59:52] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikicitevis/SAL
[11:00:44] <arturo>	 !log wikidocumentaries remove my own project membership (user & projectadmin)
[11:00:44] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikidocumentaries/SAL
[11:01:15] <arturo>	 !log wikitextexp remove my own project membership (user & projectadmin)
[11:01:20] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikitextexp/SAL
[11:01:52] <arturo>	 !log packagist-mirror remove my own project membership (user & projectadmin)
[11:01:53] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Packagist-mirror/SAL
[12:27:56] <arturo>	 !log toolsbeta created VM toolsbeta-test-proxy-01 for testing stuff related to T234037
[12:27:59] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL
[12:28:00] <stashbot>	 T234037: Toolforge ingress: decide on final layout of north-south proxy setup - https://phabricator.wikimedia.org/T234037
[13:13:58] <halfak>	 !log ores restarting celery workers on ores-worker-01,02,03,04
[13:14:00] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Ores/SAL
[13:24:59] <halfak>	 !log ores running FLUSHDB on ores-redis-02
[13:25:02] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Ores/SAL
[13:49:05] <halfak>	 !log ores Temporarily stopped nginx on ores-lb-03
[13:49:08] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Ores/SAL
[14:07:05] <halfak>	 !log ores Restarting nginx on ores-lb-03
[14:07:06] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Ores/SAL
[14:14:54] <arturo>	 !log toolsbeta created puppet prefix `toolsbeta-test-proxy` for testing stuff related to T234037
[14:14:58] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL
[14:14:59] <stashbot>	 T234037: Toolforge ingress: decide on final layout of north-south proxy setup - https://phabricator.wikimedia.org/T234037
[14:21:55] <mdholloway>	 !log services RIP appservice
[14:21:56] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Services/SAL
[14:32:53] <dschwen>	 the cloud purge is happening and just in time my login stopped working :-(
[14:37:33] <Lucas_WMDE>	 I thought it’s not until December?
[14:47:41] <dschwen>	 Well, I cannot claim my projects if my login is not working
[14:48:25] <arturo>	 dschwen: can I help you?
[14:54:24] <dschwen>	 That'd be great
[14:54:37] <dschwen>	 neither ssh nor wikitech logins wor for me
[14:54:50] <dschwen>	 user 'Dschwen' (or 'dschwen')
[14:55:40] <arturo>	 dschwen: which kind of error do you see?
[14:55:59] <dschwen>	 ssh login.tools.wmflabs.orgPermission denied (publickey,hostbased).
[14:56:11] <arturo>	 could that be missing ssh key on your side?
[14:56:14] <dschwen>	 Incorrect username or password entered. Please try again. Usernames are case-sensitive. See phab:T165795 for more details.
[14:56:19] <stashbot>	 T165795: Ldap auth extension vs. ldap vs. username Case - https://phabricator.wikimedia.org/T165795
[14:56:54] <dschwen>	 no, my ssh key would not suddenly go missing (on multiple machines)
[14:57:23] <arturo>	 try again the ssh connection please, I'm staring at the logs right now dschwen 
[14:58:10] <dschwen>	 tried from two machines
[14:58:15] <dschwen>	 did you get anything?
[14:58:20] <arturo>	 `Oct  8 14:57:34 tools-sgebastion-07 sshd[25014]: Failed publickey for daniel from x.x.x.x port 55674 ssh2: RSA SHA256:VDtSwevWIcLAks5OteoQb9Z0tcTVVQvdToCaNabh27M`
[14:58:49] <arturo>	 `Oct  8 14:57:53 tools-sgebastion-07 sshd[25022]: input_userauth_request: invalid user schwd [preauth]`
[14:59:02] <arturo>	 could any of these be yours?
[14:59:07] <dschwen>	 if I was able to log in wikitech I could check my keys...
[14:59:16] <dschwen>	 ugh, one sec
[14:59:54] <dschwen>	 D'Oh
[15:00:03] <arturo>	 `Oct  8 14:59:38 tools-sgebastion-07 sshd[25064]: Accepted publickey for dschwen from x.x.x.x port 13828 ssh2: RSA SHA256:eA5ydudTBFDhRA32LaAIlLJ2zSGGArH3aDTH7u8qnZ8`
[15:00:07] <arturo>	 :-)
[15:00:09] <dschwen>	 username was wrong... 
[15:00:38] <dschwen>	 wikitech site login remains. Is my username Dschwen or dschwen?
[15:00:55] <arturo>	 well... honestly I don't know
[15:01:17] <arturo>	 I could try doing some search
[15:02:06] <dschwen>	 resetting password now
[15:02:08] <dschwen>	 hold on
[15:03:22] <dschwen>	 I'm back in
[15:03:25] <dschwen>	 thx!
[15:03:39] <arturo>	 cool! welcome back to Wikimedia Cloud Services :-)
[15:41:47] <arturo>	 !log tools deleted VM instance tools-sgebastion-0test. No longer in use.
[15:41:50] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[19:24:09] <bstorm_>	 !log tools drained tools-worker-1003 and 1009 for rebalancing
[19:24:12] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[19:27:17] <bstorm_>	 !log tools drained tools-worker-1005 for rebalancing (and put these back in service as I went)
[19:27:19] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[19:34:12] <bstorm_>	 !log tools drained tools-worker-1009 and then 1014 for rebalancing
[19:34:14] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[19:39:58] <bstorm_>	 !log tools drained tools-worker-1007/8 to rebalance the cluster
[19:40:02] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL