[08:53:25] <vgutierrez>	 Krenair: https://gerrit.wikimedia.org/r/c/operations/puppet/+/528624 this CR still makes sense or may I abandon it?
[08:53:53] <vgutierrez>	 back in the day when I deployed the acme-chief traffic labs instances that dependency was missing
[09:03:25] <vgutierrez>	 hmm I see it's pretty similar to https://gerrit.wikimedia.org/r/c/operations/puppet/+/545081
[09:03:29] <vgutierrez>	 I'll abandon mine
[09:25:51] <arturo>	 !log tools created the `tools.eqiad1.wikimedia.cloud.` DNS zone
[09:25:54] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[11:16:18] <hauskater>	 legoktm: was https://gerrit.wikimedia.org/r/plugins/gitiles/labs/libraryupgrader/+/dddaf4b43c01df7752eaa1d46aa91760173aa77d deployed? Cause I still don't see the project on libup2 webpage
[11:54:16] <arturo>	 !log toolsbeta created puppet prefix `toolsbeta-test-k8s-haproxy` and delete `toolsbeta-test-k8s-lb` T236074
[11:54:19] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL
[11:54:20] <stashbot>	 T236074: Toolforge: rebuild the new k8s toolsbeta deployment and write final docs - https://phabricator.wikimedia.org/T236074
[11:57:27] <arturo>	 !log toolsbeta created 2 new VMS `toolsbeta-test-k8s-haproxy-{1,2}` T236074
[11:57:29] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL
[12:14:08] <arturo>	 !log toolsbeta delete FQDN `toolsbeta-k8s-master.toolsbeta.wmflabs.org` T236074
[12:14:11] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL
[12:14:11] <stashbot>	 T236074: Toolforge: rebuild the new k8s toolsbeta deployment and write final docs - https://phabricator.wikimedia.org/T236074
[12:15:03] <arturo>	 !log toolsbeta refresh IP addr of FQDN `k8s.toolsbeta.eqiad1.wikimedia.cloud` T236074
[12:15:05] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL
[12:26:08] <arturo>	 !log toolsbeta created 3 VMs `toolsbeta-test-k8s-control-{1,2,3}` T236074
[12:26:11] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL
[12:26:11] <stashbot>	 T236074: Toolforge: rebuild the new k8s toolsbeta deployment and write final docs - https://phabricator.wikimedia.org/T236074
[12:27:41] <arturo>	 !log toolsbeta refreshed puppet prefix `toolsbeta-test-k8s-control` with latest info T236074
[12:27:46] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL
[13:45:57] <dcausse>	 I having troubles connection through bastion.wmflabs.org
[13:46:33] <Samwalton9>	 I also can't connect
[13:48:18] <andrewbogott>	 I'm investigating
[13:49:29] <dcausse>	 thanks!
[13:53:04] <andrewbogott>	 dcausse: is the behavior you see any different now?
[13:54:21] <dcausse>	 andrewbogott: it's weird sometimes I get 'dcausse@bastion.wmflabs.org: Permission denied (publickey)'
[13:54:29] <Samwalton9>	 I'm getting 'permission denied' on my ssh key
[13:54:32] <andrewbogott>	 ok.  It might just be intermitten
[13:54:37] <andrewbogott>	 I see different things on different tries
[13:54:47] <dcausse>	 sometimes the ssh server simply close the connection
[14:14:07] <tarrow>	 is there a ticket that it would help to add other things to? Mine is intermittently totally failing and very slow the rest of the time
[14:14:59] <tarrow>	 I'm aiming at primary.bastion.wmflabs.org
[14:17:36] <bd808>	 tarrow: I don't think we have a ticket yet. The problem appears to be network connectivity/packet loss talking to the LDAP directory servers
[14:18:06] <bd808>	 The error is coming from inside the network! (spooky halloween music)
[14:18:14] <tarrow>	 Cool :P
[14:32:23] * andrewbogott creates https://phabricator.wikimedia.org/T236173
[14:54:59] <andrewbogott>	 tarrow, dcausse, Samwalton9, working now?
[14:55:31] <dcausse>	 andrewbogott: yes, thanks!
[14:55:33] <Samwalton9>	 Yep!
[14:55:36] <tarrow>	 works for me right now :)
[14:55:38] <tarrow>	 thanks!
[14:55:55] <andrewbogott>	 ok — I'm not sure if the current state is a long-term fix or just a short-term experiment, but at least we have a handle on the problem now.
[14:56:03] <andrewbogott>	 thanks for reporting and double-checking
[15:30:33] <arturo>	 !log toolsbeta created puppet prefix `toolsbeta-test-k8s-control` and delete `toolsbeta-test-k8s-master` T236074
[15:30:37] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL
[15:30:38] <stashbot>	 T236074: Toolforge: rebuild the new k8s toolsbeta deployment and write final docs - https://phabricator.wikimedia.org/T236074
[15:48:39] <arturo>	 !log toolsbeta point DNS record `k8s.toolsbeta.eqiad1.wikimedia.cloud` to the first controller node for the bootstrap T236074
[15:48:45] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL
[15:48:45] <stashbot>	 T236074: Toolforge: rebuild the new k8s toolsbeta deployment and write final docs - https://phabricator.wikimedia.org/T236074
[16:56:45] <bstorm_>	 !log tools drained tools-worker-1025.tools.eqiad.wmflabs which was malfunctioning
[16:56:48] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[17:43:15] <arturo>	 !log toolsbeta re-create VM `toolsbeta-test-k8s-control-1` T236074
[17:43:19] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL
[17:43:19] <stashbot>	 T236074: Toolforge: rebuild the new k8s toolsbeta deployment and write final docs - https://phabricator.wikimedia.org/T236074
[21:06:53] <doctaxon>	 andrewbogott: are you on?
[21:16:30] <bd808>	 doctaxon: maybe I can help? What do you need?
[21:39:19] <doctaxon>	 bd808: I sent a mail to andrewbogott and wonder why I get no answer, and no answer from other technical wmf services too
[21:41:32] <doctaxon>	 bd808: Is here anyone who can correct a mistake on the code of https://wikimediafoundation.org/about/vision/ ?
[21:42:21] <bd808>	 doctaxon: nobody on my team, no. The Wikimedia Foundation's Communications team manages that webseite.
[21:43:10] <bd808>	 They have a phabricator tag -- https://phabricator.wikimedia.org/project/view/3501/ -- but I have no idea how actively they look for and respond to issues posted there.
[21:43:34] <doctaxon>	 bd808: so I wonder why nobody of the technical team could reply my mails
[21:43:39] <andrewbogott>	 doctaxon: I don't see the email but I concur with bd808 that I don't have anything to do with wikimediafoundation.org
[21:44:26] <doctaxon>	 andrewbogott: I sent it 10/18/2019 
[21:44:57] <doctaxon>	 from info@wikipedia.org
[21:46:19] <doctaxon>	 yes, but if it's communications, okay then. Do they have a irc channel too?
[21:48:24] <bd808>	 doctaxon: I do not know of an irc channel used by the Foundation's Communications team unfortunately.
[21:48:37] <doctaxon>	 okay
[21:48:54] <doctaxon>	 but thank you for your information
[21:49:35] <Krenair>	 you sent it from info@wikipedia.org doctaxon ?
[21:50:13] <doctaxon>	 Krenair: yes
[21:50:25] <doctaxon>	 out of the OTRS
[21:50:42] <bd808>	 doctaxon: I found a group contact for the Communications team: talktocomms(_AT_)wikimedia.org
[21:51:17] <Krenair>	 ok
[21:51:18] <doctaxon>	 oh, thank you
[21:51:25] <bd808>	 sneakily public via https://meta.wikimedia.org/wiki/Communications/Wikimania_2019/en#Meetings
[21:51:39] <doctaxon>	 Krenair: I am OTRS agent for a long time now
[21:51:44] <Krenair>	 yes
[21:51:52] <Krenair>	 it's just unusual for us to use OTRS to create threads
[21:52:17] <doctaxon>	 it's forwarded, not directly created
[21:52:29] <Krenair>	 oh, well okay then
[21:53:03] <doctaxon>	 Wikimedia Belgium found a mistake and I wanted to forward it
[21:55:40] <mutante>	 unfortunately there is no way to report bugs on that site as far as i can tell