[08:39:41] <_joe_> jayme / akosiaris around? would you mind having a quick video chat? We have a flaming turd on our doorstep, I'd like to see how we're going to put out the fire without stomping on it [08:40:01] _joe_: 2min [08:46:06] _joe_: meet.google.com/ehv-bmko-nsx [08:48:43] <_joe_> jayme: can you admit me with my private account? I just got logged out ffs [08:51:37] I guess that flaming turd is from me :-P [08:52:27] <_joe_> not you :P [08:52:58] :-D [09:02:09] effie, _joe_ I added you as subs of https://phabricator.wikimedia.org/T281135 (instead of me) for the relocation of mc-gp2002 [09:03:31] also, unrelated announcement - me and Kevin are going to deploy ORES in the EU afternoon for https://phabricator.wikimedia.org/T278723 [09:05:12] <_joe_> elukey: you say that like you're proud of being the new owner of ORES [09:05:53] * elukey looks outside the window [09:13:19] 10serviceops, 10DBA, 10SRE, 10ops-codfw: codfw: Relocate servers in 10G racks - https://phabricator.wikimedia.org/T281135 (10jijiki) [09:31:50] congrats, elukey! :-D [11:06:27] Hi all! I'm about to setup helmcharts etc for Speechoid (the backend of Wikispeech, a WMSE project) and was wondering what environment you recommend for local testing and deployment. Does it even matter? What k8s architechture is running in the WMF infrastructure? [11:07:27] I usually go for minicube, but perhaps that's not a good fit? [11:08:50] Oh, and I never looked at Helmcharts before, so perhaps I'm asking strange questions. [11:10:26] Most examples on wikitech use minikube so I think you're safe to use that [11:43:20] 10serviceops, 10Add-Link, 10Data-Persistence (Consultation), 10Growth-Team (Current Sprint): Determine why service responses are slow and what we can do about it - https://phabricator.wikimedia.org/T279411 (10kostajh) Another snapshot from today: {F34441646} It looks like we're getting pretty close to th... [12:45:41] 10serviceops, 10SRE: mcrouter proxies and scap proxies - https://phabricator.wikimedia.org/T245841 (10hashar) [12:50:53] 10serviceops, 10Product-Infrastructure-Team-Backlog, 10Push-Notification-Service: Migrate push-notifications service to latest node LTS version - https://phabricator.wikimedia.org/T256131 (10hashar) [13:35:39] 10serviceops, 10SRE, 10cloud-services-team (Kanban): Replace bootstrap-vz to generate Docker base images - https://phabricator.wikimedia.org/T281984 (10MoritzMuehlenhoff) [13:40:11] 10serviceops, 10SRE: Publish wikimedia-bullseye base docker image - https://phabricator.wikimedia.org/T281596 (10Volans) If possible it would be great if we could get a base bullseye image somehow, even if not auto-updated right from the start and otherwise created. We have already a couple of hosts on bullsey... [14:22:18] 10serviceops, 10SRE: Publish wikimedia-bullseye base docker image - https://phabricator.wikimedia.org/T281596 (10Joe) >>! In T281596#7061654, @Volans wrote: > If possible it would be great if we could get a base bullseye image somehow, even if not auto-updated right from the start and otherwise created. > We h... [14:24:14] ORES deployed! :) [15:02:07] <_joe_> elukey: you're so proud of your baby [15:05:47] _joe_ I am just crying that it didn't explode in a horrible way [15:59:37] <_joe_> jayme: I'm almost done writing a "puppet compiler" for charts/helmfile deployments [15:59:42] <_joe_> this is the good news [15:59:50] <_joe_> the bad news is, it's a bunch of ruby [16:00:22] <_joe_> the rakefile for that repo is now 600 lines [16:00:29] <_joe_> it's probably time to restructure it a bit [16:26:55] that sounds...scary :) [17:13:30] 10serviceops, 10Add-Link, 10Data-Persistence (Consultation), 10Growth-Team (Current Sprint): Determine why service responses are slow and what we can do about it - https://phabricator.wikimedia.org/T279411 (10Tgr) The maintenance script was working fine during that time, no timeouts or anything.