[00:11:04] 10serviceops, 10Operations, 10ops-eqiad: (No Need By Date Provided) rack/setup/install kubernetes10[07-14].eqiad.wmnet - https://phabricator.wikimedia.org/T241850 (10wiki_willy) ++ @Cmjohnson / @Jclark-ctr - just following up on Effie's previous comment, can you guys decide on a doable turnover date for this... [01:58:27] 10serviceops, 10Deployments, 10Release-Engineering-Team, 10Performance-Team (Radar): Cache of wmf-config/InitialiseSettings often 1 step behind - https://phabricator.wikimedia.org/T236104 (10Catrope) I encountered this bug again today, when deploying [[https://gerrit.wikimedia.org/r/c/565398/|this config c... [06:05:50] 10serviceops, 10Operations, 10Performance-Team (Radar): Increased latency in CODFW API and APP monitoring urls (~07:20 UTC 19 Jan 2020) - https://phabricator.wikimedia.org/T243149 (10Marostegui) >>! In T243149#5821443, @Krinkle wrote: > Looks like the main action is to avoid these alarms in the future, askin... [07:01:05] 10serviceops, 10Deployments, 10Release-Engineering-Team, 10Performance-Team (Radar): Cache of wmf-config/InitialiseSettings often 1 step behind - https://phabricator.wikimedia.org/T236104 (10Catrope) Thanks to Giuseppe I got dsh+keyholder to work: `SSH_AUTH_SOCK=/run/keyholder/proxy.sock dsh -r ssh -F 20 -... [09:01:14] 10serviceops, 10Operations, 10Wikimedia-Etherpad, 10Patch-For-Review: Migrate etherpad1001 to Buster - https://phabricator.wikimedia.org/T224580 (10MoritzMuehlenhoff) >>! In T224580#5820607, @Dzahn wrote: > The following packages are used by the puppet role but so far missing on buster: > > * prometheus-e... [10:51:07] 10serviceops, 10Operations, 10Scap: Make canary wait time configurable - https://phabricator.wikimedia.org/T217924 (10jijiki) @thcipriani ping! :) [10:57:02] 10serviceops, 10Operations, 10Patch-For-Review: Move debugging symbols and tools to a new class - https://phabricator.wikimedia.org/T236048 (10jijiki) [11:23:42] o/ QU: Is it possible to have a blubberfile and build docker images in CI prior to the security review of the service that is being built for? [12:27:53] <_joe_> barking up the wrong tree addshore [12:28:18] <_joe_> you'd have more luck in #releng in getting a meaningful answer [12:46:05] Woof woof, ack, thanks :) [12:46:18] I always seem to struggle to cross over with them as much as service ops ;) [13:13:44] <_joe_> heh I get why [13:13:48] <_joe_> :P [13:14:14] <_joe_> the lines between where the pipeline involves us and where it is under their control are quite blurred [13:14:32] <_joe_> but overall anything regarding blubber => releng [13:14:55] <_joe_> anything regarding architecture,deployment charts, performance => serviceops [14:01:03] in that case, hypothetically if these 3 services would be deployed, would that end up being 1 chart or 3? [14:19:56] akosiaris: I'd love to integrate whatever we are going to do with service.name + canary stuff into the eventstreams chart, thoughts on recent comment if you have time? [14:21:40] 10serviceops, 10Operations, 10Wikimedia-Etherpad, 10Patch-For-Review: Migrate etherpad1001 to Buster - https://phabricator.wikimedia.org/T224580 (10akosiaris) >>! In T224580#5820607, @Dzahn wrote: > The following packages are used by the puppet role but so far missing on buster: > > * prometheus-etherpad-... [14:55:49] <_joe_> addshore: I guess one chart, it's one chart per "service" right [14:56:23] <_joe_> probably separated deployments /pods though [14:57:08] <_joe_> or, three charts, plus one chart gluing them together :) [15:31:19] one chart per 'app' [15:49:01] <_joe_> it depends a lot on how we want these things to interact with each other [15:49:18] <_joe_> we could have the 3 softwares run in bundle in a single pod [15:57:24] aye, i've just been told that the app label, which is set to wmf.chartname, should be the 'app', e.g. apache, or eventgate, or whatever [16:10:11] 10serviceops, 10Operations, 10Scap: Make canary wait time configurable - https://phabricator.wikimedia.org/T217924 (10thcipriani) >>! In T217924#5822404, @jijiki wrote: > @thcipriani ping! :) pong! >>! In T217924#5806546, @jijiki wrote: > @thcipriani as per our discussion, we can consider merging and testi... [16:13:07] "we could have the 3 softwares run in bundle in a single pod", I was just going to come and ask you how you would feel about that _joe_ [16:13:29] I guess the main issue there is right now the idea is 1 language per base image / per service / per deployment [16:13:53] But of course it is technically possible to have all of the requirements for all the services they want in 1 image and have all of the things in there too. [16:14:26] A seperate unrelated question, I guess services are allowed to store tmp files on their own disk for some seconds or minutes without an issue? [16:21:43] 10serviceops, 10Operations, 10ops-eqiad: (Need By Dec 20) rack/setup/install mw13[49-84].eqiad.wmnet - https://phabricator.wikimedia.org/T236437 (10jijiki) @Jclark-ctr @Cmjohnson @wiki_willy please updates of the status of racking these servers and update the task accordingly, thank you! [16:25:05] 10serviceops, 10Operations, 10ops-eqiad: (No Need By Date Provided) rack/setup/install mw[1385-1413].eqiad.wmnet - https://phabricator.wikimedia.org/T241849 (10jijiki) @wiki_willy @Cmjohnson @Jclark-ctr Please provide a date that works for you regarding those servers, thank you! [16:27:05] <_joe_> addshore: not great, but I guess this can use some discussion with techcom too :) [16:27:45] Ack, Let me add write an RFC to my todo list :P [16:28:01] <_joe_> that's not a precondition [16:28:10] <_joe_> but having a clear idea of what you want to do is [16:43:46] 10serviceops, 10Operations, 10ops-codfw: rack/setup/install new codfw mw systems - https://phabricator.wikimedia.org/T241852 (10jijiki) @Papaul Great, let's rack A, B, D, and we will do C in two batches afterwards. Thank you! [16:47:11] 10serviceops, 10Deployments, 10Release-Engineering-Team, 10Performance-Team (Radar): Cache of wmf-config/InitialiseSettings often 1 step behind - https://phabricator.wikimedia.org/T236104 (10thcipriani) >>! In T236104#5821812, @Catrope wrote: > - `InitialiseSettings.php` had the correct value for the confi... [20:40:47] 10serviceops, 10Operations, 10ops-eqiad, 10Patch-For-Review: (Need By: Jan 10) rack/setup/install mc-gp100[123].eqiad.wmnet - https://phabricator.wikimedia.org/T241795 (10Cmjohnson)