[09:21:09] 10serviceops, 10Wikidata-Query-Service, 10Discovery-Search (Current work): Additional capacity on the k8s Flink cluster for WCQS updater - https://phabricator.wikimedia.org/T280485 (10Gehel) [09:22:05] effie / akosiaris: ^ we're looking at additional resources in k8s for the WCQS updater. It's all crystal ball prediction at this point. [09:22:43] gehel: that sounds about right. Please proceed as you see fit :-) [09:22:48] :) [09:23:02] oh and if anyone tells you that they can predict the future better, I want the lottery numbers [09:23:40] If I remember correctly, we requested about 2x that for WDQS, so we should have spare capacity for WCQS when the time comes (probably early Q1) [09:24:04] 1 per DC, yeah [12:07:23] <_joe_> @all: I added the ability to calculate changes in the produced manifests for deployments and charts, you can see a sample here [12:07:32] <_joe_> https://integration.wikimedia.org/ci/job/helm-lint/4087/console (bottom of the output) [12:08:03] <_joe_> jayme / akosiaris / effie now I just need someone to review the bowl of ruby that does that [12:13:20] I thought you liked us [12:14:34] effie: that's probably why it's ruby and not some other language [12:14:48] whitespace would have been awesome :P [12:16:31] the output is awesome, enjoy yer bowl of ruby... [12:29:08] <_joe_> https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/685721/5 and then if you fancy reviewing cut and paste https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/688265 [14:31:48] 10serviceops, 10Prod-Kubernetes, 10SRE, 10Kubernetes: Set resource requests and limits for calico PODs - https://phabricator.wikimedia.org/T277877 (10JMeybohm) I've looked into typha and kube-controllers component as well as they shot a similar patterns (different magnitude, though). Unfortunately we lack... [14:55:42] you will have to forgive me, I may need 2; minutes [14:57:28] <_joe_> likewise, i got lost in code :/ [16:00:07] akosiaris: if you want to be in a hangout to debug mic issues happy to join :) [16:06:35] legoktm: thanks. I seems like I needed to restart chrome [16:07:23] it turns out that pulseaudio restarted for some reason [16:07:38] and again for some reason chrome can't connect to it again (????????) [16:08:15] systemd[2235]: pulseaudio.service: Main process exited, code=killed, status=9/KILL [16:08:19] aha! [16:09:27] I have no idea why chrome does not retry to connect to pulseaudio, I am pretty sure pipewire will fix everything (not) [16:38:43] :P [16:40:36] I had a hangout mic issue earlier today too, might be a bad deploy on their end [16:55:50] hello, I've a weird question for you [16:56:29] I was looking at the example of cumin commands on wikitech to update obselete ones and I found onethat was looking for [16:56:33] /etc/ssl/localcerts/api.svc.codfw.wmnet.chained.crt [16:57:11] now, the fun part is that that file is not managed by puppet (no host found) but if I look for the equivalent for eqiad that's present in both codfw and eqiad hosts [16:57:16] is that expected/normal? [16:57:37] see ls -la /etc/ssl/localcerts/ on mw2251 for example [17:05:40] what's the timestamp on it anyways? [17:06:00] The part that the eqiad cert is on codfw hosts seems expected because profile::tlsproxy::envoy::global_cert_name: "api.svc.eqiad.wmnet" is in hieradata/role/common/mediawiki/appserver/api.yaml [17:06:20] why that cert is not in secret/secrets/certificates.. i dont know [18:22:28] <_joe_> mutante: that cert was created *before* cergen [18:22:51] *nod* makes sense, ack