[07:17:02] 10serviceops, 10MW-on-K8s, 10Release-Engineering-Team: Generate non-restricted versions of the mediawiki-webserver image - https://phabricator.wikimedia.org/T281079 (10Joe) [07:54:46] 10serviceops, 10SRE, 10Patch-For-Review: upgrade conf2* servers to stretch - https://phabricator.wikimedia.org/T271573 (10ops-monitoring-bot) Icinga downtime set by jayme@cumin1001 for 1:00:00 1 host(s) and their services with reason: for initial etcd replication ` conf2005.codfw.wmnet ` [08:44:18] 10serviceops, 10SRE, 10Thumbor, 10Performance-Team (Radar), 10User-jijiki: Build python-thumbor-wikimedia 2.9 Debian package and upload to apt.wikimedia.org - https://phabricator.wikimedia.org/T254845 (10jijiki) 05Open→03Resolved It has been deployed to production, I forgot to close that task, sorry! [09:15:56] 10serviceops, 10SRE, 10Patch-For-Review: upgrade conf2* servers to stretch - https://phabricator.wikimedia.org/T271573 (10ops-monitoring-bot) Icinga downtime set by jayme@cumin1001 for 1:00:00 1 host(s) and their services with reason: for initial etcd replication ` conf2005.codfw.wmnet ` [10:46:54] <_joe_> akosiaris, jayme we're now actually using the networkpolicies in the charts for egress too, correct? [10:47:23] _joe_: that's correct [10:48:28] <_joe_> ok then I'm going to prepare a new version of common_templates that autogenerates a networkpolicy from the discovery data [10:50:38] <_joe_> it's a bit thorny because we need to consider the work akosiaris was doing to remove LVS from the equation, but it can't harm in the meantime [11:21:01] _joe_: em there is another complication https://phabricator.wikimedia.org/T253058 [11:21:10] I was planning to work on that this week [11:21:31] so I am be able to bundle both together. What is it that you wanted to do ? [11:24:04] <_joe_> akosiaris: I have already the file opened :) So I wanted to add the list of IP addresses to the upstream stanza in modules/profile/manifests/kubernetes/deployment_server/global_config.pp [11:24:23] <_joe_> at line 54 [11:24:38] <_joe_> then use that list to construct egress policies for envoy [11:25:04] <_joe_> the IP addresses are available in service::catalog IIRC [11:25:25] <_joe_> if you want to do that, be my guest :) [11:25:40] <_joe_> it's the usual puppet vertigo of nested data structures ofc [11:26:00] yeah that's actually more straightforward than kafka [11:26:02] <_joe_> I'm going afk now anyways, I'll tackle this later if you don't want to mess with that [11:26:24] <_joe_> yeah and the helm part is going to be a tad more complicated, to balance that :P [11:26:30] Yup [11:26:36] but not much (I think) [11:27:25] <_joe_> I thought of that while writing https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/670220/14..15/charts/mediawiki/templates/networkpolicy.yaml.tpl [11:27:45] <_joe_> "oh, but mediawiki connects to a ton of services, not just restbase 🤔" [11:28:09] <_joe_> ok, going afk for a while, ttyl [13:28:07] 10serviceops, 10SRE, 10Patch-For-Review: upgrade conf2* servers to stretch - https://phabricator.wikimedia.org/T271573 (10ops-monitoring-bot) Icinga downtime set by jayme@cumin1001 for 2:00:00 6 host(s) and their services with reason: for zookeeper migration ` conf[2001-2006].codfw.wmnet ` [13:39:44] 10serviceops, 10SRE, 10Thumbor, 10Performance-Team (Radar), 10User-jijiki: Build python-thumbor-wikimedia 2.9 Debian package and upload to apt.wikimedia.org - https://phabricator.wikimedia.org/T254845 (10Gilles) [14:03:36] 10serviceops, 10SRE, 10Patch-For-Review: upgrade conf2* servers to stretch - https://phabricator.wikimedia.org/T271573 (10ops-monitoring-bot) Icinga downtime set by jayme@cumin1001 for 1 day, 0:00:00 1 host(s) and their services with reason: for zookeeper migration ` conf2001.codfw.wmnet ` [14:15:56] 10serviceops, 10SRE, 10Patch-For-Review: upgrade conf2* servers to stretch - https://phabricator.wikimedia.org/T271573 (10JMeybohm) Switched zookeeper from conf2001 to conf2004. We decided to leave it like this for today and see if anything comes up. [14:16:57] 10serviceops, 10SRE, 10Thumbor, 10Performance-Team (Radar), 10User-jijiki: Build python-thumbor-wikimedia 2.9 Debian package and upload to apt.wikimedia.org - https://phabricator.wikimedia.org/T254845 (10Gilles) [14:19:06] 10serviceops, 10SRE, 10Thumbor, 10Performance-Team (Radar), 10User-jijiki: Build python-thumbor-wikimedia 2.9 Debian package and upload to apt.wikimedia.org - https://phabricator.wikimedia.org/T254845 (10Gilles) [14:35:39] 10serviceops, 10SRE, 10WMF-JobQueue, 10Patch-For-Review, 10Sustainability (Incident Followup): Have some dedicated jobrunners that aren't active videoscalers - https://phabricator.wikimedia.org/T279100 (10akosiaris) >>! In T279100#7033661, @jijiki wrote: >>>! In T279100#7019419, @akosiaris wrote: >> So,... [15:57:20] <_joe_> legoktm / mutante: does tomorrow at 16:00Z work for you two for an introductory meeting? [15:57:33] <_joe_> we already talked about shellbox with legoktm anyways [15:58:14] yep [16:03:25] ok [18:15:07] 10serviceops, 10Performance-Team, 10SRE, 10User-jijiki: Run latest Thumbor on Docker with Buster + Python 3 - https://phabricator.wikimedia.org/T267327 (10Gilles) a:05Gilles→03cmassaro [18:50:02] 10serviceops, 10SRE, 10Thumbor, 10User-jijiki: Upgrade Thumbor to Buster - https://phabricator.wikimedia.org/T216815 (10AntiCompositeNumber)