[07:41:32] 10serviceops, 10DBA, 10Operations, 10Goal: Strengthen backup infrastructure and support - https://phabricator.wikimedia.org/T229209 (10jcrespo) [08:35:40] 10serviceops, 10DBA, 10Operations, 10Goal: Strengthen backup infrastructure and support - https://phabricator.wikimedia.org/T229209 (10jcrespo) [08:36:44] 10serviceops, 10DBA, 10Operations, 10Goal: Strengthen backup infrastructure and support - https://phabricator.wikimedia.org/T229209 (10jcrespo) We may need some firmware updates, but hw is ready to go as soon as background raid initialization finishes on array2 of backup1001. Hosts installed with buster.... [09:20:26] 10serviceops, 10Operations, 10Continuous-Integration-Infrastructure (phase-out-jessie): Upload docker-ce 18.06.3 upstream package for Stretch - https://phabricator.wikimedia.org/T226236 (10MoritzMuehlenhoff) Sorry, no. We're not going to intentionally downgrade to an old, known-insecure version. If there's a... [09:38:15] 10serviceops, 10Operations, 10Release Pipeline, 10local-charts, 10Kubernetes: Set up CI for the deployment-charts repository - https://phabricator.wikimedia.org/T233291 (10Joe) [09:45:27] 10serviceops, 10Operations, 10Release Pipeline, 10local-charts, and 2 others: Set up CI for the deployment-charts repository - https://phabricator.wikimedia.org/T233291 (10MarcoAurelio) [09:51:06] 10serviceops, 10Operations, 10Continuous-Integration-Infrastructure (phase-out-jessie): Upload docker-ce 18.06.3 upstream package for Stretch - https://phabricator.wikimedia.org/T226236 (10hashar) 05Open→03Declined It is not about downgrading Docker, but rather to keep the same version we are currently u... [10:03:49] 10serviceops, 10Operations, 10Release Pipeline, 10local-charts, and 2 others: Set up CI for the deployment-charts repository - https://phabricator.wikimedia.org/T233291 (10hashar) @Jdforrester-WMF has added an experimental `helm-lint` job to the repository: T216049. It runs `help lint --strict charts/*/` :] [10:05:22] <_joe_> hashar: oh meh I never saw that task [10:05:35] <_joe_> or maybe I did [10:05:49] <_joe_> anyways, what is that experimental job using as a docker image? [10:07:05] 10serviceops, 10Operations, 10Continuous-Integration-Infrastructure (phase-out-jessie): Upload docker-ce 18.06.3 upstream package for Stretch - https://phabricator.wikimedia.org/T226236 (10MoritzMuehlenhoff) >>! In T226236#5505608, @hashar wrote: > Anywa,y I am declining this and postpone the migration to St... [10:09:25] <_joe_> yeah found it and I think I'll just fix it then [10:34:58] 10serviceops, 10Operations, 10Release Pipeline, 10local-charts, and 2 others: Set up CI for the deployment-charts repository - https://phabricator.wikimedia.org/T233291 (10Joe) >>! In T233291#5505631, @hashar wrote: > @Jdforrester-WMF has added an experimental `helm-lint` job to the repository: T216049. It... [10:35:19] 10serviceops, 10Operations, 10Release Pipeline, 10local-charts, and 2 others: Set up CI for the deployment-charts repository - https://phabricator.wikimedia.org/T233291 (10Joe) p:05Triage→03High a:03Joe [10:36:17] 10serviceops, 10Operations, 10Release Pipeline, 10local-charts, and 2 others: Set up CI for the deployment-charts repository - https://phabricator.wikimedia.org/T233291 (10Joe) [10:36:21] 10serviceops, 10Continuous-Integration-Infrastructure, 10Continuous-Integration-Config, 10Kubernetes, 10User-fsero: add CI job into operations/deployments-charts repo that helm lint packages and perform the helm index after merge. - https://phabricator.wikimedia.org/T216049 (10Joe) [16:31:58] 10serviceops, 10Mobile-Content-Service, 10Page Content Service, 10Patch-For-Review, 10Product-Infrastructure-Team-Backlog (Kanban): "worker died, restarting" mobileapps issue - https://phabricator.wikimedia.org/T229286 (10Mholloway) Increasing the number of workers by 50% appears had a different effect t... [18:43:53] 10serviceops, 10Mobile-Content-Service, 10Page Content Service, 10Patch-For-Review, 10Product-Infrastructure-Team-Backlog (Kanban): "worker died, restarting" mobileapps issue - https://phabricator.wikimedia.org/T229286 (10mobrovac) >>! In T229286#5506798, @Mholloway wrote: > Increasing the number of work... [19:00:31] 10serviceops, 10Operations, 10Patch-For-Review: Reallocate former image scalers - https://phabricator.wikimedia.org/T192457 (10Dzahn) [19:00:36] 10serviceops, 10Operations, 10Patch-For-Review: Reallocate former image scalers - https://phabricator.wikimedia.org/T192457 (10Dzahn) 05Stalled→03Resolved a:03Dzahn [19:13:37] 10serviceops, 10Core Platform Team, 10Performance-Team, 10Scap, and 5 others: Define variant Wikimedia production config in compiled, static files - https://phabricator.wikimedia.org/T223602 (10Jdforrester-WMF) [19:14:05] 10serviceops, 10Core Platform Team, 10Performance-Team, 10Scap, and 5 others: Define variant Wikimedia production config in compiled, static files - https://phabricator.wikimedia.org/T223602 (10Jdforrester-WMF) [20:12:36] 10serviceops, 10Core Platform Team, 10Performance-Team, 10Scap, and 5 others: Define variant Wikimedia production config in compiled, static files - https://phabricator.wikimedia.org/T223602 (10Krinkle) >>! In T223602#5455095, @gerritbot wrote: > Change 507729 had a related patch set uploaded (by Jforreste... [22:25:05] 10serviceops, 10Release-Engineering-Team-TODO: Blubberoid endpoint intermittently routing to MediaWiki backend - https://phabricator.wikimedia.org/T233369 (10dduvall) [22:26:16] akosiaris, _joe_, et al: ^ [22:26:57] considered making it UBN but thought i'd ping you all first [22:28:02] cc thcipriani [22:28:44] 10serviceops, 10Release-Engineering-Team-TODO: Blubberoid endpoint intermittently routing to MediaWiki backend - https://phabricator.wikimedia.org/T233369 (10dduvall) p:05Triage→03High [22:29:46] after midnight for most of Europe, don't know if there's anyone else lurking here that can take a look, otherwise it might be overnight [22:30:15] i'm fine with that if you are [22:30:38] i mean, with respect to what i was working on. just seems generally not so good? :) [22:31:16] heh, yes, generally not so good, but not bad enough :) [22:32:36] :) [22:33:02] * marxarelli looks up the opposite of hyperbole [22:38:11] <_joe_> marxarelli: I'm just passing by, but tag traffic for it [22:38:28] _joe_: got it [22:39:00] <_joe_> cp1075 is the experimental ats text backend [22:40:01] oic. experimental backend seems to square with intermittent behavior [22:40:57] <_joe_> yes [22:41:07] <_joe_> so ping brandon or valentin I guess [22:41:13] <_joe_> I'll try to remember tomorrow [22:58:45] marxarelli: "understatement" :) [22:59:08] word! word... [23:46:33] 10serviceops, 10Core Platform Team, 10Performance-Team, 10Scap, and 5 others: Define variant Wikimedia production config in compiled, static files - https://phabricator.wikimedia.org/T223602 (10Jdforrester-WMF) >>! In T223602#5507683, @Krinkle wrote: >>>! In T223602#5455095, @gerritbot wrote: >> Change 507...