[15:37:38] 10serviceops, 10MW-on-K8s, 10SRE, 10observability: Add support for scraping php applications to the kubernetes prometheus scraper - https://phabricator.wikimedia.org/T271822 (10Joe) [16:07:48] 10serviceops, 10Product-Infrastructure-Team-Backlog, 10Recommendation-API, 10Release-Engineering-Team, and 2 others: Migrate recommendation-api to kubernetes - https://phabricator.wikimedia.org/T241230 (10Andrew) Changes associated with this patch (mostly 27152427487bed18321d582a7f77a301ea114968) have left... [16:33:57] 10serviceops, 10Product-Infrastructure-Team-Backlog, 10Recommendation-API, 10Release-Engineering-Team, and 2 others: Migrate recommendation-api to kubernetes - https://phabricator.wikimedia.org/T241230 (10akosiaris) >>! In T241230#6740058, @Andrew wrote: > Changes associated with this patch (mostly 2715242... [16:56:51] btw, I can't say I 've ever conformed to this pattern of 2.4.38-1-s2 so if we want to actually use it we need to talk a bit about it and what problems it solves for us (and what problems it creates of course). [16:57:17] I think across production-images we very sparsely use it if at all. [19:07:21] 10serviceops, 10SRE, 10Patch-For-Review, 10Release-Engineering-Team (Deployment services), and 2 others: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for host... [19:08:38] 10serviceops, 10SRE, 10Patch-For-Review, 10Release-Engineering-Team (Deployment services), and 2 others: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['mw2224.codfw.wmnet'] ` Of which those **F... [19:11:26] o/ jayme. mforns and I need to restart the eventgate-wikimedia containers in the eventgate-analytics-external deployment [19:12:04] to clear caches...we made a mistake with modifying a schema that should have been immuatable [19:12:11] and forgotten that eventgate would have it cached [19:12:23] what's the best way to do this? i can kill pods one at a time until lthey are all relaunched [19:12:25] is there a better way? [19:13:26] or akosiaris advice maybe ^ ? [21:24:37] ottomata: helmfile -e production/main/whatever sync --args --recreate-pods [21:25:53] 10serviceops, 10SRE, 10Patch-For-Review, 10Release-Engineering-Team (Deployment services), and 2 others: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for host... [21:27:22] 10serviceops, 10SRE, 10Patch-For-Review, 10Release-Engineering-Team (Deployment services), and 2 others: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by dzahn on cumin1001.eqiad.wmnet for host... [21:48:28] awewsooOMe [21:48:31] thanks akosiaris updating docs [22:44:28] 10serviceops, 10SRE, 10Patch-For-Review, 10Release-Engineering-Team (Deployment services), and 2 others: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['mw2224.codfw.wmnet'] ` and were **ALL** s... [22:47:13] 10serviceops, 10SRE, 10Patch-For-Review, 10Release-Engineering-Team (Deployment services), and 2 others: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['mw2225.codfw.wmnet'] ` and were **ALL** s...