[00:32:29] 10Continuous-Integration-Infrastructure (Zuul upgrade): Create service account user for OpenTofu automation - https://phabricator.wikimedia.org/T396902#11007420 (10bd808) 05In progress→03Resolved Credentials added to Release Engineering 1password vault as: * ZuulDevOpsBot Developer account * ZuulDevOpsBo... [03:44:28] (03PS1) 10MusikAnimal: zuul: add CodeMirror as Phan dependency for JsonConfig, TemplateStyles [integration/config] - 10https://gerrit.wikimedia.org/r/1169805 (https://phabricator.wikimedia.org/T399659) [03:45:06] (03PS2) 10MusikAnimal: zuul: add CodeMirror as Phan dependency for JsonConfig, TemplateStyles [integration/config] - 10https://gerrit.wikimedia.org/r/1169805 (https://phabricator.wikimedia.org/T373711) [05:04:41] 06Release-Engineering-Team, 06SRE: Archiva Mirror Maven Central cache no space left on device - https://phabricator.wikimedia.org/T399679 (10amastilovic) 03NEW [05:59:03] 10Beta-Cluster-Infrastructure, 10Pywikibot, 07Pywikibot-tests: Find a way for pywikibot GitHub Actions to avoid IP range blocks of Microsoft Azure hosted runners - https://phabricator.wikimedia.org/T399485#11007690 (10Xqt) Proxy usage sounds promising . It’s usage is already available through requests packag... [06:24:35] 06Release-Engineering-Team, 10MediaWiki-Interwiki, 06translatewiki.net: Merge Interwiki extension into MediaWiki core - https://phabricator.wikimedia.org/T33951#11007731 (10A_smart_kitten) todo: - [X] finish undeploying the extension from `operations/mediawiki-config` - [ ] modify [[https://gitlab.wikimedi... [06:34:04] 10MediaWiki-Releasing, 10Release-Engineering-Team (Seen), 10MediaWiki-General: Use semantic(-ish) versioning for MediaWiki by dropping the "1." part of the version number - https://phabricator.wikimedia.org/T336128#11007771 (10Bugreporter) >>! In T336128#8831938, @Peachey88 wrote: > https://www.mediawiki.org... [07:19:12] (03PS2) 10Hashar: Zuul: [operations/software/opensearch/plugins] mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1169729 (https://phabricator.wikimedia.org/T399617) (owner: 10Ebernhardson) [07:19:42] (03CR) 10Hashar: [C:03+2] "We have an helper to archive the repo:" [integration/config] - 10https://gerrit.wikimedia.org/r/1169729 (https://phabricator.wikimedia.org/T399617) (owner: 10Ebernhardson) [07:21:04] (03Merged) 10jenkins-bot: Zuul: [operations/software/opensearch/plugins] mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1169729 (https://phabricator.wikimedia.org/T399617) (owner: 10Ebernhardson) [07:21:22] 06Release-Engineering-Team, 06Abstract Wikipedia team, 06collaboration-services, 10function-evaluator, and 2 others: gitlab CI mem issue - https://phabricator.wikimedia.org/T399348#11007932 (10Jelto) The jobs linked above were executed on the //Memory optimized Cloud Runners//. This runners are running in... [07:29:54] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: upgrade gitlab hosts to bookworm - https://phabricator.wikimedia.org/T399306#11007974 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1003 for host gitlab1004.wikimedia.org with OS bookworm [07:47:46] (03PS2) 10MusikAnimal: zuul: add CodeMirror as Phan dependency for JsonConfig, TemplateStyles [integration/config] - 10https://gerrit.wikimedia.org/r/1169805 (https://phabricator.wikimedia.org/T373711) [07:47:46] (03CR) 10MusikAnimal: "Very well then! Isn't there a deployment involved? I just didn't want to hassle you with that over and over :)" [integration/config] - 10https://gerrit.wikimedia.org/r/1169805 (https://phabricator.wikimedia.org/T373711) (owner: 10MusikAnimal) [07:50:28] (03PS1) 10Hashar: Stop setting EXECUTOR_NUMBER environment variable [integration/quibble] - 10https://gerrit.wikimedia.org/r/1170081 (https://phabricator.wikimedia.org/T299283) [07:51:46] (03PS2) 10Hashar: Stop setting EXECUTOR_NUMBER environment variable [integration/quibble] - 10https://gerrit.wikimedia.org/r/1170081 (https://phabricator.wikimedia.org/T399283) [07:52:44] (03CR) 10Hashar: [C:03+2] Stop setting EXECUTOR_NUMBER environment variable [integration/quibble] - 10https://gerrit.wikimedia.org/r/1170081 (https://phabricator.wikimedia.org/T399283) (owner: 10Hashar) [07:53:10] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06Fundraising-Backlog, 10Quibble, 13Patch-For-Review: Remove reliance on EXECUTOR_NUMBER environment variable in CI - https://phabricator.wikimedia.org/T399283#11008023 (10hashar) 05Open→03Resolved a:03hashar [08:09:15] (03CR) 10CI reject: [V:04-1] Stop setting EXECUTOR_NUMBER environment variable [integration/quibble] - 10https://gerrit.wikimedia.org/r/1170081 (https://phabricator.wikimedia.org/T399283) (owner: 10Hashar) [08:15:45] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: upgrade gitlab hosts to bookworm - https://phabricator.wikimedia.org/T399306#11008118 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1003 for host gitlab1004.wikimedia.org with OS bookworm completed: -... [08:21:55] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: upgrade gitlab hosts to bookworm - https://phabricator.wikimedia.org/T399306#11008139 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1003 for host gitlab1004.wikimedia.org with OS bookworm [08:56:00] 10GitLab: GitLab Private Repository Request for: - https://phabricator.wikimedia.org/T399696 (10Pablo) 03NEW [08:58:43] 10GitLab: GitLab Private Repository Request for: - https://phabricator.wikimedia.org/T399696#11008269 (10Pablo) [09:01:49] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: upgrade gitlab hosts to bookworm - https://phabricator.wikimedia.org/T399306#11008274 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1003 for host gitlab1004.wikimedia.org with OS bookworm executed wit... [09:04:17] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: upgrade gitlab hosts to bookworm - https://phabricator.wikimedia.org/T399306#11008275 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1003 for host gitlab1004.wikimedia.org with OS bookworm [09:05:39] 10GitLab: GitLab Private Repository Request for: research/npov-workstream-research - https://phabricator.wikimedia.org/T399696#11008276 (10Peachey88) [09:09:27] 10Phabricator (Upstream), 10Release-Engineering-Team (Radar), 07Developer Productivity, 07Upstream: Add Open Graph support to Phabricator Maniphest Tasks to have link preview on Telegram, Slack, and other messaging apps - https://phabricator.wikimedia.org/T288117#11008280 (10fnegri) FWIW, both the Link... [09:16:45] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: upgrade gitlab hosts to bookworm - https://phabricator.wikimedia.org/T399306#11008308 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1003 for host gitlab1004.wikimedia.org with OS bookworm executed wit... [09:19:09] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: upgrade gitlab hosts to bookworm - https://phabricator.wikimedia.org/T399306#11008311 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jelto@cumin1003 for host gitlab1004.wikimedia.org with OS bookworm [10:02:32] 10Phabricator: Update herald rule H126 - Add 'Data Platform' tag to list of trigger conditions - https://phabricator.wikimedia.org/T399700 (10BTullis) 03NEW [10:02:39] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: upgrade gitlab hosts to bookworm - https://phabricator.wikimedia.org/T399306#11008469 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jelto@cumin1003 for host gitlab1004.wikimedia.org with OS bookworm completed: -... [10:04:40] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: upgrade gitlab hosts to bookworm - https://phabricator.wikimedia.org/T399306#11008471 (10ops-monitoring-bot) Host gitlab1004.wikimedia.org rebooted by jelto@cumin1003 with reason: None [10:12:20] 10Continuous-Integration-Config, 10ConfirmEdit (CAPTCHA extension): Load Captcha sub extensions in CI - https://phabricator.wikimedia.org/T399702 (10Reedy) 03NEW [10:12:46] 10Phabricator (Upstream), 10Release-Engineering-Team (Radar), 07Developer Productivity, 07Upstream: Add Open Graph support to Phabricator Maniphest Tasks to have link preview on Telegram, Slack, and other messaging apps - https://phabricator.wikimedia.org/T288117#11008513 (10AntiCompositeNumber) >>! In... [10:13:33] 10Continuous-Integration-Config, 10ConfirmEdit (CAPTCHA extension): Load Captcha sub extensions in CI - https://phabricator.wikimedia.org/T399702#11008515 (10Reedy) [10:13:34] 10Continuous-Integration-Config, 10ConfirmEdit (CAPTCHA extension): ConfirmEdit: Load all sub-extensions in CI - https://phabricator.wikimedia.org/T399593#11008518 (10Reedy) →14Duplicate dup:03T399702 [10:13:53] 10Continuous-Integration-Config, 10ConfirmEdit (CAPTCHA extension): ConfirmEdit: Load all sub-extensions in CI - https://phabricator.wikimedia.org/T399593#11008531 (10Reedy) 05Duplicate→03Open Wrong way [10:13:55] 10Continuous-Integration-Config, 10ConfirmEdit (CAPTCHA extension): Load Captcha sub extensions in CI - https://phabricator.wikimedia.org/T399702#11008533 (10Dreamy_Jazz) [10:14:02] 10Continuous-Integration-Config, 10ConfirmEdit (CAPTCHA extension): Load Captcha sub extensions in CI - https://phabricator.wikimedia.org/T399702#11008537 (10Reedy) →14Duplicate dup:03T399593 [10:14:03] 10Continuous-Integration-Config, 10ConfirmEdit (CAPTCHA extension): ConfirmEdit: Load all sub-extensions in CI - https://phabricator.wikimedia.org/T399593#11008539 (10Reedy) [10:15:11] 10Continuous-Integration-Config, 10ConfirmEdit (CAPTCHA extension): Load Captcha sub extensions in CI - https://phabricator.wikimedia.org/T399702#11008543 (10Dreamy_Jazz) [10:24:32] 10Phabricator (Upstream), 10Release-Engineering-Team (Radar), 07Developer Productivity, 07Upstream: Add Open Graph support to Phabricator Maniphest Tasks to have link preview on Telegram, Slack, and other messaging apps - https://phabricator.wikimedia.org/T288117#11008585 (10AntiCompositeNumber) I trie... [10:24:42] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: upgrade gitlab hosts to bookworm - https://phabricator.wikimedia.org/T399306#11008586 (10ops-monitoring-bot) Host gitlab1004.wikimedia.org rebooted by jelto@cumin1003 with reason: None [10:51:51] 10Phabricator (Upstream), 10Release-Engineering-Team (Radar), 07Developer Productivity, 07Upstream: Add Open Graph support to Phabricator Maniphest Tasks to have link preview on Telegram, Slack, and other messaging apps - https://phabricator.wikimedia.org/T288117#11008664 (10AntiCompositeNumber) To eli... [11:43:26] 10Phabricator (Upstream), 07CSS, 07Upstream: Expanded long task title ("{T9999999}" markup) does not wrap in task description, so cannot be read fully - https://phabricator.wikimedia.org/T148829#11008810 (10Aklapper) 05Open→03Stalled [11:43:47] 10Phabricator (Upstream), 07Upstream: Alternative hashtag of milestone has no primary slug, thus project URL redirects to 404 /tag// - https://phabricator.wikimedia.org/T186173#11008824 (10Aklapper) 05Open→03Stalled [11:44:34] 10Phabricator (Upstream), 07Upstream: Calendar: Call to a member function format() on null at $this->getQueryDateFrom($query)->getDateTime() - https://phabricator.wikimedia.org/T372857#11008826 (10Aklapper) [11:44:35] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Update to Phorge upstream 2025.18 release - https://phabricator.wikimedia.org/T386558#11008827 (10Aklapper) [11:44:40] 10Phabricator (Upstream), 07Upstream: Calendar: Call to a member function format() on null at $this->getQueryDateFrom($query)->getDateTime() - https://phabricator.wikimedia.org/T372857#11008829 (10Aklapper) 05Open→03Stalled [11:44:56] 10Phabricator (Upstream), 10Release-Engineering-Team (Priority Backlog 📥), 07Upstream: New Column Trigger: Assign only unassigned tasks - https://phabricator.wikimedia.org/T393464#11008832 (10Aklapper) 05Open→03Stalled [11:46:33] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10MediaWiki-extensions-CentralAuth, 06MediaWiki-Platform-Team, and 2 others: CentralAuth locks should disable linked Phabricator account - https://phabricator.wikimedia.org/T338384#11008852 (10Aklapper) a:05Aklapper→03None [11:47:18] 10Phabricator (Upstream), 07Upstream: Option to measure WIP limits based on card count instead of points, to more closely adhere to Kanban standards. - https://phabricator.wikimedia.org/T120042#11008855 (10Aklapper) a:05Aklapper→03None [11:49:49] 10GitLab (Infrastructure), 06collaboration-services: GitLab partman config is unreliable - https://phabricator.wikimedia.org/T399714 (10Jelto) 03NEW [11:57:41] 10Phabricator, 10Release-Engineering-Team (Doing 😎): Update herald rule H126 - Add 'Data Platform' tag to list of trigger conditions - https://phabricator.wikimedia.org/T399700#11008889 (10Aklapper) 05Open→03Resolved a:03Aklapper Done [12:04:07] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 07Wikimedia-Incident: Investigate out of date refs following gerrit switchover - https://phabricator.wikimedia.org/T393034#11008919 (10LSobanski) 05Open→03Resolved a:03LSobanski Follow up work is happening in {T387833}, resolving. [12:25:06] (03update) 10jforrester: Stop branching of the Interwiki extension [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/192 (https://phabricator.wikimedia.org/T399636) (owner: 10zabe) [12:38:33] (03update) 10jforrester: Stop branching of the Interwiki extension for Wikimedia production and the MediaWiki tarball [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/192 (https://phabricator.wikimedia.org/T399636) (owner: 10zabe) [12:38:51] (03merge) 10jforrester: Stop branching of the Interwiki extension for Wikimedia production and the MediaWiki tarball [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/192 (https://phabricator.wikimedia.org/T399636) (owner: 10zabe) [12:40:29] (03update) 10jforrester: Added Metrics Platform Experimentation Lab deployment window to deployments calendar [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/186 (https://phabricator.wikimedia.org/T396742) (owner: 10sfaci) [12:43:22] (03approved) 10jforrester: Added Metrics Platform Experimentation Lab deployment window to deployments calendar [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/186 (https://phabricator.wikimedia.org/T396742) (owner: 10sfaci) [12:44:34] (03update) 10jforrester: Start branching CommunityConfigurationExample [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/188 (https://phabricator.wikimedia.org/T372049) (owner: 10urbanecm) [12:45:55] (03approved) 10jforrester: Start branching CommunityConfigurationExample [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/188 (https://phabricator.wikimedia.org/T372049) (owner: 10urbanecm) [12:47:22] 06Release-Engineering-Team, 10MediaWiki-Interwiki, 06translatewiki.net: Merge Interwiki extension into MediaWiki core - https://phabricator.wikimedia.org/T33951#11009073 (10Jdforrester-WMF) [12:47:47] (03CR) 10Hashar: [C:03+2] Stop setting EXECUTOR_NUMBER environment variable [integration/quibble] - 10https://gerrit.wikimedia.org/r/1170081 (https://phabricator.wikimedia.org/T399283) (owner: 10Hashar) [12:49:37] (03CR) 10Hashar: [C:03+2] zuul: add CodeMirror as Phan dependency for JsonConfig, TemplateStyles [integration/config] - 10https://gerrit.wikimedia.org/r/1169805 (https://phabricator.wikimedia.org/T373711) (owner: 10MusikAnimal) [12:50:46] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: upgrade gitlab hosts to bookworm - https://phabricator.wikimedia.org/T399306#11009081 (10Jelto) `gitlab1004` was reimaged (multiple times because of issues with the partman config T399714). Currently the raid is syncing and 50% done for... [12:51:13] (03Merged) 10jenkins-bot: zuul: add CodeMirror as Phan dependency for JsonConfig, TemplateStyles [integration/config] - 10https://gerrit.wikimedia.org/r/1169805 (https://phabricator.wikimedia.org/T373711) (owner: 10MusikAnimal) [13:08:16] (03Merged) 10jenkins-bot: Stop setting EXECUTOR_NUMBER environment variable [integration/quibble] - 10https://gerrit.wikimedia.org/r/1170081 (https://phabricator.wikimedia.org/T399283) (owner: 10Hashar) [13:47:42] (03PS6) 10Hashar: puppet: use bind mounts for local runs [integration/config] - 10https://gerrit.wikimedia.org/r/1135116 (owner: 10JHathaway) [13:47:53] (03Abandoned) 10Hashar: puppet: use bind mounts for local runs [integration/config] - 10https://gerrit.wikimedia.org/r/1142674 (owner: 10JHathaway) [13:48:05] (03CR) 10CI reject: [V:04-1] puppet: use bind mounts for local runs [integration/config] - 10https://gerrit.wikimedia.org/r/1135116 (owner: 10JHathaway) [13:48:06] (03CR) 10Zuul test: "Merge Failed." [integration/config] - 10https://gerrit.wikimedia.org/r/1135116 (owner: 10JHathaway) [13:48:30] (03CR) 10Hashar: "Patchset 6 is https://gerrit.wikimedia.org/r/c/integration/config/+/1142674 which was submitted independently." [integration/config] - 10https://gerrit.wikimedia.org/r/1135116 (owner: 10JHathaway) [13:58:16] 10MediaWiki-Releasing, 10Release-Engineering-Team (Seen), 10MediaWiki-General: Use semantic(-ish) versioning for MediaWiki by dropping the "1." part of the version number - https://phabricator.wikimedia.org/T336128#11009348 (10Tgr) Calendar versioning would be nice, too. We have a predictable cadence and the... [13:58:54] (03PS7) 10Hashar: utils: script to manually launch jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1132644 (https://phabricator.wikimedia.org/T389998) [13:59:19] (03CR) 10Hashar: "Restore change, patchset 6 was a resend of the obsolete patchset 2." [integration/config] - 10https://gerrit.wikimedia.org/r/1132644 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [14:00:53] (03PS8) 10Hashar: utils: script to manually launch jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1132644 (https://phabricator.wikimedia.org/T389998) [14:02:17] (03CR) 10Hashar: "Change php74 jobs to php81 one." [integration/config] - 10https://gerrit.wikimedia.org/r/1132644 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [14:11:45] (03merge) 10dancy: Add opensearch plugins deb repo [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/124 (https://phabricator.wikimedia.org/T399162) (owner: 10bking) [14:14:42] (03update) 10dancy: Start branching CommunityConfigurationExample [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/188 (https://phabricator.wikimedia.org/T372049) (owner: 10urbanecm) [14:17:11] (03PS1) 10Jforrester: Docker: [php83] Re-build based on Wikimedia-provided binaries [integration/config] - 10https://gerrit.wikimedia.org/r/1170158 (https://phabricator.wikimedia.org/T398245) [15:30:22] 06Release-Engineering-Team, 06Abstract Wikipedia team, 06collaboration-services, 10function-evaluator, and 2 others: gitlab CI mem issue - https://phabricator.wikimedia.org/T399348#11009805 (10Dzahn) We could possibly discuss with releng if Digital Ocean runners should still be default or maybe the other w... [15:34:02] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06cloud-services-team, 10Cloud-VPS: Cloud VPS project member (admin role) unable to grant k8s_admin, k8s_developer, k8s_viewer via `openstack role add` - https://phabricator.wikimedia.org/T399731 (10bd808) 03NEW [15:35:28] 06Release-Engineering-Team, 06collaboration-services, 10function-evaluator, 10function-orchestrator, and 3 others: gitlab CI mem issue - https://phabricator.wikimedia.org/T399348#11009837 (10Jdforrester-WMF) [15:35:40] 06Release-Engineering-Team, 06collaboration-services, 10function-evaluator, 10function-orchestrator, and 3 others: gitlab CI mem issue - https://phabricator.wikimedia.org/T399348#11009839 (10Jdforrester-WMF) 05Open→03In progress [15:55:03] (03update) 10dancy: spiderpig: Handle orphaned jobs [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/900 [16:19:48] I need a hashtag #CopyPasteIsForbidden [16:20:30] (03CR) 10Jforrester: [C:03+1] utils: script to manually launch jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1132644 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [16:35:13] hashar: Ship it! [16:35:28] I am looking at the composer* images :) [16:35:35] and raging at Docker and docker-pkg :] [16:36:29] (03update) 10dduvall: spiderpig: Handle orphaned jobs [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/900 (owner: 10dancy) [16:36:30] (03approved) 10dduvall: spiderpig: Handle orphaned jobs [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/900 (owner: 10dancy) [16:37:11] hashar: Fair. Also see my PHP 8.3 re-build, above. [16:37:15] (03merge) 10dancy: spiderpig: Handle orphaned jobs [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/900 [16:41:20] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06cloud-services-team, 10Cloud-VPS: Cloud VPS project member (admin role) unable to grant k8s_admin, k8s_developer, k8s_viewer via `openstack role add` - https://phabricator.wikimedia.org/T399731#11010170 (10bd808) Poking around in Puppet to see the Ope... [16:50:36] (03PS2) 10Hashar: Docker: [php83] Re-build based on Wikimedia-provided binaries [integration/config] - 10https://gerrit.wikimedia.org/r/1170158 (https://phabricator.wikimedia.org/T398245) (owner: 10Jforrester) [16:55:30] (03PS3) 10Hashar: Docker: [php83] Re-build based on Wikimedia-provided binaries [integration/config] - 10https://gerrit.wikimedia.org/r/1170158 (https://phabricator.wikimedia.org/T398491) (owner: 10Jforrester) [16:57:08] (03CR) 10Hashar: [C:03+1] "I have:" [integration/config] - 10https://gerrit.wikimedia.org/r/1170158 (https://phabricator.wikimedia.org/T398491) (owner: 10Jforrester) [16:57:11] hashar: Re-building locally to verify. [16:57:37] James_F: I have also changed the Bug: field [16:57:38] :) [16:57:42] hashar: But good spot on the node-browser-php ones. [16:57:47] yeah [16:57:53] you can do those in a follow up change I think [16:58:00] that might be easier to manage [16:58:01] :) [16:58:31] Those aren't using Wikimedia PHP for 8.1 either. [16:58:42] So yes, let's not do those now. [16:58:44] yeah it is probably fine [16:58:46] +1 [16:59:08] and I wonder how SRE builds those php 8.3 packages. I was looking at a clue in Gerrit but could not find the sources [16:59:43] more or lessrelated, php-tideways has been abandoned back in 2023 [16:59:50] I think they're manually building in root ~ on the build machines. [16:59:54] Yeah. [17:00:01] I think that is used for profiling, but we don'thave a performance team to babysit that anymore [17:00:07] 🤷 [17:00:28] manually directly on build host? [17:00:34] hmm yeah [17:01:04] I think I once had some php extension in Gerrit with CI and tests running [17:04:16] feel free to build the php83 images and update the jobs! [17:05:28] Ack. [17:30:28] (03CR) 10Jforrester: "I get an error:" [integration/config] - 10https://gerrit.wikimedia.org/r/1170158 (https://phabricator.wikimedia.org/T398491) (owner: 10Jforrester) [17:32:05] James_F: PHP Warning: PHP Startup: ^(text/|application/xhtml\+xml) (offset=0): unrecognised compile-time option bit(s) in Unknown on line 0 [17:32:09] Yup. [17:32:13] that is worth filing a task [17:32:26] it is almost certain the package has an issue of some sort [17:32:46] I am off, dinner time! [17:46:42] (03PS1) 10MusikAnimal: zuul: add CodeMirror as Phan dependency for UploadWizard [integration/config] - 10https://gerrit.wikimedia.org/r/1170191 (https://phabricator.wikimedia.org/T373711) [17:52:34] (03PS1) 10MusikAnimal: zuul: Add CodeMirror as Phan dependency of Gadgets [integration/config] - 10https://gerrit.wikimedia.org/r/1170193 (https://phabricator.wikimedia.org/T373711) [18:19:03] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): "Unhandled Exception ("RuntimeException") page with Undefined offset: 2" browsing any tasks for certain UI languages - https://phabricator.wikimedia.org/T399752#11010506 (10Aklapper) Crashes in https://we.phorge.it/source/arcanist/browse/master/sr... [18:21:39] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): "Unhandled Exception ("RuntimeException") page with Undefined offset: 2" browsing any tasks for certain UI languages - https://phabricator.wikimedia.org/T399752 (10Aklapper) 03NEW [18:29:57] 10Phabricator, 10Release-Engineering-Team (Seen), 10Wikimedia-Phabricator-Extensions: On project workboard cards, show status of associated Gerrit patches - https://phabricator.wikimedia.org/T215148#11010535 (10Aklapper) [18:46:43] 10Phabricator (Upstream), 10Release-Engineering-Team (Priority Backlog 📥), 07Upstream: "Unhandled Exception ("RuntimeException") page with Undefined offset: 2" browsing any tasks for certain UI languages - https://phabricator.wikimedia.org/T399752#11010661 (10Aklapper) [18:51:18] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06cloud-services-team, 10Cloud-VPS: Cloud VPS project member (formerly 'projectadmin') unable to grant k8s_admin, k8s_developer, k8s_viewer via `openstack role add` - https://phabricator.wikimedia.org/T399731#11010683 (10Andrew) [18:55:25] 10Phabricator (Upstream), 10Release-Engineering-Team (Priority Backlog 📥), 07Upstream: "Unhandled Exception ("RuntimeException") page with Undefined offset: 2" browsing any tasks for certain UI languages - https://phabricator.wikimedia.org/T399752#11010697 (10Aklapper) I guess a quick dirty hack would be to... [19:06:56] James_F: 100% that is libpcre2 [19:06:57] :) [19:23:26] (03CR) 10Hashar: [C:03+1] "That `libpcre2-8-0` which has the wrong version. PHP 8.3.0 came with libpcre 10.42 and thus our package should `Depends: libpcre-2-8.0>=10" [integration/config] - 10https://gerrit.wikimedia.org/r/1170158 (https://phabricator.wikimedia.org/T398491) (owner: 10Jforrester) [19:23:48] (03PS4) 10Hashar: Docker: [php83] Re-build based on Wikimedia-provided binaries [integration/config] - 10https://gerrit.wikimedia.org/r/1170158 (https://phabricator.wikimedia.org/T398245) (owner: 10Jforrester) [19:32:52] (03PS5) 10Hashar: Docker: [php83] Re-build based on Wikimedia-provided binaries [integration/config] - 10https://gerrit.wikimedia.org/r/1170158 (https://phabricator.wikimedia.org/T398491) (owner: 10Jforrester) [19:33:09] (03CR) 10Hashar: "Changed the task back to T398491 ... I guess I mixed up things" [integration/config] - 10https://gerrit.wikimedia.org/r/1170158 (https://phabricator.wikimedia.org/T398491) (owner: 10Jforrester) [19:38:23] (03CR) 10Hashar: "I went with the same workaround as for the php81 image: install `libpcre2-8-0` again which makes apt to pick the version from `component/p" [integration/config] - 10https://gerrit.wikimedia.org/r/1170158 (https://phabricator.wikimedia.org/T398491) (owner: 10Jforrester) [19:42:10] fixed [19:42:45] (03CR) 10Hashar: [C:03+2] zuul: add CodeMirror as Phan dependency for UploadWizard [integration/config] - 10https://gerrit.wikimedia.org/r/1170191 (https://phabricator.wikimedia.org/T373711) (owner: 10MusikAnimal) [19:42:46] (03CR) 10Hashar: [C:03+2] zuul: Add CodeMirror as Phan dependency of Gadgets [integration/config] - 10https://gerrit.wikimedia.org/r/1170193 (https://phabricator.wikimedia.org/T373711) (owner: 10MusikAnimal) [19:43:22] Dreamy_Jazz: I am not sure how UploadWizard would end up using CodeMirror :) [19:44:07] (03Merged) 10jenkins-bot: zuul: add CodeMirror as Phan dependency for UploadWizard [integration/config] - 10https://gerrit.wikimedia.org/r/1170191 (https://phabricator.wikimedia.org/T373711) (owner: 10MusikAnimal) [19:44:12] (03Merged) 10jenkins-bot: zuul: Add CodeMirror as Phan dependency of Gadgets [integration/config] - 10https://gerrit.wikimedia.org/r/1170193 (https://phabricator.wikimedia.org/T373711) (owner: 10MusikAnimal) [19:50:13] (03CR) 10Hashar: [C:03+2] "I have deploy this change a minute ago. I will recheck the two sources changes mentioned in the commit message" [integration/config] - 10https://gerrit.wikimedia.org/r/1169805 (https://phabricator.wikimedia.org/T373711) (owner: 10MusikAnimal) [20:06:13] 10Continuous-Integration-Infrastructure (Zuul upgrade): Make it possible to target other Cloud VPS projects with the zuul tofu automation - https://phabricator.wikimedia.org/T399768 (10bd808) 03NEW [20:12:19] (03CR) 10Hashar: [C:03+2] utils: script to manually launch jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1132644 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [20:13:34] (03CR) 10Hashar: [C:03+2] "That is merely so that we have it around :]" [integration/config] - 10https://gerrit.wikimedia.org/r/1132644 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [20:13:42] (03Merged) 10jenkins-bot: utils: script to manually launch jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1132644 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [20:14:12] * hashar ZzzZzzz [20:20:57] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06cloud-services-team, 10Cloud-VPS: Cloud VPS project member (formerly 'projectadmin') unable to grant k8s_admin, k8s_developer, k8s_viewer via `openstack role add` - https://phabricator.wikimedia.org/T399731#11010935 (10bd808) [20:49:02] 10Phabricator (Upstream), 10Release-Engineering-Team (Priority Backlog 📥), 07Upstream: "Unhandled Exception ("RuntimeException") page with Undefined offset: 2" browsing any tasks for certain UI languages - https://phabricator.wikimedia.org/T399752#11010981 (10Aklapper) [20:58:30] 06Release-Engineering-Team, 06collaboration-services, 10function-evaluator, 10function-orchestrator, and 3 others: gitlab CI mem issue - https://phabricator.wikimedia.org/T399348#11010993 (10bd808) >>! In T399348#11009805, @Dzahn wrote: > We could possibly discuss with releng if Digital Ocean runners shoul... [20:59:02] (03PS6) 10Jforrester: Docker: [php83] Re-build based on Wikimedia-provided binaries [integration/config] - 10https://gerrit.wikimedia.org/r/1170158 (https://phabricator.wikimedia.org/T398491) [20:59:16] (03CR) 10Jforrester: "Also applied to the Quibble one!" [integration/config] - 10https://gerrit.wikimedia.org/r/1170158 (https://phabricator.wikimedia.org/T398491) (owner: 10Jforrester) [20:59:22] (03CR) 10Jforrester: [C:03+2] Docker: [php83] Re-build based on Wikimedia-provided binaries [integration/config] - 10https://gerrit.wikimedia.org/r/1170158 (https://phabricator.wikimedia.org/T398491) (owner: 10Jforrester) [21:00:45] (03Merged) 10jenkins-bot: Docker: [php83] Re-build based on Wikimedia-provided binaries [integration/config] - 10https://gerrit.wikimedia.org/r/1170158 (https://phabricator.wikimedia.org/T398491) (owner: 10Jforrester) [21:01:06] !log Docker: [php83] Re-build based on Wikimedia-provided binaries, for T398491 [21:01:08] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:01:08] T398491: Move CI images to WMF PHP 8.3 packages - https://phabricator.wikimedia.org/T398491 [21:06:34] (03CR) 10Jforrester: [C:03+2] "Successfully published image docker-registry.discovery.wmnet/releng/php83-compile:8.3.23" [integration/config] - 10https://gerrit.wikimedia.org/r/1170158 (https://phabricator.wikimedia.org/T398491) (owner: 10Jforrester) [21:06:43] (03PS1) 10Jforrester: jjb: Update PHP 8.3 jobs to use Wikimedia-provide binary images [integration/config] - 10https://gerrit.wikimedia.org/r/1170212 (https://phabricator.wikimedia.org/T398491) [21:10:41] (03PS2) 10Jforrester: jjb: Update PHP 8.3 jobs to use Wikimedia-provide binary images [integration/config] - 10https://gerrit.wikimedia.org/r/1170212 (https://phabricator.wikimedia.org/T398491) [21:11:00] (03CR) 10Jforrester: [C:03+2] jjb: Update PHP 8.3 jobs to use Wikimedia-provide binary images [integration/config] - 10https://gerrit.wikimedia.org/r/1170212 (https://phabricator.wikimedia.org/T398491) (owner: 10Jforrester) [21:11:48] 10Continuous-Integration-Config, 13Patch-For-Review, 07PHP 8.3 support: Move CI images to WMF PHP 8.3 packages - https://phabricator.wikimedia.org/T398491#11011049 (10Jdforrester-WMF) 05Open→03Resolved a:03Jdforrester-WMF [21:12:34] (03Merged) 10jenkins-bot: jjb: Update PHP 8.3 jobs to use Wikimedia-provide binary images [integration/config] - 10https://gerrit.wikimedia.org/r/1170212 (https://phabricator.wikimedia.org/T398491) (owner: 10Jforrester) [21:13:13] (03PS5) 10Jforrester: zuul: Remove uncontrolled recursion in extra deps in MW jobs (take 2) [integration/config] - 10https://gerrit.wikimedia.org/r/1132543 (https://phabricator.wikimedia.org/T389998) (owner: 10Krinkle) [21:13:53] (03CR) 10Jforrester: "This should be re-generated (again, sorry), and then immediately deployed so that it's landed." [integration/config] - 10https://gerrit.wikimedia.org/r/1132543 (https://phabricator.wikimedia.org/T389998) (owner: 10Krinkle) [21:17:55] (03open) 10swfrench: make-container-image: revert to mediawiki-httpd [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/193 (https://phabricator.wikimedia.org/T378128) [21:27:41] !log Unblocked 191.19.0.0/17 (T399720) [21:27:43] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:28:59] 10Beta-Cluster-Infrastructure: Warning about /etc/acmecerts/unified contents during puppet run on deployment-cache-text08 & deployment-cache-upload08 - https://phabricator.wikimedia.org/T399419#11011103 (10bd808) [21:29:29] 10Beta-Cluster-Infrastructure, 10Acme-chief, 07Beta-Cluster-reproducible: Warning about /etc/acmecerts/unified contents during puppet run on deployment-cache-text08 & deployment-cache-upload08 - https://phabricator.wikimedia.org/T399419#11011106 (10bd808) [21:33:19] 10Beta-Cluster-Infrastructure, 10Acme-chief, 07Beta-Cluster-reproducible: Warning about /etc/acmecerts/unified contents during puppet run on deployment-cache-text08 & deployment-cache-upload08 - https://phabricator.wikimedia.org/T399419#11011122 (10Vgutierrez) You can clean old certs in the acme-chief instan... [21:34:55] 10Release-Engineering-Team (Yak Shaving 🐃🪒), 10Scap (SpiderPig 🕸️): SpiderPig should support train deployments - https://phabricator.wikimedia.org/T392610#11011129 (10thcipriani) 05In progress→03Resolved It lives! Clicking on a group on SpiderPig and then clicking "Let's Roll" moves the current release... [21:38:26] 10Scap: Integrate deploy security patch script to scap - https://phabricator.wikimedia.org/T389325#11011149 (10thcipriani) →14Duplicate dup:03T361709 [21:38:29] 10Release-Engineering-Team (Yak Shaving 🐃🪒), 10Scap: scap: Please provide a subcommand to deploy a new security patch in scap itself - https://phabricator.wikimedia.org/T361709#11011151 (10thcipriani) [21:42:45] 10Release-Engineering-Team (Yak Shaving 🐃🪒), 10Scap: scap: Please provide a subcommand to deploy a new security patch in scap itself - https://phabricator.wikimedia.org/T361709#11011156 (10thcipriani) @sbassett what do you think of think of this? I think I'm interested here since: * looks like there's some d... [21:58:58] (03open) 10dancy: Log.vue: Sort by descending count [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/901 (https://phabricator.wikimedia.org/T397091) [21:59:01] (03update) 10dancy: Log.vue: Sort by descending count [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/901 (https://phabricator.wikimedia.org/T397091) [22:01:05] (03merge) 10dancy: Log.vue: Sort by descending count [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/901 (https://phabricator.wikimedia.org/T397091) [22:05:07] 10Scap (SpiderPig 🕸️): Move error log view to its own tab - https://phabricator.wikimedia.org/T399777 (10dancy) 03NEW [22:06:29] (03open) 10dancy: spiderpig: move error log viewer to its own tab [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/902 (https://phabricator.wikimedia.org/T399777) [22:06:32] (03update) 10dancy: spiderpig: move error log viewer to its own tab [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/902 (https://phabricator.wikimedia.org/T399777) [22:07:10] 10Scap (SpiderPig 🕸️), 13Patch-For-Review: Move error log view to its own tab - https://phabricator.wikimedia.org/T399777#11011252 (10dancy) 05Open→03In progress p:05Triage→03Medium [22:08:43] (03update) 10dancy: spiderpig: move error log viewer to its own tab [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/902 (https://phabricator.wikimedia.org/T399777) [22:09:03] 06Release-Engineering-Team, 06collaboration-services, 10function-evaluator, 10function-orchestrator, and 3 others: gitlab CI mem issue - https://phabricator.wikimedia.org/T399348#11011257 (10Dzahn) Well, we also had (different) CI failures in the other direction. That were not an issue on wmcs runners but... [22:13:22] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06cloud-services-team, 10Cloud-VPS: Cloud VPS project member (formerly 'projectadmin') unable to grant k8s_admin, k8s_developer, k8s_viewer via `openstack role add` - https://phabricator.wikimedia.org/T399731#11011261 (10Andrew) I have confirmed that th... [22:27:03] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06cloud-services-team, 10Cloud-VPS: Cloud VPS project member (formerly 'projectadmin') unable to grant k8s_admin, k8s_developer, k8s_viewer via `openstack role add` - https://phabricator.wikimedia.org/T399731#11011323 (10bd808) >>! In T399731#11011261,... [22:43:15] 10Continuous-Integration-Infrastructure (Zuul upgrade): Update ZuulDevOpsBot app credentials to include admin role access - https://phabricator.wikimedia.org/T399780 (10bd808) 03NEW [22:43:51] 10Continuous-Integration-Infrastructure (Zuul upgrade): Update ZuulDevOpsBot app credentials to include admin role access - https://phabricator.wikimedia.org/T399780#11011383 (10bd808) p:05Triage→03High a:03bd808 [22:57:11] 10Beta-Cluster-Infrastructure, 10Acme-chief, 07Beta-Cluster-reproducible: Warning about /etc/acmecerts/unified contents during puppet run on deployment-cache-text08 & deployment-cache-upload08 - https://phabricator.wikimedia.org/T399419#11011417 (10bd808) >>! In T399419#11011122, @Vgutierrez wrote: > You can... [22:58:07] (03approved) 10brennen: spiderpig: move error log viewer to its own tab [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/902 (https://phabricator.wikimedia.org/T399777) (owner: 10dancy) [23:28:45] 10Phabricator (Upstream), 10Release-Engineering-Team (Doing 😎), 07Upstream: "Unhandled Exception ("RuntimeException") page with Undefined offset: 2" browsing any tasks for certain UI languages - https://phabricator.wikimedia.org/T399752#11011486 (10Aklapper) a:03Aklapper [23:38:28] 10Beta-Cluster-Infrastructure, 10Pywikibot, 07Pywikibot-tests: Find a way for pywikibot GitHub Actions to avoid IP range blocks of Microsoft Azure hosted runners - https://phabricator.wikimedia.org/T399485#11011491 (10bd808) >>! In T399485#11007690, @Xqt wrote: > Proxy usage sounds promising . It’s usage is...