[00:12:35] 06Release-Engineering-Team (Doing 😎), 07Essential-Work: 500 error "attempt to write a readonly database" on releng-data.wmcloud.org - https://phabricator.wikimedia.org/T415139#11539878 (10thcipriani) >>! In T415139#11539734, @SLong-WMF wrote: > So was the root cause here that external crawlers were causing... [00:23:19] 10Fresh, 06Release-Engineering-Team, 13Patch-For-Review: Fresh lacks a persistent npm cache across multiple sessions - https://phabricator.wikimedia.org/T349276#11539908 (10Krinkle) [00:38:57] (03approved) 10thcipriani: scap update-patch/remove-patch: Lock staging directory during operation [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1068 (https://phabricator.wikimedia.org/T414628) (owner: 10dancy) [00:39:10] (03update) 10thcipriani: scap update-patch/remove-patch: Lock staging directory during operation [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1068 (https://phabricator.wikimedia.org/T414628) (owner: 10dancy) [01:00:01] 10Fresh, 06Release-Engineering-Team, 13Patch-For-Review: Fresh lacks a persistent npm cache across multiple sessions - https://phabricator.wikimedia.org/T349276#11540029 (10Krinkle) >>! In T349276#11429160, @Lucas_Werkmeister_WMDE wrote: > […] > FWIW, what I had in mind was no option / CLI API at all. I thin... [03:11:28] FIRING: InstanceDown: Project deployment-prep instance deployment-sessionstore06 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [03:16:28] RESOLVED: InstanceDown: Project deployment-prep instance deployment-sessionstore06 is down - https://prometheus-alerts.wmcloud.org/?q=alertname%3DInstanceDown [04:18:05] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for Keerthan 16 - https://phabricator.wikimedia.org/T415162 (10Keerthan_16) 03NEW [07:54:10] 06Release-Engineering-Team (Priority Backlog 📥), 07Essential-Work, 05Release, 05Train Deployments: 1.46.0-wmf.12 deployment blockers - https://phabricator.wikimedia.org/T413803#11540392 (10IKhitron) >>! In T413803#11539595, @matmarex wrote: > Thanks for catching that GlobalWatchlist bug @IKhitron. Summary... [08:57:41] (03PS1) 10Hashar: Switch OpenSearch Quibble jobs to php 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1229521 [08:59:02] (03CR) 10Hashar: [C:03+2] Switch OpenSearch Quibble jobs to php 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1229521 (owner: 10Hashar) [08:59:50] (03CR) 10Hashar: "recheck I have moved the OpenSearch job from PHP 8.1 to 8.3 ( I59d32a7fb6e8d0a398e240600d024c41f6c24788 )" [integration/quibble] - 10https://gerrit.wikimedia.org/r/1229064 (owner: 10Hashar) [09:00:34] (03Merged) 10jenkins-bot: Switch OpenSearch Quibble jobs to php 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1229521 (owner: 10Hashar) [09:21:20] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for Keerthan 16 - https://phabricator.wikimedia.org/T415162#11540509 (10Aklapper) 05Open→03Invalid That existing account does not seem to require activation, so I do not know why this ticket was created? If yo... [09:49:17] 06Release-Engineering-Team (Priority Backlog 📥), 07Essential-Work, 05Release, 05Train Deployments: 1.46.0-wmf.12 deployment blockers - https://phabricator.wikimedia.org/T413803#11540542 (10Aklapper) Since deployment of 1.46.0-wmf.12 to group1, Commons experiences DBTransactionErrors on webvideotranscode ho... [10:02:33] [Train] I'm a bit nervous about https://phabricator.wikimedia.org/T415169 (Transaction error on webvideotranscode hosts) but in my judgment it is not a reason to roll back. [10:26:40] (03PS1) 10Hashar: Zuul: [WikiEditor] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229528 (https://phabricator.wikimedia.org/T389998) [10:28:14] (03CR) 10Hashar: [C:03+2] Zuul: [WikiEditor] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229528 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [10:29:43] (03Merged) 10jenkins-bot: Zuul: [WikiEditor] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229528 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [10:35:38] 06Release-Engineering-Team (Radar), 10Ceph, 06ServiceOps new, 10SRE-swift-storage, and 3 others: Move the docker registry's /restricted prefix to Docker Distribution backed up by Ceph - https://phabricator.wikimedia.org/T412951#11540661 (10MatthewVernon) There is currently 3T of apus quota allocated to the... [11:00:11] 06Release-Engineering-Team (Radar), 10Ceph, 06ServiceOps new, 10SRE-swift-storage, and 3 others: Move the docker registry's /restricted prefix to Docker Distribution backed up by Ceph - https://phabricator.wikimedia.org/T412951#11540714 (10elukey) >>! In T412951#11539601, @Scott_French wrote: > Thank you v... [11:07:58] 10Fresh, 06Release-Engineering-Team, 13Patch-For-Review: Fresh lacks a persistent npm cache across multiple sessions - https://phabricator.wikimedia.org/T349276#11540741 (10Lucas_Werkmeister_WMDE) >>! In T349276#11540029, @Krinkle wrote: > I default workflow that I recommend, which is reflected in docs for M... [11:35:16] (03PS1) 10Hashar: Zuul: [AbuseFilter] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229546 (https://phabricator.wikimedia.org/T389998) [11:44:54] hashar: thanks for deploying the IPInfo/IPReputation dependencies config patch. There still seems to be an issue, though, because IPReputation isn't loaded for this job https://integration.wikimedia.org/ci/job/quibble-with-gated-extensions-vendor-mysql-php83/8111/consoleFull [12:39:54] 10Continuous-Integration-Config, 07Epic, 07PHP 8.4 support: Make PHP 8.4 voting on development (master) branch of MW ecosystem (core, vendor, extensions, skins, libraries) in CI - https://phabricator.wikimedia.org/T386108#11540991 (10Michael) [12:43:19] (03CR) 10Hashar: [C:03+2] Zuul: [AbuseFilter] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229546 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [12:44:57] (03Merged) 10jenkins-bot: Zuul: [AbuseFilter] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229546 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [13:05:46] MichaelG_WMF: re https://phabricator.wikimedia.org/T386108#11540991; FWIW, I think the Parsoid PHP 8.4 issue has been resolved as far as CI is concerned, but (I think) the task was reopened due to questions about further backports. [13:12:26] (03PS1) 10Dreamy Jazz: Zuul: [mediawiki/extensions/LoginNotify] Indicate CheckUser dependency status [integration/config] - 10https://gerrit.wikimedia.org/r/1229559 (https://phabricator.wikimedia.org/T412939) [13:26:09] (03PS1) 10Hashar: Zuul: [Flow] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229561 (https://phabricator.wikimedia.org/T389998) [13:26:26] (03CR) 10Hashar: [C:03+2] Zuul: [Flow] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229561 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [13:27:58] (03Merged) 10jenkins-bot: Zuul: [Flow] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229561 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [13:29:37] @A_smart_kitten I might be wrong, but my reading of that conversation was that the backports are needed to make PHP8.4 job voting for those release branches. But maybe I misunderstood this, that would be quite likely. [13:33:24] (03open) 10trueg: flink-rdf-streaming-updater: new Gitlab namespace [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/140 (https://phabricator.wikimedia.org/T414430) [13:56:55] MichaelG_WMF, A_smart_kitten: Indeed. Not resolved until (a) merged, (b) CI configured to enforce, and (c) released for third parties to use. [13:58:28] James_F: ah, okay. FWIW, as far as I *understood*, I didn't think it'd be necessary to backport https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/704745 to 1.44/1.43 in order to get PHP 8.4 CI to work for those branches. (I'm testing PHPUnit tests on REL1_44/PHP8.4 just now to try and prove/disprove that for myself, but I seem to be running into an unrelated error...yay) [13:59:06] (03PS1) 10Hashar: Zuul: [skins/Cosmos] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229566 (https://phabricator.wikimedia.org/T389998) [13:59:08] A_smart_kitten: It's not required that you personally do the checking and backporting, no worries. The Parsoid team whose code it is should decide. :-) [13:59:08] (03PS1) 10Hashar: Zuul: [skins/Refreshed] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229567 (https://phabricator.wikimedia.org/T389998) [13:59:48] James_F: Oh no I know it's not required, don't worry. I just like to figure out these things if I can (and also now I want to figure out why PHPUnit is crashing on REL1_44/PHP8.4!) [14:01:26] (03CR) 10Hashar: [C:03+2] Zuul: [skins/Cosmos] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229566 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [14:01:37] (03CR) 10Hashar: [C:03+2] Zuul: [skins/Refreshed] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229567 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [14:02:50] 06Release-Engineering-Team (Radar), 10Scap, 10MediaWiki-Internationalization, 13Patch-For-Review, 07Wikimedia-Performance-recommendation: Use static php array files for l10n cache at WMF (instead of CDB) - https://phabricator.wikimedia.org/T99740#11541292 (10Krinkle) [14:03:35] (03Merged) 10jenkins-bot: Zuul: [skins/Cosmos] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229566 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [14:03:37] (03Merged) 10jenkins-bot: Zuul: [skins/Refreshed] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229567 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [14:07:33] 10Phabricator maintenance bot, 06MW-Interfaces-Team: Consider also automatically tagging tasks for #MediaWiki-libs-Rdbms & #MediaWiki-Core-JobQueue with #MW-Interfaces-Team - https://phabricator.wikimedia.org/T414268#11541303 (10HCoplin-WMF) p:05Triage→03Low a:03HCoplin-WMF Thanks for the catch, @A_smart... [14:14:57] (03PS1) 10Hashar: utils: zuul-mw-jobs-runner: add mediawiki/services/parsoid [integration/config] - 10https://gerrit.wikimedia.org/r/1229575 [14:15:10] (03CR) 10Hashar: [C:03+2] utils: zuul-mw-jobs-runner: add mediawiki/services/parsoid [integration/config] - 10https://gerrit.wikimedia.org/r/1229575 (owner: 10Hashar) [14:16:48] (03Merged) 10jenkins-bot: utils: zuul-mw-jobs-runner: add mediawiki/services/parsoid [integration/config] - 10https://gerrit.wikimedia.org/r/1229575 (owner: 10Hashar) [14:28:55] (03PS1) 10Hashar: utils: zuul-mw-jobs-runner: mediawiki/services/parsoid/testreduce [integration/config] - 10https://gerrit.wikimedia.org/r/1229578 [14:29:38] (03PS2) 10Hashar: utils: zuul-mw-jobs-runner: mediawiki/services/parsoid/testreduce [integration/config] - 10https://gerrit.wikimedia.org/r/1229578 [14:31:34] (03PS1) 10Hashar: Zuul: [mediawiki/services/parsoid] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229580 (https://phabricator.wikimedia.org/T389998) [14:32:04] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/services/parsoid] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229580 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [14:32:09] (03CR) 10Hashar: [C:03+2] utils: zuul-mw-jobs-runner: mediawiki/services/parsoid/testreduce [integration/config] - 10https://gerrit.wikimedia.org/r/1229578 (owner: 10Hashar) [14:33:28] (03CR) 10Hashar: [C:03+2] "Thanks for the note! Later the dependencies will be directly in the repositories and hence would be able to be different between branches " [integration/config] - 10https://gerrit.wikimedia.org/r/1229559 (https://phabricator.wikimedia.org/T412939) (owner: 10Dreamy Jazz) [14:33:55] (03Merged) 10jenkins-bot: utils: zuul-mw-jobs-runner: mediawiki/services/parsoid/testreduce [integration/config] - 10https://gerrit.wikimedia.org/r/1229578 (owner: 10Hashar) [14:33:55] (03Merged) 10jenkins-bot: Zuul: [mediawiki/services/parsoid] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229580 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [14:35:14] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/LoginNotify] Indicate CheckUser dependency status [integration/config] - 10https://gerrit.wikimedia.org/r/1229559 (https://phabricator.wikimedia.org/T412939) (owner: 10Dreamy Jazz) [14:40:11] (03CR) 10Hashar: [C:03+2] "I023bee0d45cf605efefb4a29a6d545e766355c8d has fixed it!" [integration/config] - 10https://gerrit.wikimedia.org/r/1226822 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [15:10:55] (03CR) 10Hashar: [C:03+2] "I can't `+2` it yet cause the `REL1_43` branches has some other pending fixes" [integration/config] - 10https://gerrit.wikimedia.org/r/1226822 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [15:12:29] (03Merged) 10jenkins-bot: Zuul: [Wikistories] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1226822 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [15:20:39] 06Release-Engineering-Team (Doing 😎), 10wikimedia.biterg.io: Bitergia account for Tajh Taylor - https://phabricator.wikimedia.org/T415203 (10thcipriani) 03NEW [15:20:56] 06Release-Engineering-Team (Doing 😎), 10wikimedia.biterg.io: Bitergia account for Tajh Taylor - https://phabricator.wikimedia.org/T415203#11542202 (10thcipriani) p:05Triage→03Medium a:03thcipriani [15:42:28] (03merge) 10dancy: flink-rdf-streaming-updater: new Gitlab namespace [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/140 (https://phabricator.wikimedia.org/T414430) (owner: 10trueg) [15:51:34] (03update) 10dancy: scap update-patch/remove-patch: Lock staging directory during operation [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1068 (https://phabricator.wikimedia.org/T414628) [15:57:29] (03PS1) 10Hashar: Zuul: [AdhocTranslation] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229593 (https://phabricator.wikimedia.org/T389998) [15:57:31] (03PS1) 10Hashar: Zuul: [CognitiveProcessDesigner] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229594 (https://phabricator.wikimedia.org/T389998) [15:59:02] (03PS1) 10Hashar: Zuul: [FlexiSkin] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229595 (https://phabricator.wikimedia.org/T389998) [16:01:12] (03PS1) 10Hashar: Zuul: [MenuEditor] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229596 (https://phabricator.wikimedia.org/T389998) [16:09:36] (03PS1) 10Hashar: Zuul: [UserProfile] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229602 (https://phabricator.wikimedia.org/T389998) [16:10:54] (03CR) 10Hashar: [C:03+2] Zuul: [AdhocTranslation] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229593 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [16:10:58] (03CR) 10Hashar: [C:03+2] Zuul: [CognitiveProcessDesigner] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229594 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [16:11:01] (03CR) 10Hashar: [C:03+2] Zuul: [FlexiSkin] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229595 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [16:11:05] (03CR) 10Hashar: [C:03+2] Zuul: [MenuEditor] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229596 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [16:11:18] (03CR) 10Hashar: [C:03+2] Zuul: [UserProfile] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229602 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [16:12:53] (03Merged) 10jenkins-bot: Zuul: [AdhocTranslation] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229593 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [16:13:09] (03Merged) 10jenkins-bot: Zuul: [CognitiveProcessDesigner] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229594 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [16:13:10] (03Merged) 10jenkins-bot: Zuul: [FlexiSkin] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229595 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [16:13:12] (03Merged) 10jenkins-bot: Zuul: [MenuEditor] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229596 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [16:13:17] (03Merged) 10jenkins-bot: Zuul: [UserProfile] disable recursion [integration/config] - 10https://gerrit.wikimedia.org/r/1229602 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [16:14:25] (03merge) 10dancy: scap update-patch/remove-patch: Lock staging directory during operation [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1068 (https://phabricator.wikimedia.org/T414628) [16:14:56] (03open) 10dancy: test_patches.py: Factor out common patterns [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1071 [16:15:00] (03update) 10dancy: test_patches.py: Factor out common patterns [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1071 [16:17:44] (03merge) 10dancy: test_patches.py: Factor out common patterns [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1071 [16:18:43] (03open) 10dancy: Release 4.234.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1072 [16:20:09] (03merge) 10dancy: Release 4.234.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1072 [16:28:49] 06Release-Engineering-Team, 10Scap, 07Essential-Work, 13Patch-For-Review: scap patch management commands should lock the staging directory - https://phabricator.wikimedia.org/T414628#11542512 (10dancy) 05In progress→03Resolved Change deployed via scap 4.234.0 [16:47:50] 06Release-Engineering-Team, 10Scap, 06ServiceOps new: scap --unlock-all asks for confirmation even with --bg flag - https://phabricator.wikimedia.org/T415062#11542559 (10dancy) 05Open→03Resolved p:05Triage→03Medium [17:04:49] (03PS1) 10Hashar: Zuul: fix disabling recursion on requires-only jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1229611 (https://phabricator.wikimedia.org/T407797) [17:04:51] (03PS1) 10Hashar: jjb: cleanup requires-only jobs definitions [integration/config] - 10https://gerrit.wikimedia.org/r/1229612 (https://phabricator.wikimedia.org/T407797) [17:11:30] (03PS2) 10Hashar: jjb: cleanup requires-only jobs definitions [integration/config] - 10https://gerrit.wikimedia.org/r/1229612 (https://phabricator.wikimedia.org/T407797) [17:11:30] (03PS1) 10Hashar: jjb: requires-only jobs for voting/non-voting [integration/config] - 10https://gerrit.wikimedia.org/r/1229615 (https://phabricator.wikimedia.org/T407797) [17:16:28] (03PS2) 10Hashar: jjb: requires-only jobs for voting/non-voting [integration/config] - 10https://gerrit.wikimedia.org/r/1229615 (https://phabricator.wikimedia.org/T407797) [17:16:48] (03CR) 10Hashar: [C:03+2] jjb: requires-only jobs for voting/non-voting [integration/config] - 10https://gerrit.wikimedia.org/r/1229615 (https://phabricator.wikimedia.org/T407797) (owner: 10Hashar) [17:16:51] (03CR) 10Hashar: [C:03+2] jjb: cleanup requires-only jobs definitions [integration/config] - 10https://gerrit.wikimedia.org/r/1229612 (https://phabricator.wikimedia.org/T407797) (owner: 10Hashar) [17:16:54] (03CR) 10Hashar: [C:03+2] Zuul: fix disabling recursion on requires-only jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1229611 (https://phabricator.wikimedia.org/T407797) (owner: 10Hashar) [17:18:24] (03Merged) 10jenkins-bot: Zuul: fix disabling recursion on requires-only jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1229611 (https://phabricator.wikimedia.org/T407797) (owner: 10Hashar) [17:19:19] (03Merged) 10jenkins-bot: jjb: cleanup requires-only jobs definitions [integration/config] - 10https://gerrit.wikimedia.org/r/1229612 (https://phabricator.wikimedia.org/T407797) (owner: 10Hashar) [17:19:21] (03Merged) 10jenkins-bot: jjb: requires-only jobs for voting/non-voting [integration/config] - 10https://gerrit.wikimedia.org/r/1229615 (https://phabricator.wikimedia.org/T407797) (owner: 10Hashar) [17:52:11] 06Release-Engineering-Team (Priority Backlog 📥), 07Essential-Work, 05Release, 05Train Deployments: 1.46.0-wmf.12 deployment blockers - https://phabricator.wikimedia.org/T413803#11542761 (10Scott_French) Following up from T415169, I suspect video transcodes are no longer completing due to this error. Can we... [17:53:52] hi folks o/ I think we need to roll wmf.12 back to group 0 ^^ to get commons back to wmf.11. looks like this week's conductors are andre and jeena? [17:59:25] 06Release-Engineering-Team (Priority Backlog 📥), 07Essential-Work, 05Release, 05Train Deployments: 1.46.0-wmf.12 deployment blockers - https://phabricator.wikimedia.org/T413803#11542773 (10thcipriani) [17:59:41] hi swfrench-wmf yes, we can rollback, jeena or andre around for rollback? [18:01:41] thcipriani: thanks! I can also run it if it's as simple as `scap train` -> select group0 [18:02:09] swfrench-wmf: it is that simple, yes [18:02:20] swfrench-wmf, thcipriani: yes, I'm around, can do [18:02:21] Just wondering if I may collide with may collide with any potential SRE k8s deployment plans in the next hour per https://wikitech.wikimedia.org/wiki/Deployments#Wednesday,_January_21 [18:02:30] let me ask in #wikimedia-operations [18:02:37] thanks andre <3 [18:02:39] I'll answer in operations :) [18:02:43] thanks, andre! [18:06:06] 06Release-Engineering-Team (Priority Backlog 📥), 07Essential-Work, 13Patch-For-Review, 05Release, 05Train Deployments: 1.46.0-wmf.12 deployment blockers - https://phabricator.wikimedia.org/T413803#11542789 (10Aklapper) Due to T415169 I am rolling back wmf.12 to group0 right now, so group1 will be back on... [18:11:17] Rollback finished [18:16:18] (03PS1) 10Hashar: utils: zuul-mw-jobs-runner: add branches support [integration/config] - 10https://gerrit.wikimedia.org/r/1229626 [18:43:11] (03open) 10dancy: blubber.yaml: Run pvc cleaner under tini [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/38 [18:43:13] (03update) 10dancy: blubber.yaml: Run pvc cleaner under tini [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/38 [18:44:48] (03merge) 10dancy: blubber.yaml: Run pvc cleaner under tini [repos/releng/k8s-pvc-cleaner] - 10https://gitlab.wikimedia.org/repos/releng/k8s-pvc-cleaner/-/merge_requests/38 [18:58:12] (03open) 10dancy: k8s-pvc-cleaner-values.yaml.tftpl: Use image v1.0.9 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/548 [18:58:14] (03update) 10dancy: k8s-pvc-cleaner-values.yaml.tftpl: Use image v1.0.9 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/548 [19:02:11] (03merge) 10dancy: k8s-pvc-cleaner-values.yaml.tftpl: Use image v1.0.9 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/548 [19:23:31] (03open) 10dancy: prod.tfvars: Increase prometheus_storage_size to 100Gi [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/549 [19:23:34] (03update) 10dancy: prod.tfvars: Increase prometheus_storage_size to 100Gi [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/549 [19:24:41] (03merge) 10dancy: prod.tfvars: Increase prometheus_storage_size to 100Gi [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/549 [19:35:02] maintenance-disconnect-full-disks build 774206 integration-agent-docker-1056 (/: 36%, /srv: 100%, /var/lib/docker: 37%): OFFLINE due to disk space [19:45:02] maintenance-disconnect-full-disks build 774208 integration-agent-docker-1056 (/: 36%, /srv: 65%, /var/lib/docker: 37%): RECOVERY disk space OK [21:27:40] 06Release-Engineering-Team, 10Scap, 10MW-on-K8s, 06ServiceOps new, 10ServiceOps-Mediawiki: helmfile/scap does not reliably bootstrap mediawiki - https://phabricator.wikimedia.org/T397685#11543405 (10Scott_French) p:05Triage→03Medium a:03Scott_French Revisiting this, I believe we understand what hap... [22:25:50] (03PS1) 10Daimona Eaytoy: Always use `composer phpunit` for PHPUnit tests [integration/quibble] - 10https://gerrit.wikimedia.org/r/1229739 (https://phabricator.wikimedia.org/T395470) [22:38:05] (03PS1) 10Daimona Eaytoy: dockerfiles: Drop legacy PHPUnit entrypoint from quibble-coverage [integration/config] - 10https://gerrit.wikimedia.org/r/1229756 (https://phabricator.wikimedia.org/T395470) [22:40:17] (03CR) 10Daimona Eaytoy: "Note: untested!" [integration/config] - 10https://gerrit.wikimedia.org/r/1229756 (https://phabricator.wikimedia.org/T395470) (owner: 10Daimona Eaytoy) [22:42:02] 10Continuous-Integration-Config, 10MediaWiki-Core-Tests, 13Patch-For-Review, 07Technical-Debt: Drop deprecated phpunit.php and suite.xml - https://phabricator.wikimedia.org/T395470#11543539 (10Daimona) >>! In T395470#11523366, @Jdforrester-WMF wrote: >>>! In T395470#11493671, @Daimona wrote: >>>>! In T3954...