[01:37:43] 10Phabricator, 07PM: Reset stale good first tasks - https://phabricator.wikimedia.org/T407716 (10Pppery) 03NEW [07:44:55] 10Phabricator, 07PM: Reset stale good first tasks - https://phabricator.wikimedia.org/T407716#11288522 (10Urbanecm) I'm not sure about this. On one hand, I absolutely agree we should care of the good first tasks to make sure they're good. On the other hand, I'm unsure if time is a good indicator. From my exper... [09:05:29] (03CR) 10Hashar: [C:03+2] inference-services: Remove the outlink-transformer CI job. [integration/config] - 10https://gerrit.wikimedia.org/r/1187752 (https://phabricator.wikimedia.org/T404294) (owner: 10Bartosz Wójtowicz) [09:07:04] (03Merged) 10jenkins-bot: inference-services: Remove the outlink-transformer CI job. [integration/config] - 10https://gerrit.wikimedia.org/r/1187752 (https://phabricator.wikimedia.org/T404294) (owner: 10Bartosz Wójtowicz) [09:37:01] it would be nice if Zuul let you cancel / unqueue jobs before starting them 🤔 [09:37:37] like, in https://integration.wikimedia.org/zuul/#q=1196931, I’m only going to need the selenium and node changes for now (it’s a WIP change), but because Zuul is so full, it’ll be some time before it even starts the PHP jobs that I would then immediately abort [09:37:47] propose it in the upstream zuul task tracker? [09:38:28] * Lucas_WMDE looks [09:38:41] is that going to be useful if, according to https://www.mediawiki.org/wiki/Continuous_integration/Zuul#References, we’re using an older major version? [09:39:32] you would probably need to cross check with the new version, i think there is some work bringing that on line [10:18:18] 10GitLab (CI & Job Runners), 06collaboration-services, 10Toolforge, 06cloud-services-team (FY2025/26-Q1), 13Patch-For-Review: tofu-provisioning: Failed to install provider - https://phabricator.wikimedia.org/T405742#11288948 (10fnegri) The last patch I merged today is working as expected, but the `gitlab... [11:06:19] 10GitLab (CI & Job Runners), 06collaboration-services, 10Toolforge, 06cloud-services-team (FY2025/26-Q1): tofu-provisioning: Failed to install provider - https://phabricator.wikimedia.org/T405742#11289047 (10fnegri) 05In progress→03Resolved All `runner-XXXX` hosts in the `gitlab-runners` project ha... [11:10:16] `13:03:03 npm warn tar TAR_ENTRY_ERROR ENOSPC: no space left on device, write` - see this on integration-agent-docker-1040 (https://integration.wikimedia.org/ci/job/quibble-with-gated-extensions-selenium-php81/6520/consoleFull) [11:30:42] 10Phabricator: Consider excluding tasks with 'Security Issue' subtype from H412 that changes the subtype of tasks to 'Production Error' - https://phabricator.wikimedia.org/T407689#11289121 (10Aklapper) [11:58:17] 10Phabricator (Upstream), 07Regression, 07Upstream: No notification is sent to a task when a new subtask of it is created & that new subtask's status is "Open" - https://phabricator.wikimedia.org/T405084#11289162 (10Aklapper) @A_smart_kitten: Thanks for digging into this, very helpful! This is an upstream is... [12:13:05] 10Phabricator (Upstream), 07Regression, 07Upstream: No notification is sent to a task when a new subtask of it is created & that new subtask's status is "Open" - https://phabricator.wikimedia.org/T405084#11289175 (10Aklapper) p:05Triage→03Low a:03Aklapper [12:37:48] 10Phabricator, 07PM: Reset stale good first tasks - https://phabricator.wikimedia.org/T407716#11289214 (10Aklapper) I support the idea behind but we end up in the usual bikesheddy question how to define or detect "stale" - see {T248034}. [12:44:14] (03PS1) 10Hashar: Zuul: [mediawiki/extensions/JsonConfig] add WikimediaMessages [integration/config] - 10https://gerrit.wikimedia.org/r/1197256 (https://phabricator.wikimedia.org/T389998) [12:46:42] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/extensions/JsonConfig] add WikimediaMessages [integration/config] - 10https://gerrit.wikimedia.org/r/1197256 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [12:49:10] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/JsonConfig] add WikimediaMessages [integration/config] - 10https://gerrit.wikimedia.org/r/1197256 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [13:03:16] (03CR) 10Hashar: [C:03+2] "Sorry I should have given more context before adding you as reviewer! The context is to have CI to stop injecting dependencies recursively" [integration/config] - 10https://gerrit.wikimedia.org/r/1195368 (owner: 10Hashar) [13:04:51] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/GrowthExperiments] add EventBus, EventStreamConfig [integration/config] - 10https://gerrit.wikimedia.org/r/1195368 (owner: 10Hashar) [13:35:25] (03CR) 10Tacsipacsi: "This is actually a bug: it means that Kartographer doesn’t work properly without WikimediaMessages, making it unfit for third-party use. h" [integration/config] - 10https://gerrit.wikimedia.org/r/1197256 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [13:45:21] maintenance-disconnect-full-disks build 747351 integration-agent-docker-1043 (/: 35%, /srv: 100%, /var/lib/docker: 24%): OFFLINE due to disk space [13:48:13] (03PS3) 10Bartosz Dziewoński: Add description [pywikibot] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197263 [13:48:15] (03PS4) 10Bartosz Dziewoński: Improve description [mediawiki] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197264 [13:48:17] (03PS3) 10Bartosz Dziewoński: Add description [pywikibot/core] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197268 [13:48:29] (03PS3) 10Bartosz Dziewoński: Add description [extensions] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197266 [13:48:33] (03PS3) 10Bartosz Dziewoński: Add description [skins] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197267 [13:49:55] (03CR) 10Bartosz Dziewoński: "This affects the description shown at https://gerrit.wikimedia.org/r/admin/repos/q/filter:pywikibot." [pywikibot/core] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197268 (owner: 10Bartosz Dziewoński) [13:50:00] (03CR) 10Bartosz Dziewoński: "This affects the description shown at https://gerrit.wikimedia.org/r/admin/repos/q/filter:pywikibot." [pywikibot] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197263 (owner: 10Bartosz Dziewoński) [13:50:07] maintenance-disconnect-full-disks build 747353 integration-agent-docker-1043 (/: 35%, /srv: 89%, /var/lib/docker: 24%): RECOVERY disk space OK [13:53:16] (03CR) 10Hashar: [C:03+2] "**You are correct**, `modules/box/Map.js` is the place relying on the message, I guess when WikimediaMessages is missing, the message key " [integration/config] - 10https://gerrit.wikimedia.org/r/1197256 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [13:54:32] (03CR) 10Hashar: [C:03+2] Add description [extensions] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197266 (owner: 10Bartosz Dziewoński) [13:54:34] (03CR) 10Xqt: [C:03+2] Add description [pywikibot/core] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197268 (owner: 10Bartosz Dziewoński) [13:54:35] (03CR) 10Hashar: [C:03+2] Add description [skins] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197267 (owner: 10Bartosz Dziewoński) [13:54:36] (03CR) 10Hashar: [C:03+2] Improve description [mediawiki] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197264 (owner: 10Bartosz Dziewoński) [13:54:40] (03CR) 10Hashar: [V:03+2 C:03+2] Improve description [mediawiki] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197264 (owner: 10Bartosz Dziewoński) [13:54:40] (03CR) 10Hashar: [V:03+2 C:03+2] Add description [skins] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197267 (owner: 10Bartosz Dziewoński) [13:54:43] (03CR) 10Hashar: [V:03+2 C:03+2] Add description [extensions] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197266 (owner: 10Bartosz Dziewoński) [13:54:52] (03CR) 10Xqt: [C:03+2] Add description [pywikibot] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197263 (owner: 10Bartosz Dziewoński) [13:55:49] (03CR) 10Hashar: [V:03+2] Add description [pywikibot/core] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197268 (owner: 10Bartosz Dziewoński) [13:55:51] (03CR) 10Hashar: [V:03+2] Add description [pywikibot] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197263 (owner: 10Bartosz Dziewoński) [13:56:48] (03CR) 10Hashar: [V:03+2] "I went ahead since @info@gno.de is the main contributor to pywikibot and approved the description." [pywikibot] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197263 (owner: 10Bartosz Dziewoński) [13:56:52] (03CR) 10Hashar: [V:03+2] "I went ahead since @info@gno.de is the main contributor to pywikibot and approved the description." [pywikibot/core] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1197268 (owner: 10Bartosz Dziewoński) [14:05:03] maintenance-disconnect-full-disks build 747356 integration-agent-docker-1043 (/: 35%, /srv: 97%, /var/lib/docker: 24%): OFFLINE due to disk space [14:08:23] !log Cleaned up integration-agent-docker-1043 , there was some old /srv/jenkins/workspace/back dir [14:08:24] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:10:03] maintenance-disconnect-full-disks build 747357 integration-agent-docker-1043 (/: 35%, /srv: 84%, /var/lib/docker: 24%): RECOVERY disk space OK [15:18:30] 10Scap (SpiderPig 🕸️): scap crash in SpiderPig job #775 (change was edited after creating job): TypeError: prompt_for_approval_or_exit() missing 1 required positional argument: 'exit_message' - https://phabricator.wikimedia.org/T407767 (10Lucas_Werkmeister_WMDE) 03NEW [15:19:31] 10Scap (SpiderPig 🕸️): scap crash in SpiderPig job #775 (change was edited after creating job): TypeError: prompt_for_approval_or_exit() missing 1 required positional argument: 'exit_message' - https://phabricator.wikimedia.org/T407767#11290200 (10Lucas_Werkmeister_WMDE) [15:33:46] (03open) 10dancy: backport.py: Fix _prompt_for_approval_or_exit call [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1015 (https://phabricator.wikimedia.org/T407767) [15:33:49] (03update) 10dancy: backport.py: Fix _prompt_for_approval_or_exit call [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1015 (https://phabricator.wikimedia.org/T407767) [15:36:15] (03merge) 10dancy: backport.py: Fix _prompt_for_approval_or_exit call [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1015 (https://phabricator.wikimedia.org/T407767) [15:38:17] (03open) 10dancy: Release 4.215.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1016 [15:39:24] (03merge) 10dancy: Release 4.215.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1016 [15:49:02] 10Scap (SpiderPig 🕸️), 13Patch-For-Review: scap crash in SpiderPig job #775 (change was edited after creating job): TypeError: prompt_for_approval_or_exit() missing 1 required positional argument: 'exit_message' - https://phabricator.wikimedia.org/T407767#11290364 (10dancy) 05Open→03Resolved p:05Triag... [15:57:29] !log Unblock 85.134.224.0/21 (T407752) [15:57:30] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:58:02] Project beta-scap-sync-world build #228884: 04FAILURE in 45 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/228884/ [15:58:34] 16:58:02 15:58:02 sudo: PAM account management error: Authentication service cannot retrieve authentication info [15:58:34] 16:58:02 sudo: a password is required [16:00:09] huh [16:01:00] that sounds like ldap taking an unauthorized break. [16:01:41] A followup run is working [16:01:57] I wonder if we still have the timer to restart ldap once a day or so because of an old problem with memory leaking... [16:03:10] Yippee, build fixed! [16:03:10] Project beta-scap-sync-world build #228885: 09FIXED in 2 min 28 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/228885/ [16:06:09] wow. I thought I remembered it being once a day, but it looks like is't once an hour. https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/production/modules/profile/manifests/openldap/restarts.pp T130593 [16:06:10] T130593: investigate slapd memory leak - https://phabricator.wikimedia.org/T130593 [16:20:29] That LDAP restart timer isn't as bad as it appears at first glance. The timer runs a script that uses `ps -C slapd -o pmem=` to check memory utilization and only restarts the slapd process if >50% of ram is being used. [16:28:06] ooh, neat [16:41:06] 10Scap (SpiderPig 🕸️): scap crash in SpiderPig job #775 (change was edited after creating job): TypeError: prompt_for_approval_or_exit() missing 1 required positional argument: 'exit_message' - https://phabricator.wikimedia.org/T407767#11290603 (10Lucas_Werkmeister_WMDE) Cool, thanks for the quick response! [16:41:12] 10Scap (SpiderPig 🕸️): scap crash in SpiderPig job #775 (change was edited after creating job): TypeError: prompt_for_approval_or_exit() missing 1 required positional argument: 'exit_message' - https://phabricator.wikimedia.org/T407767#11290604 (10Lucas_Werkmeister_WMDE) [17:06:33] 10Continuous-Integration-Config, 05PES1.3.3 WP25 Easter Eggs: Setup continuous integration on the WP25EasterEggs extension - https://phabricator.wikimedia.org/T407682#11290659 (10Jdlrobson-WMF) 05Open→03Resolved a:03Jdlrobson-WMF Thanks @reedy ! [17:14:18] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services, 13Patch-For-Review: puppetize setup of new zuul VMs - https://phabricator.wikimedia.org/T395938#11290680 (10Dzahn) [18:06:24] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services, 13Patch-For-Review: puppetize setup of new zuul VMs - https://phabricator.wikimedia.org/T395938#11290888 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by dzahn@cumin2002 for host zuul1001.eqiad.wmnet w... [18:26:48] (03update) 10dduvall: image: Dynamically enforce published image repo [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/154 (https://phabricator.wikimedia.org/T407294) [18:26:50] (03update) 10dduvall: image: Dynamically enforce published image repo [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/154 (https://phabricator.wikimedia.org/T407294) [18:26:52] (03update) 10dduvall: image: Dynamically enforce published image repo [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/154 (https://phabricator.wikimedia.org/T407294) [18:28:52] (03update) 10dduvall: image: Dynamically enforce published image repo [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/154 (https://phabricator.wikimedia.org/T407294) [19:38:48] (03PS1) 10Hashar: Zuul: [mediawiki/extensions/Scribunto] drop SyntaxHighlight_GeSHi [integration/config] - 10https://gerrit.wikimedia.org/r/1197325 (https://phabricator.wikimedia.org/T389998) [19:42:17] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/extensions/Scribunto] drop SyntaxHighlight_GeSHi [integration/config] - 10https://gerrit.wikimedia.org/r/1197325 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [19:44:12] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/Scribunto] drop SyntaxHighlight_GeSHi [integration/config] - 10https://gerrit.wikimedia.org/r/1197325 (https://phabricator.wikimedia.org/T389998) (owner: 10Hashar) [19:47:56] (03update) 10dancy: image: Dynamically enforce published image repo [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/154 (https://phabricator.wikimedia.org/T407294) (owner: 10dduvall) [19:47:56] (03approved) 10dancy: image: Dynamically enforce published image repo [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/154 (https://phabricator.wikimedia.org/T407294) (owner: 10dduvall) [20:23:02] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services, 13Patch-For-Review: puppetize setup of new zuul VMs - https://phabricator.wikimedia.org/T395938#11291405 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by dzahn@cumin2002 for host zuul1001.eqiad.wmnet with... [21:04:08] 10Continuous-Integration-Config: Create a CI job to enforce tests to pass with solely required extensions - https://phabricator.wikimedia.org/T407797 (10hashar) 03NEW [21:08:35] (03PS1) 10Hashar: Job running tests with solely required extensions [integration/config] - 10https://gerrit.wikimedia.org/r/1197340 (https://phabricator.wikimedia.org/T407797) [21:29:52] (03update) 10dduvall: image: Dynamically enforce published image repo [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/154 (https://phabricator.wikimedia.org/T407294) [21:29:53] (03update) 10dduvall: image: Dynamically enforce published image repo [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/154 (https://phabricator.wikimedia.org/T407294) [21:29:56] (03update) 10dduvall: image: Dynamically enforce published image repo [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/154 (https://phabricator.wikimedia.org/T407294) [21:32:25] (03merge) 10dduvall: image: Dynamically enforce published image repo [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/154 (https://phabricator.wikimedia.org/T407294) [21:58:14] 06Release-Engineering-Team, 10MediaWiki-REST-API, 05MW-1.45-release, 06MW-Interfaces-Team (MWI-Sprint-21 (2025-10-21 to 2025-11-04)): Remove SwaggerUI from MW-1.45 release - https://phabricator.wikimedia.org/T397473#11291705 (10HCoplin-WMF) [21:58:15] 06Release-Engineering-Team, 10MediaWiki-REST-API, 05MW-1.45-release, 06MW-Interfaces-Team (MWI-Sprint-21 (2025-10-21 to 2025-11-04)): Remove SwaggerUI from MW-1.45 release - https://phabricator.wikimedia.org/T397473#11291706 (10HCoplin-WMF) p:05Triage→03High [22:11:37] (03update) 10dduvall: variables: Remove JWT claim related variables [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/155 (https://phabricator.wikimedia.org/T407294) [22:11:41] (03update) 10dduvall: variables: Remove JWT claim related variables [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/155 (https://phabricator.wikimedia.org/T407294) [22:11:43] (03update) 10dduvall: variables: Remove JWT claim related variables [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/155 (https://phabricator.wikimedia.org/T407294) [22:11:49] (03open) 10dduvall: variables: Remove JWT claim related variables [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/155 (https://phabricator.wikimedia.org/T407294) [23:03:01] (03approved) 10dancy: variables: Remove JWT claim related variables [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/155 (https://phabricator.wikimedia.org/T407294) (owner: 10dduvall) [23:17:34] 10Continuous-Integration-Config, 05FY2025-26 WE3.3 Engaging core audiences, 06Reader Experience Team (REx Sprint 7 [Q2 Oct 7-Oct 21 '25]), 07Unplanned-Sprint-Work: Add BetaFeatures and MetricsPlatform as CI dependencies for ReadingLists - https://phabricator.wikimedia.org/T407249#11291950 (10HFan-WMF)... [23:25:40] (03merge) 10dduvall: variables: Remove JWT claim related variables [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/155 (https://phabricator.wikimedia.org/T407294) [23:31:17] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services, 13Patch-For-Review: puppetize setup of new zuul VMs - https://phabricator.wikimedia.org/T395938#11292002 (10Dzahn) [23:34:51] (03open) 10dduvall: version: 2.11.0 [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/156 [23:36:42] (03merge) 10dduvall: version: 2.11.0 [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/156 [23:39:37] (03PS2) 10Felds: Adding myself to allow-list per RhinosF1's suggestion [integration/config] (zuul3) - 10https://gerrit.wikimedia.org/r/1197362 [23:42:13] (03CR) 10Felds: [C:03+1] Adding myself to allow-list per RhinosF1's suggestion [integration/config] (zuul3) - 10https://gerrit.wikimedia.org/r/1197362 (owner: 10Felds)