[02:06:47] 10Tool-wsindex, 10Wikisource Reader App, 10Outreachy (Round 31): Outreachy 31: Improve the Wikisource Reader App - https://phabricator.wikimedia.org/T405593#11510808 (10Muguro) **Weekly Internship Report** //Week 5: January 5 - January 9// Blog Week 5: [[ https://levymuguro.hashnode.dev/think-about-your-au... [05:31:28] RESOLVED: NfsAlmostFull: The NFS drive is over 85% capacity (currently 85.12%) at host tools-nfs-3 in project tools - https://prometheus-alerts.wmcloud.org/?q=alertname%3DNfsAlmostFull [08:43:44] 10VPS-project-Codesearch: Codesearch down/unreachable (2026-01-11) - https://phabricator.wikimedia.org/T414290#11511128 (10Ladsgroup) https://codesearch.wmcloud.org/_health/ says "only" search is down. Unlike previous times, I can ssh into the VM, the inode and storage are also healthy: ` ladsgroup@codesearch9:~... [09:03:42] 10VPS-project-Codesearch: Codesearch down/unreachable (2026-01-11) - https://phabricator.wikimedia.org/T414290#11511157 (10Ladsgroup) Nothing in the logs of the systemd service. A restart of the service made everything back to "okay". [09:04:04] 10VPS-project-Codesearch: Codesearch down/unreachable (2026-01-11) - https://phabricator.wikimedia.org/T414290#11511158 (10Dzahn) Right now it works normal for me. [09:13:43] 10VPS-project-Wikistats: Add kaiwiki to wikistats - https://phabricator.wikimedia.org/T414242#11511169 (10Dzahn) [09:13:55] 10VPS-project-Wikistats: Add kaiwiki to wikistats - https://phabricator.wikimedia.org/T414242#11511172 (10Dzahn) 05Open→03Stalled [09:19:50] 10VPS-project-Codesearch: Codesearch down/unreachable (2026-01-11) - https://phabricator.wikimedia.org/T414290#11511181 (10A_smart_kitten) >>! In T414290#11511128, @Ladsgroup wrote: > https://codesearch.wmcloud.org/_health/ says "only" /search endpoint is down. FWIW I can also confirm that everything was down... [09:27:28] 10VPS-project-Codesearch: Codesearch down/unreachable (2026-01-11) - https://phabricator.wikimedia.org/T414290#11511204 (10Dzahn) 05Open→03Resolved a:03Dzahn [09:27:34] 10VPS-project-Codesearch: Codesearch down/unreachable (2026-01-11) - https://phabricator.wikimedia.org/T414290#11511206 (10Dzahn) a:05Dzahn→03None [09:40:30] 06cloud-services-team (FY2025/2026-Q3-Q4), 10Cloud-VPS: [tofu-infra] "tofu plan" failing in codfw - https://phabricator.wikimedia.org/T410265#11511306 (10fnegri) 05Open→03In progress [09:41:30] 06cloud-services-team (FY2025/2026-Q3-Q4), 10Cloud-VPS: [tofu-infra] "tofu plan" failing in codfw - https://phabricator.wikimedia.org/T410265#11511312 (10fnegri) a:03Andrew @Andrew assigning to you as you're working on it :) [11:06:22] 06cloud-services-team, 10Toolforge: lima-kilo: INJECT_FACTS_AS_VARS default to `True` is deprecated - https://phabricator.wikimedia.org/T413782#11511752 (10Volans) p:05Triage→03Medium [11:06:35] 06cloud-services-team, 10Toolforge: lima-kilo: Migrate away from overlay2 storage - https://phabricator.wikimedia.org/T413793#11511753 (10Volans) p:05Triage→03Medium [11:06:48] 06cloud-services-team, 10Toolforge, 06tools-platform-team: New upstream release for Pywikibot - https://phabricator.wikimedia.org/T410948#11511754 (10Volans) p:05Triage→03Medium [11:08:20] 06cloud-services-team, 10PAWS: Add end-to-end monitoring for PAWS - https://phabricator.wikimedia.org/T414106#11511758 (10Volans) p:05Triage→03Medium [11:11:04] 06cloud-services-team, 10Toolforge, 13Patch-For-Review: jobs-api does not properly handle quota errors when restarting a job - https://phabricator.wikimedia.org/T414229#11511766 (10Volans) p:05Triage→03Medium [11:14:11] 06cloud-services-team, 10Data-Services, 06Data-Engineering, 06Data-Persistence: Add datetime versions of timestamp fields to Wikireplica databases - https://phabricator.wikimedia.org/T414199#11511794 (10Volans) p:05Triage→03Medium Adding #data-persistence and #data-engineering for visibility and feasib... [11:14:41] 06cloud-services-team, 10Horizon: No new changesets added to cloud/instance-puppet.git since df63d60 - https://phabricator.wikimedia.org/T414249#11511797 (10Volans) p:05Triage→03Medium [11:17:00] 06cloud-services-team, 10Data-Services: [wikireplicas] Create views for new wiki kaiwiki - https://phabricator.wikimedia.org/T414240#11511808 (10Volans) p:05Triage→03Medium [11:21:48] 06cloud-services-team, 10Data-Services, 06Data-Engineering, 06Data-Persistence: Add datetime versions of timestamp fields to Wikireplica databases - https://phabricator.wikimedia.org/T414199#11511823 (10Marostegui) I believe at the moment the only schema differences are indexes but for columns these would... [11:22:02] 06cloud-services-team, 10Horizon: No new changesets added to cloud/instance-puppet.git since df63d60 - https://phabricator.wikimedia.org/T414249#11511828 (10taavi) 05Open→03Resolved Someone appears to have fixed this by restarting the worker service on Friday evening. [11:23:55] 06cloud-services-team, 10Cloud-VPS: No new changesets added to cloud/instance-puppet.git since df63d60 - https://phabricator.wikimedia.org/T414249#11511835 (10taavi) [12:25:29] (03open) 10l10n-bot: Localisation updates from https://translatewiki.net. [toolforge-repos/ranker] - 10https://gitlab.wikimedia.org/toolforge-repos/ranker/-/merge_requests/32 [12:27:07] (03CR) 10CI reject: [V:04-1] Localisation updates from https://translatewiki.net. [labs/tools/commons-mass-description] - 10https://gerrit.wikimedia.org/r/1225545 (owner: 10L10n-bot) [12:27:08] (03CR) 10CI reject: [V:04-1] Localisation updates from https://translatewiki.net. [labs/tools/Isa] - 10https://gerrit.wikimedia.org/r/1225546 (owner: 10L10n-bot) [13:19:04] (03CR) 10Thiemo Kreuz (WMDE): Fix typo on user setting route (031 comment) [labs/tools/Isa] - 10https://gerrit.wikimedia.org/r/1012648 (https://phabricator.wikimedia.org/T343438) (owner: 10Josefanthony) [13:22:13] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.tofu running tofu plan for main branch [13:22:45] !log andrew@cloudcumin1001 admin END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for main branch [13:30:47] (03open) 10nurahwakili: Resolve T412023 "Oauth implementation" [toolforge-repos/paulina] (OAuth-implementation) - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/175 [13:40:33] 10VPS-project-Codesearch, 06Abstract Wikipedia team: toolforge-repos/abstract-wiki-prototype should not be indexed in Codesearch - https://phabricator.wikimedia.org/T413322#11512252 (10Jdforrester-WMF) 05Open→03Resolved a:03Ammarpad This has been deployed, e.g. https://codesearch.wmcloud.org/search/?... [13:46:04] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/toolforge-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-cli/-/merge_requests/59 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:46:08] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/toolforge-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-cli/-/merge_requests/59 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:46:15] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/toolforge-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-cli/-/merge_requests/59 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:46:21] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/maintain-harbor] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/maintain-harbor/-/merge_requests/59 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:46:23] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/maintain-harbor] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/maintain-harbor/-/merge_requests/59 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:46:26] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/maintain-harbor] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/maintain-harbor/-/merge_requests/59 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:46:31] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/299 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:46:35] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/299 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:46:36] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/299 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:47:00] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/builds-builder] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-builder/-/merge_requests/79 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:47:01] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/builds-builder] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-builder/-/merge_requests/79 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:47:06] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/builds-builder] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-builder/-/merge_requests/79 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:47:14] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/api-gateway] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/api-gateway/-/merge_requests/85 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:47:15] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/api-gateway] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/api-gateway/-/merge_requests/85 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:47:18] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/api-gateway] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/api-gateway/-/merge_requests/85 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:47:23] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/builds-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-api/-/merge_requests/151 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:47:24] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/builds-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-api/-/merge_requests/151 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:47:30] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/builds-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-api/-/merge_requests/151 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:48:00] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/envvars-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-api/-/merge_requests/68 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:48:02] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/envvars-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-api/-/merge_requests/68 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:48:05] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/envvars-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-api/-/merge_requests/68 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:48:14] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/volume-admission] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/volume-admission/-/merge_requests/42 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:48:16] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/volume-admission] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/volume-admission/-/merge_requests/42 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:48:22] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/volume-admission] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/volume-admission/-/merge_requests/42 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:48:23] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/webservice-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/webservice-cli/-/merge_requests/93 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:48:24] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/webservice-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/webservice-cli/-/merge_requests/93 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:48:33] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/webservice-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/webservice-cli/-/merge_requests/93 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:48:41] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/envvars-admission] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-admission/-/merge_requests/29 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:48:45] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/envvars-admission] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-admission/-/merge_requests/29 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:48:49] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/envvars-admission] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-admission/-/merge_requests/29 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:48:53] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/envvars-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-cli/-/merge_requests/102 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:48:57] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/envvars-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-cli/-/merge_requests/102 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:49:06] (03update) 10raymond-ndibe: build: Upgrade Poetry dependencies [repos/cloud/toolforge/jobs-emailer] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-emailer/-/merge_requests/37 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:49:10] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/envvars-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-cli/-/merge_requests/102 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:49:14] (03approved) 10raymond-ndibe: build: Upgrade Poetry dependencies [repos/cloud/toolforge/jobs-emailer] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-emailer/-/merge_requests/37 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:49:18] (03merge) 10raymond-ndibe: build: Upgrade Poetry dependencies [repos/cloud/toolforge/jobs-emailer] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-emailer/-/merge_requests/37 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:49:22] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/256 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:49:26] (03open) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: builds-builder: bump to 0.0.139-20260112134719-8aef69fd [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1096 [13:49:30] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/256 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:49:34] (03update) 10raymond-ndibe: build: Upgrade Poetry dependencies [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/151 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:49:38] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: builds-builder: bump to 0.0.139-20260112134719-8aef69fd [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1096 [13:49:42] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/builds-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-cli/-/merge_requests/126 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:49:46] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/256 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:49:50] (03approved) 10raymond-ndibe: build: Upgrade Poetry dependencies [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/151 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:49:58] (03merge) 10raymond-ndibe: build: Upgrade Poetry dependencies [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/151 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:50:02] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/builds-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-cli/-/merge_requests/126 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:50:06] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/jobs-emailer] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-emailer/-/merge_requests/36 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:50:11] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/builds-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-cli/-/merge_requests/126 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:50:15] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/299 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:50:27] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/jobs-emailer] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-emailer/-/merge_requests/36 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:50:49] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/jobs-emailer] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-emailer/-/merge_requests/36 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:50:53] (03open) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: maintain-harbor: bump to 0.0.66-20260112134638-71a01eba [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1097 [13:50:57] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: maintain-harbor: bump to 0.0.66-20260112134638-71a01eba [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1097 [13:51:05] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/ingress-admission] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/ingress-admission/-/merge_requests/33 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:51:09] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/ingress-admission] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/ingress-admission/-/merge_requests/33 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:51:13] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/ingress-admission] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/ingress-admission/-/merge_requests/33 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:51:17] (03update) 10raymond-ndibe: build: Upgrade Poetry dependencies [repos/cloud/toolforge/components-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-cli/-/merge_requests/70 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:51:21] (03approved) 10raymond-ndibe: build: Upgrade Poetry dependencies [repos/cloud/toolforge/components-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-cli/-/merge_requests/70 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:51:29] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/registry-admission] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/registry-admission/-/merge_requests/34 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:51:33] (03merge) 10raymond-ndibe: build: Upgrade Poetry dependencies [repos/cloud/toolforge/components-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-cli/-/merge_requests/70 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:51:38] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/registry-admission] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/registry-admission/-/merge_requests/34 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:51:42] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/maintain-kubeusers] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/maintain-kubeusers/-/merge_requests/81 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:51:46] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/registry-admission] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/registry-admission/-/merge_requests/34 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:51:50] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/maintain-kubeusers] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/maintain-kubeusers/-/merge_requests/81 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:51:54] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/jobs-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/merge_requests/141 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:51:58] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/maintain-kubeusers] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/maintain-kubeusers/-/merge_requests/81 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:52:04] (03open) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: api-gateway: bump to 0.0.86-20260112134731-74ea55a5 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1098 [13:52:08] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/jobs-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/merge_requests/141 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:52:12] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: api-gateway: bump to 0.0.86-20260112134731-74ea55a5 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1098 [13:52:16] (03update) 10raymond-ndibe: build: Upgrade Poetry dependencies [repos/cloud/toolforge/logs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/logs-api/-/merge_requests/9 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:52:20] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/jobs-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/merge_requests/141 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:52:24] (03approved) 10raymond-ndibe: build: Upgrade Poetry dependencies [repos/cloud/toolforge/logs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/logs-api/-/merge_requests/9 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:52:28] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/150 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:52:36] (03merge) 10raymond-ndibe: build: Upgrade Poetry dependencies [repos/cloud/toolforge/logs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/logs-api/-/merge_requests/9 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:52:40] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/150 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:52:44] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/logs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/logs-api/-/merge_requests/8 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:52:48] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/logs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/logs-api/-/merge_requests/8 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:52:52] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/components-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-cli/-/merge_requests/69 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:52:57] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/150 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:53:01] (03approved) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/components-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-cli/-/merge_requests/69 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:53:17] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/logs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/logs-api/-/merge_requests/8 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:53:21] (03update) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/components-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-cli/-/merge_requests/69 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:53:29] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/jobs-emailer] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-emailer/-/merge_requests/36 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:53:41] 06cloud-services-team, 10wikitech.wikimedia.org: Flapping wikitech-static icinga alert - https://phabricator.wikimedia.org/T409029#11512290 (10Andrew) 05Open→03Invalid wts is now running a totally different stack in a totally different place. [13:54:01] (03approved) 10pepepiton: Resolve T412023 "Oauth implementation" [toolforge-repos/paulina] (OAuth-implementation) - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/175 (owner: 10nurahwakili) [13:54:15] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: jobs-api: bump to 0.0.457-20260112135010-907ec7ed [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1099 [13:54:16] (03open) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: jobs-api: bump to 0.0.457-20260112135010-907ec7ed [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1099 [13:54:30] (03merge) 10pepepiton: Resolve T412023 "Oauth implementation" [toolforge-repos/paulina] (OAuth-implementation) - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/175 (owner: 10nurahwakili) [13:55:12] 06cloud-services-team, 10Cloud-VPS, 10Ceph: cloudceph: automate nic names in profile::cloudceph::osd::hosts - https://phabricator.wikimedia.org/T405374#11512293 (10Andrew) 05Open→03Declined [13:55:44] (03open) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: volume-admission: bump to 0.0.79-20260112134850-70a02df8 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1100 [13:55:46] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: volume-admission: bump to 0.0.79-20260112134850-70a02df8 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1100 [13:56:41] (03open) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: logs-api: bump to 0.0.9-20260112135231-32d86c4e [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1101 [13:56:59] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: jobs-emailer: bump to 0.0.73-20260112135337-a2993cc7 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1105 [13:57:01] (03open) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: maintain-kubeusers: bump to 0.0.189-20260112135303-15fc6f58 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1102 [13:57:10] (03open) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: components-api: bump to 0.0.180-20260112135251-7101fc1f [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1103 [13:57:12] (03open) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: envvars-admission: bump to 0.0.36-20260112134910-bed27ee4 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1104 [13:57:14] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/components-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-cli/-/merge_requests/69 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:57:16] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: logs-api: bump to 0.0.9-20260112135231-32d86c4e [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1101 [13:57:18] 06cloud-services-team, 10Data-Services, 06Data-Engineering, 06Data-Persistence: Add datetime versions of timestamp fields to Wikireplica databases - https://phabricator.wikimedia.org/T414199#11512297 (10Ottomata) FWIW, having non-string datetime types would be quite helpful for Data Platform usages as well... [13:57:28] (03open) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: jobs-emailer: bump to 0.0.73-20260112135337-a2993cc7 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1105 [13:57:35] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: maintain-kubeusers: bump to 0.0.189-20260112135303-15fc6f58 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1102 [13:57:42] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: components-api: bump to 0.0.180-20260112135251-7101fc1f [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1103 [13:57:44] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: envvars-admission: bump to 0.0.36-20260112134910-bed27ee4 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1104 [13:57:56] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/logs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/logs-api/-/merge_requests/8 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:57:57] 06cloud-services-team, 10Data-Services, 06Data-Engineering, 06Data-Persistence: Add datetime versions of timestamp fields to Wikireplica databases - https://phabricator.wikimedia.org/T414199#11512301 (10Ottomata) > There will be a difference between the schema of the tables on Wikireplica and on MediaWiki... [13:58:01] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: jobs-emailer: bump to 0.0.73-20260112135337-a2993cc7 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1105 [13:58:05] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: jobs-emailer: bump to 0.0.73-20260112135337-a2993cc7 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1105 [13:58:31] (03merge) 10raymond-ndibe: build: Upgrade pre-commit dependencies [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/150 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [13:58:49] (03open) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: ingress-admission: bump to 0.0.73-20260112135120-fddb9453 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1106 [13:58:53] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: ingress-admission: bump to 0.0.73-20260112135120-fddb9453 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1106 [13:59:07] (03open) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: registry-admission: bump to 0.0.70-20260112135200-937fe41c [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1107 [13:59:09] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: registry-admission: bump to 0.0.70-20260112135200-937fe41c [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1107 [13:59:51] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: envvars-api: bump to 0.0.79-20260112134846-2681ef8d [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1108 [13:59:52] (03open) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: envvars-api: bump to 0.0.79-20260112134846-2681ef8d [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1108 [14:00:24] (03open) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: builds-api: bump to 0.0.206-20260112134748-5dc9dca4 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1109 [14:00:25] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: builds-api: bump to 0.0.206-20260112134748-5dc9dca4 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1109 [14:00:54] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: logs-api: bump to 0.0.10-20260112135828-50dd5b8d [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1101 [14:01:08] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: logs-api: bump to 0.0.10-20260112135828-50dd5b8d [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1101 [14:02:14] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: components-api: bump to 0.0.181-20260112135901-15537433 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1103 [14:02:24] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: components-api: bump to 0.0.181-20260112135901-15537433 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1103 [14:19:20] 06cloud-services-team, 10Cloud-VPS (Quota-requests), 10WikiCite, 10Wikidata, 10Wikidata-Query-Service: Raise quota on wikiqlever so that an instance with 256 GB RAM and 3 x 4 TB SSD can be launched - https://phabricator.wikimedia.org/T413097#11512385 (10taavi) 05Open→03Resolved a:03Volans [14:19:29] 06cloud-services-team (FY2025/2026-Q3-Q4), 10Cloud-VPS: [tofu-infra] "tofu plan" failing in codfw - https://phabricator.wikimedia.org/T410265#11512387 (10Andrew) 05In progress→03Resolved I'm worried that there's another bug hiding behind this one but for now everything seems to be working. [14:19:45] 06cloud-services-team, 10Cloud-VPS: OpenTofu vs. radosgw in codfw1dev - https://phabricator.wikimedia.org/T394061#11512389 (10Andrew) 05Open→03Resolved Fixed, notes in T410265 [14:34:44] 10Tool-curator: Handle orphaned uploaded files and create FilePage - https://phabricator.wikimedia.org/T414342 (10DaxServer) 03NEW [14:35:21] 10Tool-curator: Handle orphaned uploaded files and create FilePage - https://phabricator.wikimedia.org/T414342#11512461 (10DaxServer) p:05Triage→03High [14:38:24] 10Tool-curator: Inform when the camera info is not available but is used in the title template - https://phabricator.wikimedia.org/T414343 (10DaxServer) 03NEW [14:38:45] 10Tool-curator: Inform when the camera info is not available but is used in the title template - https://phabricator.wikimedia.org/T414343#11512476 (10DaxServer) p:05Triage→03High [14:54:29] 06cloud-services-team, 10Cloud-VPS: Upgrade cloudgw nodes to Trixie - https://phabricator.wikimedia.org/T414346 (10Andrew) 03NEW [14:54:54] 06cloud-services-team, 10Cloud-VPS: Upgrade cloudgw nodes to Trixie - https://phabricator.wikimedia.org/T414346#11512557 (10Andrew) [14:55:31] 06cloud-services-team, 10Cloud-VPS: Upgrade cloudgw nodes to Trixie - https://phabricator.wikimedia.org/T414346#11512561 (10taavi) →14Duplicate dup:03T401899 [14:55:32] 06cloud-services-team, 10Cloud-VPS: Reimage cloudgw hosts to Trixie - https://phabricator.wikimedia.org/T401899#11512563 (10taavi) [14:55:54] 06cloud-services-team, 10Cloud-VPS: Reimage cloudgw hosts to Trixie - https://phabricator.wikimedia.org/T401899#11512566 (10taavi) a:03Andrew [15:04:37] 10VPS-project-Codesearch, 06Abstract Wikipedia team: toolforge-repos/abstract-wiki-prototype should not be indexed in Codesearch - https://phabricator.wikimedia.org/T413322#11512618 (10matmarex) Thanks! [15:30:03] 06cloud-services-team, 10Toolforge: 2026-01-12 Toolforge NFS cleanup - https://phabricator.wikimedia.org/T414352 (10taavi) 03NEW [15:36:06] 06cloud-services-team, 06SRE Observability: Move https://status.wikimedia.org/ away from rackspace - https://phabricator.wikimedia.org/T414098#11512779 (10LSobanski) [15:50:42] 06cloud-services-team, 10Toolforge: 2026-01-12 Toolforge NFS cleanup - https://phabricator.wikimedia.org/T414352#11512840 (10taavi) a:03taavi [15:55:55] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11512868 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin2002 for host cloudlb1002.eqiad.wmnet with OS trixie [15:58:17] FIRING: JobUnavailable: Reduced availability for job cloudlb-haproxy in cloud@eqiad - https://wikitech.wikimedia.org/wiki/Prometheus#Prometheus_job_unavailable - https://grafana.wikimedia.org/d/NEJu05xZz/prometheus-targets - https://alerts.wikimedia.org/?q=alertname%3DJobUnavailable [16:01:42] (03update) 10raymond-ndibe: images: resolve the image every time [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/251 (owner: 10dcaro) [16:11:12] !log raymond-ndibe@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component logs-api [16:20:10] !log raymond-ndibe@cloudcumin1001 toolsbeta END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logs-api [16:20:11] !log raymond-ndibe@cloudcumin1001 tools START - Cookbook wmcs.toolforge.component.deploy for component logs-api [16:29:28] !log raymond-ndibe@cloudcumin1001 tools END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logs-api [16:29:29] !log raymond-ndibe@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component api-gateway [16:32:22] FIRING: [6x] HAProxyBackendUnavailable: HAProxy service wikireplica-db-analytics-s1 backend clouddb1017.eqiad.wmnet is DOWN - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyBackendUnavailable [16:33:17] !log raymond-ndibe@cloudcumin1001 toolsbeta END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway [16:33:18] !log raymond-ndibe@cloudcumin1001 tools START - Cookbook wmcs.toolforge.component.deploy for component api-gateway [16:35:11] 10Tools: Reduce NFS disk space usage by cluebotng-trainer tool - https://phabricator.wikimedia.org/T414360 (10taavi) 03NEW [16:35:52] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11513118 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin2002 for host cloudlb1002.eqiad.wmnet with OS trixie completed: - cloudlb1002 (**PASS**)... [16:36:15] 10Tool-campwiz-nxt, 10Google-Summer-of-Code (Google Summer of Code (2026)): GSoC 2026: CampWiz NxT Redesign - https://phabricator.wikimedia.org/T414269#11513123 (10LGoto) Hi @Nokib_Sarkar thanks for your submission. Please be sure to include links to microtasks for candidates. Also it would be helpful if you... [16:37:22] RESOLVED: [6x] HAProxyBackendUnavailable: HAProxy service wikireplica-db-analytics-s1 backend clouddb1017.eqiad.wmnet is DOWN - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyBackendUnavailable [16:38:18] !log raymond-ndibe@cloudcumin1001 tools END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway [16:38:19] !log raymond-ndibe@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers [16:39:47] RESOLVED: JobUnavailable: Reduced availability for job cloudlb-haproxy in cloud@eqiad - https://wikitech.wikimedia.org/wiki/Prometheus#Prometheus_job_unavailable - https://grafana.wikimedia.org/d/NEJu05xZz/prometheus-targets - https://alerts.wikimedia.org/?q=alertname%3DJobUnavailable [16:39:53] 10Tool-campwiz-nxt, 10Google-Summer-of-Code (Google Summer of Code (2026)): GSoC 2026: CampWiz NxT Redesign - https://phabricator.wikimedia.org/T414269#11513132 (10Nokib_Sarkar) [16:41:25] 10Tools: commons-delinquent error.log is unreasonably large - https://phabricator.wikimedia.org/T414361 (10taavi) 03NEW [16:42:15] 10Tool-wsindex, 10Wikisource Reader App: Books update option in local library - https://phabricator.wikimedia.org/T408285#11513151 (10Saiphani02) Can we add some visual feedback while the book is downloading? a small progress bar at the top of the library saying re-downloading book? Because right now when we c... [16:42:23] 10Tools: cluebotng-staging tool uses ~560G of disk space - https://phabricator.wikimedia.org/T395006#11513152 (10taavi) 05Open→03Resolved [16:53:59] !log raymond-ndibe@cloudcumin1001 toolsbeta END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers [16:54:00] !log raymond-ndibe@cloudcumin1001 tools START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers [17:10:40] 06cloud-services-team, 10Toolforge (Toolforge iteration 25): [jobs-api] Allow customizing time to request Loki logs for - https://phabricator.wikimedia.org/T400917#11513252 (10fnegri) Extremely ugly workaround in case you need to query logs older than 1h, and you're a Toolforge admin: `lang=shell-session root... [17:14:02] !log raymond-ndibe@cloudcumin1001 tools END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers [17:14:03] !log raymond-ndibe@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component registry-admission [17:18:25] 06cloud-services-team (FY2025/2026-Q3-Q4), 10Toolforge: updatetools is frequently timing out - https://phabricator.wikimedia.org/T413431#11513281 (10fnegri) The errors stopped for a while, then we got 14 errors in the past 2 days :/ Thanks to a [ugly workaround](https://phabricator.wikimedia.org/T400917#11513... [17:22:11] !log raymond-ndibe@cloudcumin1001 toolsbeta END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission [17:22:12] !log raymond-ndibe@cloudcumin1001 tools START - Cookbook wmcs.toolforge.component.deploy for component registry-admission [17:32:13] !log raymond-ndibe@cloudcumin1001 tools END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission [17:32:14] !log raymond-ndibe@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component components-api [17:36:59] !log raymond-ndibe@cloudcumin1001 toolsbeta END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api [17:37:00] !log raymond-ndibe@cloudcumin1001 tools START - Cookbook wmcs.toolforge.component.deploy for component components-api [17:41:57] !log raymond-ndibe@cloudcumin1001 tools END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api [17:41:58] !log raymond-ndibe@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component builds-api [17:46:47] !log raymond-ndibe@cloudcumin1001 toolsbeta END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api [17:46:48] !log raymond-ndibe@cloudcumin1001 tools START - Cookbook wmcs.toolforge.component.deploy for component builds-api [17:52:34] !log raymond-ndibe@cloudcumin1001 tools END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api [17:52:35] !log raymond-ndibe@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component envvars-api [17:53:40] !log raymond-ndibe@cloudcumin1001 toolsbeta END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-api [17:53:41] !log raymond-ndibe@cloudcumin1001 tools START - Cookbook wmcs.toolforge.component.deploy for component envvars-api [17:55:00] !log raymond-ndibe@cloudcumin1001 tools END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-api [17:55:01] !log raymond-ndibe@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission [17:55:04] !log raymond-ndibe@cloudcumin1001 toolsbeta END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component ingress-admission [17:56:16] (03update) 10raymond-ndibe: envvars-api: bump to 0.0.79-20260112134846-2681ef8d [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1108 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [17:56:18] (03approved) 10raymond-ndibe: envvars-api: bump to 0.0.79-20260112134846-2681ef8d [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1108 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [17:56:25] (03merge) 10raymond-ndibe: envvars-api: bump to 0.0.79-20260112134846-2681ef8d [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1108 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [17:56:40] (03update) 10raymond-ndibe: builds-api: bump to 0.0.206-20260112134748-5dc9dca4 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1109 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [17:56:47] (03approved) 10raymond-ndibe: builds-api: bump to 0.0.206-20260112134748-5dc9dca4 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1109 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [17:56:49] (03update) 10raymond-ndibe: builds-api: bump to 0.0.206-20260112134748-5dc9dca4 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1109 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [17:57:33] (03merge) 10raymond-ndibe: builds-api: bump to 0.0.206-20260112134748-5dc9dca4 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1109 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [17:57:40] (03update) 10raymond-ndibe: components-api: bump to 0.0.181-20260112135901-15537433 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1103 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [17:57:43] (03approved) 10raymond-ndibe: components-api: bump to 0.0.181-20260112135901-15537433 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1103 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [17:57:57] (03update) 10raymond-ndibe: components-api: bump to 0.0.181-20260112135901-15537433 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1103 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [17:59:12] !log raymond-ndibe@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission [17:59:18] !log raymond-ndibe@cloudcumin1001 toolsbeta END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component ingress-admission [18:00:41] 10Tools: Reduce NFS disk space usage by cluebotng-trainer tool - https://phabricator.wikimedia.org/T414360#11513512 (10DamianZaremba) Hi, This is not yet migrated as I have been busy since September. I will clear out some old runs to reduce the disk usage later today. [18:06:21] (03approved) 10pepepiton: Add update to hide author data fields when there are no values [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/85 (owner: 10chumzine) [18:06:30] (03merge) 10pepepiton: Add update to hide author data fields when there are no values [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/85 (owner: 10chumzine) [18:17:59] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11513570 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin2002 for host cloudlb1001.eqiad.wmnet with OS trixie [18:20:17] FIRING: JobUnavailable: Reduced availability for job cloudlb-haproxy in cloud@eqiad - https://wikitech.wikimedia.org/wiki/Prometheus#Prometheus_job_unavailable - https://grafana.wikimedia.org/d/NEJu05xZz/prometheus-targets - https://alerts.wikimedia.org/?q=alertname%3DJobUnavailable [18:21:58] 10Tool-curator: Inform when the camera info is not available but is used in the title template - https://phabricator.wikimedia.org/T414343#11513589 (10DaxServer) 05Open→03In progress [18:22:09] 10Tool-paulina: Hide author data fields when they have no value - https://phabricator.wikimedia.org/T406909#11513591 (10Pepe_piton) I merged @Chumzine solution. Thanks to everyone! [18:22:30] 10Tool-paulina: Hide author data fields when they have no value - https://phabricator.wikimedia.org/T406909#11513593 (10Pepe_piton) 05Open→03Resolved a:03Pepe_piton [18:34:11] (03approved) 10lucaswerkmeister: Localisation updates from https://translatewiki.net. [toolforge-repos/ranker] - 10https://gitlab.wikimedia.org/toolforge-repos/ranker/-/merge_requests/32 (owner: 10l10n-bot) [18:34:16] (03merge) 10lucaswerkmeister: Localisation updates from https://translatewiki.net. [toolforge-repos/ranker] - 10https://gitlab.wikimedia.org/toolforge-repos/ranker/-/merge_requests/32 (owner: 10l10n-bot) [18:35:23] (03approved) 10pepepiton: Feature: Hide work data fields when they have no value [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/161 (owner: 10chumzine) [18:35:32] (03merge) 10pepepiton: Feature: Hide work data fields when they have no value [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/161 (owner: 10chumzine) [18:44:07] 10Tool-paulina: Hide work data fields when they have no value - https://phabricator.wikimedia.org/T406994#11513672 (10Pepe_piton) 05Open→03Resolved a:03Pepe_piton I merged @Chumzine 's solution. Thanks! [18:57:22] FIRING: [2x] HAProxyBackendUnavailable: HAProxy service wikireplica-db-analytics-s1 backend clouddb1017.eqiad.wmnet is DOWN - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyBackendUnavailable [19:00:17] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11513754 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin2002 for host cloudlb1001.eqiad.wmnet with OS trixie completed: - cloudlb1001 (**PASS**)... [19:00:47] RESOLVED: JobUnavailable: Reduced availability for job cloudlb-haproxy in cloud@eqiad - https://wikitech.wikimedia.org/wiki/Prometheus#Prometheus_job_unavailable - https://grafana.wikimedia.org/d/NEJu05xZz/prometheus-targets - https://alerts.wikimedia.org/?q=alertname%3DJobUnavailable [19:05:12] (03update) 10mercy-o: Added filtering feature of both the tasks and topics categories [toolforge-repos/microtask-generator] - 10https://gitlab.wikimedia.org/toolforge-repos/microtask-generator/-/merge_requests/1 [19:07:22] RESOLVED: [2x] HAProxyBackendUnavailable: HAProxy service wikireplica-db-analytics-s1 backend clouddb1017.eqiad.wmnet is DOWN - https://wikitech.wikimedia.org/wiki/HAProxy - TODO - https://alerts.wikimedia.org/?q=alertname%3DHAProxyBackendUnavailable [19:09:50] 10Cloud Services Proposals, 06cloud-services-team, 10Cloud-VPS, 06Infrastructure-Foundations, and 2 others: Easing pain points caused by divergence between cloudservices and production puppet usecases - https://phabricator.wikimedia.org/T285539#11513787 (10bd808) I just nudged https://gerrit.wikimedia.org/... [19:32:48] (03update) 10mercy-o: Added filtering feature of both the tasks and topics categories [toolforge-repos/microtask-generator] - 10https://gitlab.wikimedia.org/toolforge-repos/microtask-generator/-/merge_requests/1 [19:45:09] (03update) 10pepepiton: Add field inception to show only when available in the work page [toolforge-repos/paulina] (add-work-inception-date) - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/150 (owner: 10sopzy) [19:45:38] (03merge) 10pepepiton: Add field inception to show only when available in the work page [toolforge-repos/paulina] (add-work-inception-date) - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/150 (owner: 10sopzy) [19:52:12] (03open) 10pepepiton: Add work inception date [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/176 [20:02:17] (03update) 10pepepiton: Add work inception date [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/176 [20:08:36] (03merge) 10pepepiton: Add work inception date [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/176 [20:24:16] 10Tool-paulina: Add field "inception" to work page - https://phabricator.wikimedia.org/T406949#11514087 (10Pepe_piton) 05Open→03Resolved a:03Pepe_piton I merged @Sopzy 's solution. Thanks!! [21:29:02] 06cloud-services-team, 06SRE Observability: Move https://status.wikimedia.org/ away from rackspace - https://phabricator.wikimedia.org/T414098#11514403 (10colewhite) We (observability) asked about this in the all-SRE meeting and most preferred to 301 status.wm.o -> wikimediastatus.net on the grounds that there... [21:41:57] FIRING: ProbeDown: Service labweb-ssl:7443 has failed probes (http_labweb-ssl_ip4) - https://wikitech.wikimedia.org/wiki/Runbook#labweb-ssl:7443 - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/service&var-module=All - https://alerts.wikimedia.org/?q=alertname%3DProbeDown [21:46:57] RESOLVED: ProbeDown: Service labweb-ssl:7443 has failed probes (http_labweb-ssl_ip4) - https://wikitech.wikimedia.org/wiki/Runbook#labweb-ssl:7443 - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/service&var-module=All - https://alerts.wikimedia.org/?q=alertname%3DProbeDown [22:58:03] (03update) 10raymond-ndibe: jobs-api: bump to 0.0.457-20260112135010-907ec7ed [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1099 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [22:58:12] (03approved) 10raymond-ndibe: jobs-api: bump to 0.0.457-20260112135010-907ec7ed [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1099 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [22:58:15] (03update) 10raymond-ndibe: jobs-api: bump to 0.0.457-20260112135010-907ec7ed [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1099 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [22:58:39] (03merge) 10raymond-ndibe: jobs-api: bump to 0.0.457-20260112135010-907ec7ed [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1099 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [22:58:43] (03update) 10raymond-ndibe: volume-admission: bump to 0.0.79-20260112134850-70a02df8 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1100 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [22:58:46] (03approved) 10raymond-ndibe: volume-admission: bump to 0.0.79-20260112134850-70a02df8 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1100 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [22:58:49] (03update) 10raymond-ndibe: volume-admission: bump to 0.0.79-20260112134850-70a02df8 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1100 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [22:59:15] (03merge) 10raymond-ndibe: volume-admission: bump to 0.0.79-20260112134850-70a02df8 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1100 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [22:59:38] (03update) 10raymond-ndibe: envvars-admission: bump to 0.0.36-20260112134910-bed27ee4 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1104 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [22:59:41] (03approved) 10raymond-ndibe: envvars-admission: bump to 0.0.36-20260112134910-bed27ee4 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1104 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [22:59:43] (03update) 10raymond-ndibe: envvars-admission: bump to 0.0.36-20260112134910-bed27ee4 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1104 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:01:06] (03merge) 10raymond-ndibe: envvars-admission: bump to 0.0.36-20260112134910-bed27ee4 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1104 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:01:17] (03update) 10raymond-ndibe: jobs-emailer: bump to 0.0.73-20260112135337-a2993cc7 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1105 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:01:20] (03approved) 10raymond-ndibe: jobs-emailer: bump to 0.0.73-20260112135337-a2993cc7 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1105 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:01:22] (03update) 10raymond-ndibe: jobs-emailer: bump to 0.0.73-20260112135337-a2993cc7 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1105 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:01:49] (03merge) 10raymond-ndibe: jobs-emailer: bump to 0.0.73-20260112135337-a2993cc7 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1105 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:02:29] (03update) 10raymond-ndibe: ingress-admission: bump to 0.0.73-20260112135120-fddb9453 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1106 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:02:31] (03approved) 10raymond-ndibe: ingress-admission: bump to 0.0.73-20260112135120-fddb9453 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1106 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:02:34] (03update) 10raymond-ndibe: ingress-admission: bump to 0.0.73-20260112135120-fddb9453 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1106 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:03:54] (03merge) 10raymond-ndibe: ingress-admission: bump to 0.0.73-20260112135120-fddb9453 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1106 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:04:15] (03update) 10raymond-ndibe: components-api: bump to 0.0.181-20260112135901-15537433 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1103 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:04:27] (03update) 10raymond-ndibe: registry-admission: bump to 0.0.70-20260112135200-937fe41c [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1107 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:04:28] (03approved) 10raymond-ndibe: registry-admission: bump to 0.0.70-20260112135200-937fe41c [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1107 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:04:44] (03merge) 10raymond-ndibe: components-api: bump to 0.0.181-20260112135901-15537433 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1103 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:04:47] (03update) 10raymond-ndibe: api-gateway: bump to 0.0.86-20260112134731-74ea55a5 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1098 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:04:47] (03approved) 10raymond-ndibe: api-gateway: bump to 0.0.86-20260112134731-74ea55a5 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1098 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:05:04] (03update) 10raymond-ndibe: maintain-kubeusers: bump to 0.0.189-20260112135303-15fc6f58 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1102 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:05:05] (03approved) 10raymond-ndibe: maintain-kubeusers: bump to 0.0.189-20260112135303-15fc6f58 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1102 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:05:21] (03update) 10raymond-ndibe: maintain-kubeusers: bump to 0.0.189-20260112135303-15fc6f58 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1102 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:05:47] (03merge) 10raymond-ndibe: maintain-kubeusers: bump to 0.0.189-20260112135303-15fc6f58 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1102 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:06:00] (03update) 10raymond-ndibe: logs-api: bump to 0.0.10-20260112135828-50dd5b8d [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1101 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:06:01] (03approved) 10raymond-ndibe: logs-api: bump to 0.0.10-20260112135828-50dd5b8d [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1101 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:06:15] (03update) 10raymond-ndibe: registry-admission: bump to 0.0.70-20260112135200-937fe41c [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1107 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:06:40] (03merge) 10raymond-ndibe: registry-admission: bump to 0.0.70-20260112135200-937fe41c [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1107 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:06:47] (03approved) 10raymond-ndibe: images: resolve the image every time [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/251 (owner: 10dcaro) [23:06:52] (03merge) 10raymond-ndibe: images: resolve the image every time [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/251 (owner: 10dcaro) [23:06:54] (03update) 10raymond-ndibe: images: support harbor-based pre-built images [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/252 (https://phabricator.wikimedia.org/T409727) (owner: 10dcaro) [23:07:14] (03update) 10raymond-ndibe: api-gateway: bump to 0.0.86-20260112134731-74ea55a5 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1098 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:07:41] (03merge) 10raymond-ndibe: api-gateway: bump to 0.0.86-20260112134731-74ea55a5 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1098 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:08:33] (03update) 10raymond-ndibe: logs-api: bump to 0.0.10-20260112135828-50dd5b8d [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1101 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:08:59] (03merge) 10raymond-ndibe: logs-api: bump to 0.0.10-20260112135828-50dd5b8d [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1101 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:09:05] (03close) 10raymond-ndibe: testing without harbor cache [repos/cloud/toolforge/jobs-api] (allow_harbor_prebuilt) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/254 [23:09:55] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: jobs-api: bump to 0.0.458-20260112230704-e764311f [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1110 [23:09:55] (03update) 10raymond-ndibe: builds-builder: bump to 0.0.139-20260112134719-8aef69fd [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1096 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:10:00] (03approved) 10raymond-ndibe: builds-builder: bump to 0.0.139-20260112134719-8aef69fd [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1096 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:10:03] (03open) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: jobs-api: bump to 0.0.458-20260112230704-e764311f [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1110 [23:10:11] (03update) 10raymond-ndibe: builds-builder: bump to 0.0.139-20260112134719-8aef69fd [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1096 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:10:37] (03merge) 10raymond-ndibe: builds-builder: bump to 0.0.139-20260112134719-8aef69fd [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1096 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:15:01] (03update) 10raymond-ndibe: maintain-harbor: bump to 0.0.66-20260112134638-71a01eba [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1097 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:15:04] (03approved) 10raymond-ndibe: maintain-harbor: bump to 0.0.66-20260112134638-71a01eba [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1097 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:15:09] (03update) 10raymond-ndibe: maintain-harbor: bump to 0.0.66-20260112134638-71a01eba [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1097 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:15:43] (03update) 10raymond-ndibe: images: support harbor-based pre-built images [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/252 (owner: 10dcaro) [23:43:27] (03update) 10raymond-ndibe: images: support harbor-based pre-built images [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/252 (https://phabricator.wikimedia.org/T409727) (owner: 10dcaro) [23:44:18] (03merge) 10raymond-ndibe: maintain-harbor: bump to 0.0.66-20260112134638-71a01eba [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1097 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [23:50:39] (03update) 10raymond-ndibe: jobs-api: bump to 0.0.458-20260112230704-e764311f [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1110 (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620)