[01:26:48] 06cloud-services-team, 10Quarry: [bug] Quarry reports offering deleted, old results - https://phabricator.wikimedia.org/T414896 (10Liz) 03NEW [01:56:50] 06cloud-services-team, 10Quarry: [bug] Quarry reports offering deleted, old results - https://phabricator.wikimedia.org/T414896#11532170 (10jeremyb) > It's like the results if I ran the queries 3 or 4 days ago. Is the database not been updated since 1/14/26? which wiki? The enwiki database should be about 13... [05:52:57] 10Tool-wsindex, 10Wikisource Reader App, 10Outreachy (Round 31): Outreachy 31: Improve the Wikisource Reader App - https://phabricator.wikimedia.org/T405593#11532281 (10Muguro) **Weekly Internship Report** //Week 6: January 12 - January 16// **Overview of Tasks Completed:** Task 1: [[ https://phabricator.... [08:33:34] FIRING: DiskSpace: Disk space cloudbackup2004:9100:/srv 6.904% free - https://wikitech.wikimedia.org/wiki/Monitoring/Disk_space - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&viewPanel=12&var-server=cloudbackup2004 - https://alerts.wikimedia.org/?q=alertname%3DDiskSpace [09:32:28] FIRING: PuppetAgentFailure: Puppet agent failure detected on instance toolsbeta-puppetserver-1 in project toolsbeta - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [09:33:28] FIRING: PuppetAgentFailure: Puppet agent failure detected on instance cloudinfra-cloudvps-puppetserver-1 in project cloudinfra - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [09:35:28] FIRING: PuppetAgentFailure: Puppet agent failure detected on instance gitlab-runners-puppetserver-01 in project gitlab-runners - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [09:40:28] FIRING: PuppetAgentFailure: Puppet agent failure detected on instance metricsinfra-puppetserver-1 in project metricsinfra - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [09:45:28] FIRING: PuppetAgentFailure: Puppet agent failure detected on instance project-proxy-puppetserver-1 in project project-proxy - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [09:47:28] FIRING: PuppetAgentFailure: Puppet agent failure detected on instance paws-puppetserver-1 in project paws - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [09:53:28] FIRING: [2x] PuppetAgentFailure: Puppet agent failure detected on instance cloudinfra-cloudvps-puppetserver-1 in project cloudinfra - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [09:54:28] FIRING: PuppetAgentFailure: Puppet agent failure detected on instance tools-puppetserver-01 in project tools - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [11:04:46] (03PS1) 10Brouberol: Provision dummy oidc client secret for airflow_sre [labs/private] - 10https://gerrit.wikimedia.org/r/1228446 [11:05:08] (03CR) 10Brouberol: [C:03+2] Provision dummy oidc client secret for airflow_sre [labs/private] - 10https://gerrit.wikimedia.org/r/1228446 (owner: 10Brouberol) [11:05:10] (03CR) 10Brouberol: [V:03+2 C:03+2] Provision dummy oidc client secret for airflow_sre [labs/private] - 10https://gerrit.wikimedia.org/r/1228446 (owner: 10Brouberol) [11:25:34] FIRING: DiskSpace: Disk space cloudbackup2004:9100:/srv 3.98% free - https://wikitech.wikimedia.org/wiki/Monitoring/Disk_space - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&viewPanel=12&var-server=cloudbackup2004 - https://alerts.wikimedia.org/?q=alertname%3DDiskSpace [11:30:28] RESOLVED: PuppetAgentFailure: Puppet agent failure detected on instance project-proxy-puppetserver-1 in project project-proxy - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [11:32:28] RESOLVED: PuppetAgentFailure: Puppet agent failure detected on instance paws-puppetserver-1 in project paws - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [11:38:28] FIRING: [2x] PuppetAgentFailure: Puppet agent failure detected on instance cloudinfra-cloudvps-puppetserver-1 in project cloudinfra - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [11:44:28] RESOLVED: PuppetAgentFailure: Puppet agent failure detected on instance tools-puppetserver-01 in project tools - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [11:47:28] RESOLVED: PuppetAgentFailure: Puppet agent failure detected on instance toolsbeta-puppetserver-1 in project toolsbeta - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [11:48:28] RESOLVED: [2x] PuppetAgentFailure: Puppet agent failure detected on instance cloudinfra-cloudvps-puppetserver-1 in project cloudinfra - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [11:50:28] RESOLVED: PuppetAgentFailure: Puppet agent failure detected on instance gitlab-runners-puppetserver-01 in project gitlab-runners - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [11:55:28] RESOLVED: PuppetAgentFailure: Puppet agent failure detected on instance metricsinfra-puppetserver-1 in project metricsinfra - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [11:56:20] (03merge) 10fnegri: Don't chmod the home directory [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/304 (https://phabricator.wikimedia.org/T414824) [12:01:05] 06cloud-services-team (FY2025/2026-Q3-Q4), 10Toolforge (Toolforge iteration 25): [lima-kilo] SSH fails after stopping and starting the VM - https://phabricator.wikimedia.org/T414824#11533385 (10fnegri) 05In progress→03Resolved [12:06:54] (03CR) 10CI reject: [V:04-1] Localisation updates from https://translatewiki.net. [labs/tools/intuition] - 10https://gerrit.wikimedia.org/r/1228461 (owner: 10L10n-bot) [12:24:51] (03update) 10mercy-o: Add recommended tasks table display and multi-filtering of topics, geography, and tasks [toolforge-repos/microtask-generator] - 10https://gitlab.wikimedia.org/toolforge-repos/microtask-generator/-/merge_requests/2 [12:26:40] (03open) 10l10n-bot: Localisation updates from https://translatewiki.net. [toolforge-repos/ranker] - 10https://gitlab.wikimedia.org/toolforge-repos/ranker/-/merge_requests/33 [12:28:32] (03CR) 10CI reject: [V:04-1] Localisation updates from https://translatewiki.net. [labs/tools/weapon-of-mass-description] - 10https://gerrit.wikimedia.org/r/1228473 (owner: 10L10n-bot) [12:30:34] RESOLVED: DiskSpace: Disk space cloudbackup2004:9100:/srv 3.691% free - https://wikitech.wikimedia.org/wiki/Monitoring/Disk_space - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&viewPanel=12&var-server=cloudbackup2004 - https://alerts.wikimedia.org/?q=alertname%3DDiskSpace [12:31:54] (03update) 10mercy-o: Add recommended tasks table display and multi-filtering of topics, geography, and tasks [toolforge-repos/microtask-generator] - 10https://gitlab.wikimedia.org/toolforge-repos/microtask-generator/-/merge_requests/2 [12:33:34] RESOLVED: DiskSpace: Disk space cloudbackup2004:9100:/srv 3.69% free - https://wikitech.wikimedia.org/wiki/Monitoring/Disk_space - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&viewPanel=12&var-server=cloudbackup2004 - https://alerts.wikimedia.org/?q=alertname%3DDiskSpace [12:37:59] (03update) 10mercy-o: Add recommended tasks table display and multi-filtering of topics, geography, and tasks [toolforge-repos/microtask-generator] - 10https://gitlab.wikimedia.org/toolforge-repos/microtask-generator/-/merge_requests/2 [13:50:05] 06cloud-services-team, 10Toolforge: Output of "toolforge jobs list" is wrong or incomplete - https://phabricator.wikimedia.org/T412081#11533902 (10NicoV) Hi @Raymond_Ndibe, a few answers: * I'm using `wpcleaner` tool account * The jobs are using WPCleaner tool * The problem with toolforge jobs saying my jobs... [14:46:16] (03update) 10mercy-o: Add recommended tasks table display and multi-filtering of topics, geography, and tasks [toolforge-repos/microtask-generator] - 10https://gitlab.wikimedia.org/toolforge-repos/microtask-generator/-/merge_requests/2 [14:46:40] (03approved) 10fnegri: runtime::diff_with_running_job: temp conditional to force job version upgrade from v1 -> v2 [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/259 (https://phabricator.wikimedia.org/T359649) (owner: 10raymond-ndibe) [14:49:25] (03update) 10mercy-o: Add recommended tasks table display and multi-filtering of topics, geography, and tasks [toolforge-repos/microtask-generator] - 10https://gitlab.wikimedia.org/toolforge-repos/microtask-generator/-/merge_requests/2 [15:26:22] 06cloud-services-team, 10Toolforge (Toolforge iteration 25): [jobs-api,image-config] Deprecate/update the list of supported pre-built images - https://phabricator.wikimedia.org/T409900#11534315 (10Raymond_Ndibe) 05Duplicate→03Resolved [15:32:34] 10Toolforge (Toolforge iteration 25): resolve “jobs-framework” and “web” image variants to the same image - https://phabricator.wikimedia.org/T414978 (10Raymond_Ndibe) 03NEW [15:37:19] (03update) 10raymond-ndibe: Draft: images::from_url_or_name: match variants of the ssame image [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/260 (https://phabricator.wikimedia.org/T414978) [15:37:46] (03update) 10raymond-ndibe: Draft: images::from_url_or_name: match variants of the ssame image [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/260 (https://phabricator.wikimedia.org/T414978) [15:38:43] (03update) 10raymond-ndibe: Draft: images::from_url_or_name: match variants of the same image [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/260 (https://phabricator.wikimedia.org/T414978) [15:39:07] (03update) 10raymond-ndibe: Draft: images::from_url_or_name: match variants of the same image [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/260 (https://phabricator.wikimedia.org/T414978) [15:44:47] (03approved) 10sbisson: Add recommended tasks table display and multi-filtering of topics, geography, and tasks [toolforge-repos/microtask-generator] - 10https://gitlab.wikimedia.org/toolforge-repos/microtask-generator/-/merge_requests/2 (owner: 10mercy-o) [15:45:02] (03merge) 10sbisson: Add recommended tasks table display and multi-filtering of topics, geography, and tasks [toolforge-repos/microtask-generator] - 10https://gitlab.wikimedia.org/toolforge-repos/microtask-generator/-/merge_requests/2 (owner: 10mercy-o) [15:48:26] (03update) 10raymond-ndibe: images::from_url_or_name: match variants of the same image [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/260 (https://phabricator.wikimedia.org/T414978) [15:49:00] 10Toolforge (Toolforge iteration 25): resolve “jobs-framework” and “web” image variants to the same image - https://phabricator.wikimedia.org/T414978#11534447 (10Raymond_Ndibe) 05Open→03In progress [16:02:04] 10Tool-wsindex, 10Wikisource Reader App: Books with title having virama characters are being joined in the API - https://phabricator.wikimedia.org/T411159#11534490 (10Bodhisattwa) [16:02:49] (03open) 10raymond-ndibe: Draft: Jobs api version that deletes and recreates jobs [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/261 [16:07:28] 06cloud-services-team, 10Toolforge (Toolforge iteration 25): [jobs-api,image-config] Deprecate/update the list of supported pre-built images - https://phabricator.wikimedia.org/T409900#11534519 (10fnegri) →14Duplicate dup:03T213641 [16:07:34] 10Cloud Services Proposals, 06cloud-services-team, 10Toolforge: Design mechanism and process for upgrading Kubernetes container runtimes - https://phabricator.wikimedia.org/T213641#11534520 (10fnegri) [16:08:08] (03update) 10raymond-ndibe: Draft: Jobs api version that deletes and recreates jobs [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/261 [16:13:16] 06cloud-services-team (FY2025/2026-Q3-Q4), 10Toolforge (Toolforge iteration 25), 13Patch-For-Review: [jobs-api,infra] upgrade all the existing toolforge jobs to the latest job version - https://phabricator.wikimedia.org/T359649#11534532 (10fnegri) Discussed with @Raymond_Ndibe, we reviewed the [announcement... [16:22:31] 10Cloud-VPS (Quota-requests): Increase quoata for wikiqlever - https://phabricator.wikimedia.org/T414983 (10Physikerwelt) 03NEW [16:26:45] 10Cloud-VPS (Quota-requests): Increase quota for wikiqlever - https://phabricator.wikimedia.org/T414983#11534641 (10Physikerwelt) [16:28:02] 10Cloud-VPS (Quota-requests): Increase quota for wikiqlever - https://phabricator.wikimedia.org/T414983#11534655 (10Physikerwelt) [16:34:34] 06cloud-services-team, 10Data-Services, 10VPS-Projects: Requesting access to NFS mount /public/dumps for wikiqlever Cloud VPS project - https://phabricator.wikimedia.org/T414986 (10Physikerwelt) 03NEW [16:35:08] 06cloud-services-team, 10Data-Services, 10VPS-Projects: Requesting access to NFS mount /public/dumps for wikiqlever Cloud VPS project - https://phabricator.wikimedia.org/T414986#11534687 (10Physikerwelt) p:05Triage→03Low [16:40:53] (03update) 10raymond-ndibe: jobs-api: add jobs version migration script and docs [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/746 (https://phabricator.wikimedia.org/T359649) (owner: 10dcaro) [16:47:44] 10Tool-schedule-deployment: Extend functionality to support MediaWiki infrastructure Windows and related repos - https://phabricator.wikimedia.org/T385007#11534721 (10MLechvien-WMF) [16:49:34] 06cloud-services-team, 10Data-Services, 10VPS-Projects: Requesting access to NFS mount /public/dumps for wikiqlever Cloud VPS project - https://phabricator.wikimedia.org/T414986#11534729 (10fnegri) a:03fnegri [16:54:37] 06cloud-services-team, 10Quarry: [bug] Quarry reports offering deleted, old results - https://phabricator.wikimedia.org/T414896#11534748 (10fnegri) 05Open→03Resolved a:03fnegri Marking as Resolved as all dbs are now back in sync. It is expected that sometimes Quarry will provide slightly outdated res... [16:58:33] 06cloud-services-team, 10Quarry: [bug] Quarry reports offering deleted, old results - https://phabricator.wikimedia.org/T414896#11534767 (10taavi) 05Resolved→03Invalid a:05fnegri→03None [19:00:52] (03approved) 10lucaswerkmeister: Localisation updates from https://translatewiki.net. [toolforge-repos/ranker] - 10https://gitlab.wikimedia.org/toolforge-repos/ranker/-/merge_requests/33 (owner: 10l10n-bot) [19:00:58] (03merge) 10lucaswerkmeister: Localisation updates from https://translatewiki.net. [toolforge-repos/ranker] - 10https://gitlab.wikimedia.org/toolforge-repos/ranker/-/merge_requests/33 (owner: 10l10n-bot) [19:19:12] (03update) 10raymond-ndibe: Draft: Jobs api version that deletes and recreates jobs [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/261 [19:52:41] (03update) 10raymond-ndibe: images: support harbor-based pre-built images [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/252 (https://phabricator.wikimedia.org/T409727) (owner: 10dcaro) [19:59:09] (03update) 10raymond-ndibe: [jobs-api] save business models in a DB [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/114 [20:04:28] (03update) 10raymond-ndibe: core: add prometheus counter for jobs synced from runtime [repos/cloud/toolforge/jobs-api] (use_custom_resources_in_code) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/253 (owner: 10dcaro) [20:05:44] (03update) 10raymond-ndibe: core: add prometheus counter for jobs synced from runtime [repos/cloud/toolforge/jobs-api] (use_custom_resources_in_code) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/253 (owner: 10dcaro) [20:09:13] (03update) 10raymond-ndibe: core: add prometheus counter for jobs synced from runtime [repos/cloud/toolforge/jobs-api] (use_custom_resources_in_code) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/253 (owner: 10dcaro) [20:10:37] (03update) 10raymond-ndibe: core: add prometheus counter for jobs synced from runtime [repos/cloud/toolforge/jobs-api] (use_custom_resources_in_code) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/253 (owner: 10dcaro)