[00:12:25] 06cloud-services-team, 10Striker, 10Phabricator: Ensure that auto-created Phabricator projects for Toolforge tools are always editable by that tool's maintainers - https://phabricator.wikimedia.org/T412607#11501865 (10bd808) >>! In T412607#11498533, @Pppery wrote: > Ancient but https://wikitech.wikimedia.org... [09:40:55] 06cloud-services-team, 10Cloud-VPS (Quota-requests), 10WikiCite, 10Wikidata, 10Wikidata-Query-Service: Raise quota on wikiqlever so that an instance with 256 GB RAM and 3 x 4 TB SSD can be launched - https://phabricator.wikimedia.org/T413097#11502583 (10Volans) @Physikerwelt if you need NFS you can refer... [10:08:10] 10wikitech.wikimedia.org: Requesting Wikitech admin access for @lsobanski - https://phabricator.wikimedia.org/T414065 (10jcrespo) 03NEW [10:08:53] 10wikitech.wikimedia.org: Requesting Wikitech admin access for @lsobanski - https://phabricator.wikimedia.org/T414065#11502636 (10jcrespo) @Quiddity @bd808 I can do this myself, just asking for someone else to ok it. [10:11:59] 10wikitech.wikimedia.org: Requesting Wikitech admin access for @lsobanski - https://phabricator.wikimedia.org/T414065#11502647 (10jcrespo) [10:12:52] 10wikitech.wikimedia.org: Requesting Wikitech admin access for @lsobanski - https://phabricator.wikimedia.org/T414065#11502652 (10jcrespo) [10:33:05] 10wikitech.wikimedia.org: Requesting Wikitech admin access for @lsobanski - https://phabricator.wikimedia.org/T414065#11502683 (10Dzahn) ` (change visibility) 10:32, 8 January 2026 Mutante talk contribs block changed group membership for LSobanski (WMF): granted administrator; kept importer unchanged (https:... [10:37:59] 10wikitech.wikimedia.org: Requesting Wikitech admin access for @lsobanski - https://phabricator.wikimedia.org/T414065#11502693 (10jcrespo) 05Open→03Resolved a:03Dzahn [10:47:05] 06cloud-services-team, 10Cloud-VPS: Alert on metricsinfra prometheus storing a few days of metrics - https://phabricator.wikimedia.org/T413985#11502721 (10fgiunchedi) 05Open→03Resolved {{done}} ` MariaDB [prometheusconfig]> insert into alerts values (NULL, 10, 'PrometheusLowRetention', '( (time() - (p... [10:47:43] 06cloud-services-team, 10Cloud-VPS: Alert on metricsinfra prometheus storing a few days of metrics - https://phabricator.wikimedia.org/T413985#11502726 (10fgiunchedi) ` MariaDB [prometheusconfig]> select * from alerts where project_id = 10\G *************************** 1. row ***************************... [11:18:11] 10Cloud-VPS (Project-requests), 10Wikidata, 10Wikidata-Query-Service: Request creation of wikiqlever VPS project - https://phabricator.wikimedia.org/T377655#11502834 (10Physikerwelt) Now starting the indexing ` physikerwelt@qlever1:/srv/qlever$ qlever get-data Command: get-data curl -LRC - -O https://d... [11:36:47] 10Cloud-VPS (Project-requests), 10Wikidata, 10Wikidata-Query-Service: Request creation of wikiqlever VPS project - https://phabricator.wikimedia.org/T377655#11502920 (10taavi) This task was the request to create the Cloud VPS project. This task is resolved as the project has been created. Please use some... [13:06:55] 10Cloud-VPS (Project-requests), 10Wikidata, 10Wikidata-Query-Service: Request creation of wikiqlever VPS project - https://phabricator.wikimedia.org/T377655#11503282 (10Physikerwelt) @taavi, sorry, as per T379030, one can now log progress via IRC again. I'll use that instead. [13:14:36] 06cloud-services-team, 10Data-Services, 06Data-Engineering, 06Data-Engineering-Radar, and 3 others: Create wiki replicas views for globaljsonlinks tables - https://phabricator.wikimedia.org/T387419#11503346 (10Gehel) [13:14:44] 10Cloud Services Proposals, 06cloud-services-team (FY2025/2026-Q3-Q4), 10Data-Services, 06Data-Persistence, and 2 others: Decision request - Who runs wikireplicas cookbooks - https://phabricator.wikimedia.org/T382607#11503348 (10Gehel) [13:20:22] 06cloud-services-team, 10Data-Services, 06Data-Engineering, 06Data-Persistence, and 3 others: Set up x1 replication to an-redacteddb1001 - https://phabricator.wikimedia.org/T407485#11503483 (10Gehel) [13:20:32] 06cloud-services-team, 10Data-Services, 10BetaFeatures, 06Data-Engineering, and 2 others: Create view for betafeatures_user_counts table in wiki replicas - https://phabricator.wikimedia.org/T402145#11503491 (10Gehel) [13:22:36] 06cloud-services-team, 10Data-Services, 06Data-Engineering, 06Data-Persistence, and 3 others: Set up x1 replication to Wiki Replicas - https://phabricator.wikimedia.org/T395881#11503540 (10Gehel) [13:53:53] 06Toolforge-standards-committee (Maintainer needed), 10Tools: wikistream.toolforge.org needs new maintainers - https://phabricator.wikimedia.org/T251555#11503684 (10RoySmith) Is this the same project as https://github.com/magnusmanske/wikistream? [14:52:17] 06cloud-services-team, 10Cloud-VPS (Quota-requests), 10WikiCite, 10Wikidata, 10Wikidata-Query-Service: Raise quota on wikiqlever so that an instance with 256 GB RAM and 3 x 4 TB SSD can be launched - https://phabricator.wikimedia.org/T413097#11503946 (10Andrew) Just a bit of context for the long-run: a V... [15:10:29] 06cloud-services-team, 06Infrastructure-Foundations: Move https://status.wikimedia.org/ off of rackspace - https://phabricator.wikimedia.org/T414098 (10Andrew) 03NEW [15:10:46] 06cloud-services-team, 06Infrastructure-Foundations: Move https://status.wikimedia.org/ off of rackspace - https://phabricator.wikimedia.org/T414098#11504016 (10Andrew) [15:11:01] 10wikitech.wikimedia.org, 06serviceops-radar, 06SRE, 13Patch-For-Review, 07SRE-Unowned: Redesign wikitech-static - https://phabricator.wikimedia.org/T376400#11504015 (10Andrew) [15:40:37] (03open) 10mercy-o: Add filtering feature of both the tasks and topics categories [toolforge-repos/microtask-generator] - 10https://gitlab.wikimedia.org/toolforge-repos/microtask-generator/-/merge_requests/1 [15:41:54] (03update) 10mercy-o: Added filtering feature of both the tasks and topics categories [toolforge-repos/microtask-generator] - 10https://gitlab.wikimedia.org/toolforge-repos/microtask-generator/-/merge_requests/1 [15:44:34] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11504211 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin2002 for host cloudbackup1003.eqiad.wmnet with OS trixie [15:44:36] 06cloud-services-team, 06Infrastructure-Foundations: Move https://status.wikimedia.org/ away from rackspace - https://phabricator.wikimedia.org/T414098#11504212 (10Reedy) [15:44:42] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11504213 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin2002 for host cloudbackup2003.codfw.wmnet with OS trixie [15:49:53] 06cloud-services-team, 06Infrastructure-Foundations: Move https://status.wikimedia.org/ away from rackspace - https://phabricator.wikimedia.org/T414098#11504222 (10hnowlan) @tappof - do you think having a static site in S3 as discussed on IRC would be sufficient? Would cert management etc make it easier to ser... [15:52:14] 06cloud-services-team, 10PAWS: Add end-to-end monitoring for PAWS - https://phabricator.wikimedia.org/T414106 (10fnegri) 03NEW [16:13:31] 10PAWS, 06tools-platform-team: New upstream release for Pywikibot - https://phabricator.wikimedia.org/T410947#11504317 (10fnegri) 05Open→03Resolved a:03fnegri PAWS updated to 10.7.4 {F71469222} [16:18:29] 06cloud-services-team, 10Toolforge, 06tools-platform-team: New upstream release for Pywikibot - https://phabricator.wikimedia.org/T410948#11504337 (10fnegri) Adding #cloud-services-team for visibility. [17:04:24] 10wikitech.wikimedia.org: Requesting Wikitech admin access for @lsobanski - https://phabricator.wikimedia.org/T414065#11504634 (10bd808) >>! In T414065#11502635, @jcrespo wrote: > @Quiddity @bd808 I can do this myself, just asking for someone else to ok it. For the future you (and anyone else), it should be... [17:08:59] (03open) 10volans: Allow to force ARM kubernetes stack [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/303 [17:31:57] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11504730 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin2002 for host cloudbackup2003.codfw.wmnet with OS trixie executed with errors: - cloudba... [17:33:03] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11504735 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin2002 for host cloudbackup2003.codfw.wmnet with OS trixie [18:13:49] 06Toolforge-standards-committee (Maintainer needed), 10Tools: wikistream.toolforge.org needs new maintainers - https://phabricator.wikimedia.org/T251555#11504867 (10bd808) >>! In T251555#11503684, @RoySmith wrote: > Is this the same project as https://github.com/magnusmanske/wikistream? No, it is not. [19:31:39] 10VPS-project-Codesearch, 06Abstract Wikipedia team: toolforge-repos/abstract-wiki-prototype should not be indexed in Codesearch - https://phabricator.wikimedia.org/T413322#11505157 (10Ammarpad) I found this annoying a number of times. This particular repo result were never useful. > 'Everything' mode on Code... [19:38:47] (03PS1) 10Ammarpad: write_config: Ignore problematic repos [labs/codesearch] - 10https://gerrit.wikimedia.org/r/1224787 (https://phabricator.wikimedia.org/T413322) [19:39:56] (03CR) 10CI reject: [V:04-1] write_config: Ignore problematic repos [labs/codesearch] - 10https://gerrit.wikimedia.org/r/1224787 (https://phabricator.wikimedia.org/T413322) (owner: 10Ammarpad) [19:42:08] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11505190 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin2002 for host cloudbackup2003.codfw.wmnet with OS trixie executed with errors: - cloudba... [19:42:13] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11505191 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin2002 for host cloudbackup1003.eqiad.wmnet with OS trixie executed with errors: - cloudba... [19:42:44] (03PS2) 10Ammarpad: write_config: Ignore problematic repos [labs/codesearch] - 10https://gerrit.wikimedia.org/r/1224787 (https://phabricator.wikimedia.org/T413322) [20:17:27] 06cloud-services-team, 10Cloud-VPS, 13Patch-For-Review: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11505387 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin2002 for host cloudbackup2003.codfw.wmnet with OS trixie [20:17:40] 06cloud-services-team, 10Cloud-VPS, 13Patch-For-Review: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11505388 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin2002 for host cloudbackup1003.eqiad.wmnet with OS trixie [20:25:19] 06cloud-services-team, 10Cloud-VPS, 13Patch-For-Review: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11505420 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin2002 for host cloudbackup2003.codfw.wmnet with OS trixie executed... [20:25:27] 06cloud-services-team, 10Cloud-VPS, 13Patch-For-Review: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11505421 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin2002 for host cloudbackup1003.eqiad.wmnet with OS trixie executed... [20:28:12] 06cloud-services-team, 10Cloud-VPS, 13Patch-For-Review: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11505427 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin2002 for host cloudbackup1003.eqiad.wmnet with OS trixie [20:28:14] 06cloud-services-team, 10Cloud-VPS, 13Patch-For-Review: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11505428 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin2002 for host cloudbackup2003.codfw.wmnet with OS trixie [20:52:35] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11505488 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin2002 for host cloudbackup1003.eqiad.wmnet with OS trixie executed with errors: - cloudba... [20:52:36] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11505489 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin2002 for host cloudbackup2003.codfw.wmnet with OS trixie executed with errors: - cloudba... [20:53:50] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11505491 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin2002 for host cloudbackup1003.eqiad.wmnet with OS trixie [21:00:17] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11505493 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin2002 for host cloudbackup1003.eqiad.wmnet with OS trixie executed with errors: - cloudba... [21:00:39] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11505495 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin2002 for host cloudbackup2003.codfw.wmnet with OS trixie [21:00:40] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11505494 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin2002 for host cloudbackup1003.eqiad.wmnet with OS trixie [21:03:59] FIRING: MaintainDBUsersManyErrors: Maintain-dbusers is having sustained errors - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/MaintainDBUsersManyErrors - https://grafana.wikimedia.org/d/ae240a06-c13e-49f3-b12c-58432c551e85/wmcs-maintain-dbusers - https://alerts.wikimedia.org/?q=alertname%3DMaintainDBUsersManyErrors [21:21:33] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11505551 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin2002 for host cloudbackup1003.eqiad.wmnet with OS trixie executed with errors: - cloudba... [21:25:22] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11505553 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin2002 for host cloudbackup2003.codfw.wmnet with OS trixie executed with errors: - cloudba... [21:35:58] (03CR) 10Ladsgroup: [C:03+1] "I'll merge if noone objects in a couple of days" [labs/codesearch] - 10https://gerrit.wikimedia.org/r/1224787 (https://phabricator.wikimedia.org/T413322) (owner: 10Ammarpad) [21:42:46] 06cloud-services-team, 10Cloud-VPS: Complete upgrading WMCS bare metal hosts to Trixie - https://phabricator.wikimedia.org/T375217#11505591 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin2002 for host cloudbackup2003.codfw.wmnet with OS trixie [22:24:17] 06cloud-services-team, 10Cloud-VPS (Quota-requests), 10WikiCite, 10Wikidata, 10Wikidata-Query-Service: Raise quota on wikiqlever so that an instance with 256 GB RAM and 3 x 4 TB SSD can be launched - https://phabricator.wikimedia.org/T413097#11505656 (10So9q) >>! In T413097#11500566, @Volans wrote: > Lim... [22:27:34] 06cloud-services-team, 10Cloud-VPS (Quota-requests), 10WikiCite, 10Wikidata, 10Wikidata-Query-Service: Raise quota on wikiqlever so that an instance with 256 GB RAM and 3 x 4 TB SSD can be launched - https://phabricator.wikimedia.org/T413097#11505671 (10So9q) >>! In T413097#11502583, @Volans wrote: > @Ph...