[08:52:02] question... if mediawiki sends a metric to prometheus, when can i expect to see it in grafana? is it instantenuous? or do i need to wait for "a while"? [08:56:52] ~60s [08:57:04] that's the scraping interval right now [09:29:49] 06serviceops, 06collaboration-services, 06Data-Platform-SRE, 10Prod-Kubernetes, and 2 others: Update Kubernetes clusters to 1.31 - https://phabricator.wikimedia.org/T341984#10921973 (10JMeybohm) [09:39:41] 06serviceops, 10ChangeProp, 10Page Content Service: Investigate significant changeprop backlogs after PCS migration - https://phabricator.wikimedia.org/T397072#10922031 (10hnowlan) Since [[ https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1156389 | merging the changes to disable restbase parso... [09:49:00] 06serviceops, 10Prod-Kubernetes, 07Kubernetes: Update wikikube codfw to kubernetes 1.31 - https://phabricator.wikimedia.org/T397148 (10JMeybohm) 03NEW [11:07:22] 06serviceops, 10MediaWiki-extensions-Score, 10Wikimedia-SVG-rendering, 07Upstream: Deploy Lilypond 2.24 with cairo support to shellbox containers - https://phabricator.wikimedia.org/T385404#10922444 (10Matrix) p:05Low→03Medium Changing to medium; likely T371176 and making Score more usable in dark mode... [12:02:30] 06serviceops, 10Parsoid, 10RESTBase Sunsetting, 10Content-Transform-Team (Work In Progress), and 2 others: Reroute /v1/transform endpoints to rest gateway - https://phabricator.wikimedia.org/T396856#10922653 (10hnowlan) These APIs have been switched from restbase to the rest-gateway. [12:45:21] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q4:rack/setup/install aux-k8s-worker100[6-9].eqiad.wmnet - https://phabricator.wikimedia.org/T393053#10922962 (10Jclark-ctr) [14:16:30] 06serviceops, 06MW-Interfaces-Team, 10RESTBase Sunsetting, 07Essential-Work: Return different OpenAPI spec definitions through the REST Gateway - https://phabricator.wikimedia.org/T396806#10923588 (10HCoplin-WMF) Thanks for flagging, Hugh! We'll look into it during our next sprint to see if there's a good... [14:25:57] 06serviceops, 06MW-Interfaces-Team, 10RESTBase Sunsetting, 07Essential-Work: [SPIKE] Propose a location for hosting RESTbase OpenAPI spec definitions - https://phabricator.wikimedia.org/T397203 (10HCoplin-WMF) 03NEW [14:27:22] 06serviceops, 06MW-Interfaces-Team, 10RESTBase Sunsetting, 07Essential-Work: [SPIKE] Propose a location for hosting RESTbase OpenAPI spec definitions - https://phabricator.wikimedia.org/T397203#10923683 (10HCoplin-WMF) [15:23:11] 06serviceops, 10DiscussionTools, 10MediaWiki-REST-API, 06MW-Interfaces-Team, and 3 others: Reroute RESTbase Endpoints for Linter Errors to MediaWiki REST Endpoints - https://phabricator.wikimedia.org/T384216#10924157 (10HCoplin-WMF) p:05Medium→03High [15:29:04] swfrench-wmf: Is the plan to go to bookworm for prod things swiftly? If yes, is that also planned for the base MW image? I filed T397075 yesterday, as PHP 8.1 isn't packaged for it, but if the plan is to do the 8.1->8.3 migration first maybe it's not needed? [15:31:14] James_F: ah, good question! so the tricky part there is that going to bookworm involves stepping to ICU 72, which is a fairly involved process. instead, the plan is to prioritize the 8.3 migration on bullseye in Q1/Q2, and then follow up with the ICU + Debian upgrade in Q3/Q4. [15:31:23] Ack, OK. [15:32:08] In that case, we won't be able to bump MariaDB in CI easily. :-( [15:33:17] (MariaDB for prod is not properly packaged, so we can't re-use the Wikimedia component, which means we're stuck on bullseye's Maria DB which is older than we actually want to support in MW.) [15:33:27] (Sorry Amir1.) [15:41:55] ah, that's unfortunate ... hmmm, let me give some thought to whether there's some other way around this that doesn't involve upgrading the image to bookworm [16:58:20] 06serviceops, 06Commons, 10WMF-JobQueue: Temporarily run more refreshLinks jobs on Commons - https://phabricator.wikimedia.org/T380544#10924765 (10LucasWerkmeister) Overall, I’m inclined to say that by now it’s safe to just close this task and assume the job queue is doing its job reasonably well. However:... [17:31:35] 06serviceops, 10Parsoid, 10RESTBase Sunsetting, 10Content-Transform-Team (Work In Progress), and 2 others: Reroute /v1/transform endpoints to rest gateway - https://phabricator.wikimedia.org/T396856#10924974 (10hnowlan) 05In progress→03Resolved [17:38:45] 06serviceops, 06Infrastructure-Foundations: Upgrade httpd images to bullseye or bookworm - https://phabricator.wikimedia.org/T378128#10925040 (10Scott_French) After about and hour of soak with 1 replica per DC on the new httpd images and no issues observed, I've now moved all of syntaxhighlight forward. I've b... [21:03:56] 06serviceops, 06MediaWiki-Engineering, 13Patch-For-Review: Clean up UcfirstOverrides.php following PHP 7.4 -> 8.1 transition - https://phabricator.wikimedia.org/T394556#10925709 (10Scott_French) A couple of updates: **Comms**: Discussion is ongoing with MoveComms about including a "heads-up" for the coming... [21:14:26] 06serviceops, 06MediaWiki-Engineering, 07User-notice: Rename pages and images to reflect migration to PHP 8.1 (Unicode 14) title-casing behavior - https://phabricator.wikimedia.org/T396903#10925743 (10Scott_French) [21:20:00] 06serviceops, 06MediaWiki-Engineering, 07User-notice: Rename pages and images to reflect migration to PHP 8.1 (Unicode 14) title-casing behavior - https://phabricator.wikimedia.org/T396903#10925779 (10Scott_French)