[04:49:31] 10Gerrit, 06collaboration-services: gerrit2003 is trying to backup incrementally 3.5 million files every hour, clogging backus and filling in available disk space - https://phabricator.wikimedia.org/T406762#11267442 (10ABran-WMF) Given T387833#11267438, that should not be an issue anymore, I'll send a patch to... [05:26:42] 10Gerrit, 06collaboration-services, 13Patch-For-Review: gerrit2003 is trying to backup incrementally 3.5 million files every hour, clogging backus and filling in available disk space - https://phabricator.wikimedia.org/T406762#11267481 (10ABran-WMF) a:03ABran-WMF [07:22:04] (03PS1) 10Hslater: Zuul: [mediawiki/extensions/BlueSpiceUEModuleTable2Excel] Add BlueSpiceVisualEditorConnector as dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1195543 [07:23:31] (03CR) 10CI reject: [V:04-1] Zuul: [mediawiki/extensions/BlueSpiceUEModuleTable2Excel] Add BlueSpiceVisualEditorConnector as dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1195543 (owner: 10Hslater) [07:24:05] (03PS2) 10Hslater: Zuul: [mediawiki/extensions/BlueSpiceUEModuleTable2Excel] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1195543 [07:53:15] 10GitLab, 06collaboration-services: Remove gitlab-ce and gitlab-runner bullseye package and component - https://phabricator.wikimedia.org/T406823#11267614 (10Jelto) [09:10:15] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/extensions/BlueSpiceUEModuleTable2Excel] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1195543 (owner: 10Hslater) [09:11:50] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/BlueSpiceUEModuleTable2Excel] Add dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1195543 (owner: 10Hslater) [09:13:24] (03CR) 10Hashar: [C:03+2] "Deployed! 😊" [integration/config] - 10https://gerrit.wikimedia.org/r/1195543 (owner: 10Hslater) [09:34:26] 10GitLab (Account Approval), 06Release-Engineering-Team (Doing 😎): Requesting GitLab account activation for sanjaisid - https://phabricator.wikimedia.org/T407078#11267901 (10Aklapper) 05Open→03Resolved a:03Aklapper Done. [09:44:46] (03PS1) 10Hslater: Zuul: [mediawiki/extensions/PDFCreator] Add BlueSpiceDiscovery as dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1195637 [10:06:28] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/extensions/PDFCreator] Add BlueSpiceDiscovery as dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1195637 (owner: 10Hslater) [10:06:45] (03CR) 10Hashar: [C:03+2] Fix link to task in the motd banner [software/gerrit] (deploy/wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1194573 (owner: 10Hashar) [10:07:46] (03Merged) 10jenkins-bot: Fix link to task in the motd banner [software/gerrit] (deploy/wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1194573 (owner: 10Hashar) [10:08:07] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/PDFCreator] Add BlueSpiceDiscovery as dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1195637 (owner: 10Hslater) [10:09:52] (03CR) 10Hashar: [C:03+2] "Deployed!" [integration/config] - 10https://gerrit.wikimedia.org/r/1195637 (owner: 10Hslater) [11:28:39] 10Phabricator, 06Release-Engineering-Team (Priority Backlog 📥): Review and clean up Unresolved Setup Issues in production - https://phabricator.wikimedia.org/T401163#11268165 (10Aklapper) Fixed some low hanging fruit for the start: * https://phabricator.wikimedia.org/config/issue/config.unknown.gitblit.hostnam... [11:41:27] 10Phabricator, 06Release-Engineering-Team (Priority Backlog 📥): Review and clean up Unresolved Setup Issues in production - https://phabricator.wikimedia.org/T401163#11268206 (10A_smart_kitten) > * https://phabricator.wikimedia.org/config/issue/config.unknown.gitblit.hostname/ - `sudo ./bin/config delete gitbl... [11:42:39] 10Phabricator, 06Release-Engineering-Team (Priority Backlog 📥): Unknown Configuration Option "gitlab.api_key" in Phab Local Config - https://phabricator.wikimedia.org/T407111 (10Aklapper) 03NEW p:05Triage→03Low [11:51:38] 10GitLab, 06collaboration-services, 13Patch-For-Review: Remove gitlab-ce and gitlab-runner bullseye package and component - https://phabricator.wikimedia.org/T406823#11268242 (10Jelto) p:05Triage→03Medium a:03Jelto [11:52:00] 10GitLab, 06collaboration-services, 13Patch-For-Review: Remove gitlab-ce and gitlab-runner bullseye package and component - https://phabricator.wikimedia.org/T406823#11268245 (10Jelto) [12:06:01] 10GitLab, 06collaboration-services, 13Patch-For-Review: Remove gitlab-ce and gitlab-runner bullseye package and component - https://phabricator.wikimedia.org/T406823#11268298 (10Jelto) [12:08:49] 10GitLab, 06collaboration-services, 13Patch-For-Review: Remove gitlab-ce and gitlab-runner bullseye package and component - https://phabricator.wikimedia.org/T406823#11268300 (10Jelto) 05Open→03Resolved The `gitlab-ce` and `gitlab-runner` packages were removed from the `Update` section in the apt rep... [13:21:04] 06Release-Engineering-Team (Priority Backlog 📥), 07Essential-Work, 05Release, 05Train Deployments: 1.45.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T405679#11268497 (10hashar) @Urbanecm @Michael Thank you for the risky patch mention. I will run the train on Tuesday at 10:00 CEST, If I... [13:27:35] 06Release-Engineering-Team (Priority Backlog 📥), 07Essential-Work, 05Release, 05Train Deployments: 1.45.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T405679#11268525 (10Urbanecm_WMF) Sounds good @hashar! Please note the errors might also come from subclasses (they should all have "provi... [15:06:12] 10Phabricator, 06Security-Team, 07SecTeam-Processed, 07Security: Create a security issue reporting template - https://phabricator.wikimedia.org/T406313#11268897 (10Aklapper) @Reedy: What is exactly wanted, for which of the existing forms? [15:25:02] maintenance-disconnect-full-disks build 745357 integration-agent-docker-1043 (/: 35%, /srv: 97%, /var/lib/docker: 37%): OFFLINE due to disk space [15:30:03] maintenance-disconnect-full-disks build 745358 integration-agent-docker-1043 (/: 35%, /srv: 72%, /var/lib/docker: 36%): RECOVERY disk space OK [15:36:04] 10Phabricator (phabricator-next), 06Release-Engineering-Team (Doing 😎): PhabricatorMetaMTAMailableFunctionDatasource does not need to query PhabricatorProjectDatasource - https://phabricator.wikimedia.org/T399111#11269034 (10Aklapper) a:03Aklapper Done in https://gitlab.wikimedia.org/repos/phabricator/phabri... [17:53:36] 10Continuous-Integration-Config, 10Page-Previews, 07ci-test-error (WMF-deployed Build Failure): CI is broken for lots of MW repos due to extension/Popups explicitly requiring node 18 - https://phabricator.wikimedia.org/T382192#11269392 (10Amrok84) >>! In T382192#10405956, @Novem_Linguae wrote: > There ar... [23:21:37] 06Release-Engineering-Team, 06collaboration-services: ProbeDown - https://phabricator.wikimedia.org/T407169 (10phaultfinder) 03NEW [23:21:40] FIRING: ProbeDown: Service gerrit1003:443 has failed probes (http_gerrit_tls_ip4) - https://wikitech.wikimedia.org/wiki/Runbook#gerrit1003:443 - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://alerts.wikimedia.org/?q=alertname%3DProbeDown [23:26:31] FIRING: [2x] ProbeDown: Service gerrit1003:443 has failed probes (http_gerrit_tls_ip4) - https://wikitech.wikimedia.org/wiki/Runbook#gerrit1003:443 - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://alerts.wikimedia.org/?q=alertname%3DProbeDown [23:26:43] 06Release-Engineering-Team, 06collaboration-services: ProbeDown - https://phabricator.wikimedia.org/T407169#11269941 (10phaultfinder) [23:36:31] RESOLVED: [2x] ProbeDown: Service gerrit1003:443 has failed probes (http_gerrit_tls_ip4) - https://wikitech.wikimedia.org/wiki/Runbook#gerrit1003:443 - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://alerts.wikimedia.org/?q=alertname%3DProbeDown