[00:01:09] <MacFanBot>	 [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/4c08106c8e65bf75cc75527ca61ae079be7d2364
[00:01:10] <MacFanBot>	 02statichelp/03main 07WikiTideBot 034c08106 Bot: Auto-update Tech namespace pages 2026-01-29 00:01:07
[00:13:50] <icinga-miraheze>	 RECOVERY - cp171 Disk Space on cp171 is OK: DISK OK - free space: / 72950MiB (16.1% inode=100%);
[00:55:21] <icinga-miraheze>	 RECOVERY - cp191 Disk Space on cp191 is OK: DISK OK - free space: / 73149MiB (16.1% inode=100%);
[01:09:24] <icinga-miraheze>	 RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 72707MiB (16.0% inode=100%);
[01:25:17] <MacFanBot>	 [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/cde82dbce175b79b5a713156cdfc8ae97161e9d6
[01:25:17] <MacFanBot>	 02ssl/03main 07WikiTideBot 03cde82db Bot: Auto-update domain lists
[01:45:17] <MacFanBot>	 [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/d65a08186ad2027b2c8d6f17fb7172e25cc4dc71
[01:45:18] <MacFanBot>	 02ssl/03main 07WikiTideBot 03d65a081 Bot: Auto-update domain lists
[02:45:18] <MacFanBot>	 [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/876fefd77f456a5573da813861aaea4d9969de7c
[02:45:18] <MacFanBot>	 02ssl/03main 07WikiTideBot 03876fefd Bot: Auto-update domain lists
[03:21:46] <icinga-miraheze>	 PROBLEM - test151 MediaWiki Rendering on test151 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[03:23:42] <icinga-miraheze>	 RECOVERY - test151 MediaWiki Rendering on test151 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.576 second response time
[03:35:17] <MacFanBot>	 [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/4731684535ab3a0009f985f66fba27e6c383a27d
[03:35:17] <MacFanBot>	 02ssl/03main 07WikiTideBot 034731684 Bot: Auto-update domain lists
[03:43:42] <icinga-miraheze>	 PROBLEM - mw163 MediaWiki Rendering on mw163 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.010 second response time
[03:43:50] <icinga-miraheze>	 PROBLEM - mw152 MediaWiki Rendering on mw152 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[03:43:52] <icinga-miraheze>	 PROBLEM - mw153 MediaWiki Rendering on mw153 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[03:44:20] <icinga-miraheze>	 PROBLEM - mw193 MediaWiki Rendering on mw193 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[03:44:26] <icinga-miraheze>	 PROBLEM - mw181 MediaWiki Rendering on mw181 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[03:44:38] <icinga-miraheze>	 PROBLEM - test151 MediaWiki Rendering on test151 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[03:44:42] <icinga-miraheze>	 PROBLEM - mw173 Check unit status of php8.2-fpm_check_restart on mw173 is CRITICAL: CRITICAL: Status of the systemd unit php8.2-fpm_check_restart
[03:44:42] <icinga-miraheze>	 PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is WARNING: WARNING - NGINX Error Rate is 53%
[03:44:45] <icinga-miraheze>	 PROBLEM - mw173 MediaWiki Rendering on mw173 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.012 second response time
[03:44:51] <icinga-miraheze>	 PROBLEM - mw151 MediaWiki Rendering on mw151 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[03:44:53] <icinga-miraheze>	 PROBLEM - mw163 HTTPS on mw163 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502
[03:44:56] <icinga-miraheze>	 PROBLEM - mw183 MediaWiki Rendering on mw183 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[03:44:58] <icinga-miraheze>	 PROBLEM - mw151 HTTPS on mw151 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502
[03:45:00] <icinga-miraheze>	 PROBLEM - mw192 HTTPS on mw192 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502
[03:45:06] <icinga-miraheze>	 PROBLEM - mw182 MediaWiki Rendering on mw182 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[03:45:24] <icinga-miraheze>	 PROBLEM - cp191 HTTPS on cp191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502
[03:45:36] <MirahezeRelay>	 <tangoer_man> "CRITICAL" well thats not good
[03:45:38] <icinga-miraheze>	 PROBLEM - mw161 HTTPS on mw161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502
[03:45:46] <icinga-miraheze>	 PROBLEM - mw191 MediaWiki Rendering on mw191 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[03:45:47] <icinga-miraheze>	 PROBLEM - mw161 MediaWiki Rendering on mw161 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[03:45:48] <icinga-miraheze>	 PROBLEM - mw182 HTTPS on mw182 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502
[03:45:51] <icinga-miraheze>	 PROBLEM - mw181 HTTPS on mw181 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502
[03:46:38] <icinga-miraheze>	 PROBLEM - mw191 HTTPS on mw191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502
[03:46:42] <icinga-miraheze>	 PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is CRITICAL: CRITICAL - NGINX Error Rate is 70%
[03:47:01] <icinga-miraheze>	 PROBLEM - cp201 HTTPS on cp201 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502
[03:47:19] <icinga-miraheze>	 PROBLEM - mw172 MediaWiki Rendering on mw172 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[03:47:24] <icinga-miraheze>	 PROBLEM - mw203 HTTPS on mw203 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502
[03:47:24] <icinga-miraheze>	 PROBLEM - mw171 MediaWiki Rendering on mw171 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time
[03:47:29] <icinga-miraheze>	 PROBLEM - mw162 MediaWiki Rendering on mw162 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.010 second response time
[03:47:32] <icinga-miraheze>	 PROBLEM - mw172 HTTPS on mw172 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10006 milliseconds with 0 bytes received
[03:47:38] <icinga-miraheze>	 PROBLEM - mw183 HTTPS on mw183 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10006 milliseconds with 0 bytes received
[03:47:44] <icinga-miraheze>	 RECOVERY - mw161 HTTPS on mw161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4254 bytes in 7.219 second response time
[03:47:45] <icinga-miraheze>	 RECOVERY - mw153 MediaWiki Rendering on mw153 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 2.984 second response time
[03:47:49] <icinga-miraheze>	 RECOVERY - mw191 MediaWiki Rendering on mw191 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 7.038 second response time
[03:47:49] <icinga-miraheze>	 RECOVERY - mw161 MediaWiki Rendering on mw161 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 3.685 second response time
[03:47:50] <icinga-miraheze>	 RECOVERY - mw182 HTTPS on mw182 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4254 bytes in 5.435 second response time
[03:47:53] <icinga-miraheze>	 RECOVERY - mw181 HTTPS on mw181 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4254 bytes in 2.191 second response time
[03:48:19] <icinga-miraheze>	 RECOVERY - mw193 MediaWiki Rendering on mw193 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.198 second response time
[03:48:25] <icinga-miraheze>	 RECOVERY - mw181 MediaWiki Rendering on mw181 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.192 second response time
[03:48:35] <icinga-miraheze>	 RECOVERY - mw191 HTTPS on mw191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4254 bytes in 0.069 second response time
[03:48:42] <icinga-miraheze>	 RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 34%
[03:48:44] <icinga-miraheze>	 RECOVERY - mw183 MediaWiki Rendering on mw183 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.226 second response time
[03:48:50] <icinga-miraheze>	 RECOVERY - mw173 MediaWiki Rendering on mw173 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.213 second response time
[03:48:52] <icinga-miraheze>	 RECOVERY - mw151 HTTPS on mw151 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4254 bytes in 0.066 second response time
[03:48:54] <icinga-miraheze>	 RECOVERY - mw151 MediaWiki Rendering on mw151 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.210 second response time
[03:48:58] <icinga-miraheze>	 RECOVERY - cp201 HTTPS on cp201 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4303 bytes in 0.065 second response time
[03:49:04] <icinga-miraheze>	 RECOVERY - mw192 HTTPS on mw192 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4254 bytes in 1.659 second response time
[03:49:08] <icinga-miraheze>	 RECOVERY - mw163 HTTPS on mw163 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4254 bytes in 8.059 second response time
[03:49:12] <icinga-miraheze>	 RECOVERY - mw182 MediaWiki Rendering on mw182 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.208 second response time
[03:49:17] <icinga-miraheze>	 RECOVERY - mw172 MediaWiki Rendering on mw172 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.183 second response time
[03:49:20] <icinga-miraheze>	 RECOVERY - mw203 HTTPS on mw203 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4254 bytes in 0.064 second response time
[03:49:20] <icinga-miraheze>	 PROBLEM - puppet181 Check unit status of listdomains_github_push on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit listdomains_github_push
[03:49:24] <icinga-miraheze>	 RECOVERY - mw171 MediaWiki Rendering on mw171 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.181 second response time
[03:49:25] <icinga-miraheze>	 RECOVERY - mw162 MediaWiki Rendering on mw162 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.192 second response time
[03:49:26] <icinga-miraheze>	 RECOVERY - cp191 HTTPS on cp191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4303 bytes in 0.074 second response time
[03:49:29] <icinga-miraheze>	 RECOVERY - mw172 HTTPS on mw172 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4254 bytes in 0.049 second response time
[03:49:33] <icinga-miraheze>	 RECOVERY - mw163 MediaWiki Rendering on mw163 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.197 second response time
[03:49:35] <icinga-miraheze>	 RECOVERY - mw183 HTTPS on mw183 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4254 bytes in 0.063 second response time
[03:49:48] <icinga-miraheze>	 RECOVERY - mw152 MediaWiki Rendering on mw152 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.195 second response time
[03:50:44] <icinga-miraheze>	 RECOVERY - test151 MediaWiki Rendering on test151 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.595 second response time
[03:55:20] <icinga-miraheze>	 RECOVERY - puppet181 Check unit status of listdomains_github_push on puppet181 is OK: OK: Status of the systemd unit listdomains_github_push
[04:25:59] <icinga-miraheze>	 PROBLEM - test151 MediaWiki Rendering on test151 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[04:30:00] <icinga-miraheze>	 RECOVERY - test151 MediaWiki Rendering on test151 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.575 second response time
[06:19:29] <MirahezeRelay>	 <posix_memalign> Is there a way to just upgrade MediaWiki with mwdeploy? There's `upgrade_world` which works but it updates everything. I'm just making sure https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1230889 works after it's merged and cherry-picked upstream.
[07:26:15] <MirahezeRelay>	 <rhinosf1, replying to posix_memalign> Pull core in /srv/mediawiki-staging/ manually (you could fetch and cherry pick if you want just that commit), then run mwdeploy with --file
[07:26:31] <MirahezeRelay>	 <rhinosf1> There's a --file and --folders option
[07:27:02] <MirahezeRelay>	 <rhinosf1> Which is all most of the other options are aliases for (with other magic to do stuff like l10n
[07:27:33] <MirahezeRelay>	 <cosmicalpha> Just do --world and --pull=world, no? That doesn't upgrade everything does it?
[07:28:02] <MirahezeRelay>	 <rhinosf1> that should also work
[07:28:24] <MirahezeRelay>	 <rhinosf1> I think
[07:31:09] <MirahezeRelay>	 <cosmicalpha, replying to rhinosf1> Reminds me. Mind reviewing my PR on puppet to mwdeploy?
[07:32:47] <MirahezeLSBot>	 !log [petramagna@test151] starting deploy of {'pull': 'world', 'world': True, 'versions': '1.45'} to test151
[07:32:49] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:35:46] <MirahezeLSBot>	 !log [petramagna@test151] finished deploy of {'pull': 'world', 'world': True, 'versions': '1.45'} to test151 - SUCCESS in 178s
[07:35:47] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:36:57] <MirahezeRelay>	 <rhinosf1, replying to cosmicalpha> Oh I did have that too
[07:37:01] <MirahezeRelay>	 <rhinosf1> Remind me this evening
[07:41:04] <MirahezeRelay>	 <posix_memalign, replying to cosmicalpha> Thanks! That worked.
[12:12:24] <MacFanBot>	 [02CreateWiki] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/CreateWiki/commit/4d2d9cccb134125f7e28688b3c800afb3c87999b
[12:12:24] <MacFanBot>	 02CreateWiki/03main 07translatewiki.net 034d2d9cc Localisation updates from https://translatewiki.net.
[12:12:24] <MacFanBot>	 [02MirahezeMagic] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/MirahezeMagic/commit/686d3ffef7d361e5856e13350366cee94d5787a0
[12:12:24] <MacFanBot>	 02MirahezeMagic/03main 07translatewiki.net 03686d3ff Localisation updates from https://translatewiki.net.
[12:18:17] <miraheze-github>	 miraheze/MirahezeMagic - translatewiki the build has errored.
[12:19:59] <miraheze-github>	 miraheze/CreateWiki - translatewiki the build passed.
[14:17:48] <MirahezeRelay>	 <steverichard0969> https://cdn.discordapp.com/attachments/808001911868489748/1466437177100472380/c4080f8e-b86c-42b8-b4c2-e059485a1c32.jpg?ex=697cbd8c&is=697b6c0c&hm=e11121b403665e21f2627501731907a5fe3a67cdf92a0736795fd157f3fbef12&
[17:42:05] <MacFanBot>	 [02TSPortal] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/TSPortal/commit/7e2be21db28a79028a0f15a44bed1cb408ae0322
[17:42:05] <MacFanBot>	 02TSPortal/03main 07CosmicAlpha 037e2be21 Add lockFileMaintenance to renovate
[17:45:15] <MacFanBot>	 [02TSPortal] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/TSPortal/commit/8f233ab4d59b619634ba7e86600c71039920f0f2
[17:45:15] <MacFanBot>	 02TSPortal/03main 07CosmicAlpha 038f233ab Add schedule
[17:45:56] <MacFanBot>	 [02TSPortal] 07renovate[bot] created 03renovate/lock-file-maintenance (+1 new commit) 13https://github.com/miraheze/TSPortal/commit/2ff7f6e9c2e1
[17:45:56] <MacFanBot>	 02TSPortal/03renovate/lock-file-maintenance 07renovate[bot] 032ff7f6e Lock file maintenance
[17:49:47] <MacFanBot>	 [02TSPortal] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/TSPortal/commit/f73696d7b9b565d7fc164b17fd16acbc83f95c5d
[17:49:47] <MacFanBot>	 02TSPortal/03main 07CosmicAlpha 03f73696d Run workflow on renovate branches as well
[17:50:46] <MacFanBot>	 [02TSPortal] 07renovate[bot] 04force-pushed 03renovate/lock-file-maintenance from 032ff7f6e to 03f28646f 13https://api.github.com/repos/miraheze/TSPortal/commits/renovate/lock-file-maintenance
[17:50:46] <MacFanBot>	 02TSPortal/03renovate/lock-file-maintenance 07renovate[bot] 03f28646f Lock file maintenance
[17:56:01] <MacFanBot>	 [02puppet] 07Universal-Omega created 03matomo-5.7.0 (+1 new commit) 13https://github.com/miraheze/puppet/commit/96ac357e0867
[17:56:01] <MacFanBot>	 02puppet/03matomo-5.7.0 07CosmicAlpha 0396ac357 matomo: upgrade to 5.7.0
[17:56:06] <MacFanBot>	 [02puppet] 07Universal-Omega opened pull request #4723: matomo: upgrade to 5.7.0 (03main...03matomo-5.7.0) 13https://github.com/miraheze/puppet/pull/4723
[17:56:17] <MacFanBot>	 [02puppet] 07coderabbitai[bot] commented on pull request #4723: - [ ] <!-- {"checkboxId": "fdf320d5-c1c1-4cbc-8d68-43fb5055f35f"} --> 🔍 Trigger a full review […] 13https://github.com/miraheze/puppet/pull/4723#issuecomment-3819289390
[18:00:55] <MacFanBot>	 [02TSPortal] 07renovate[bot] pushed 1 new commit to 03main 13https://github.com/miraheze/TSPortal/commit/f28646f2f345446209e7b8ef30a298680b090562
[18:00:55] <MacFanBot>	 02TSPortal/03main 07renovate[bot] 03f28646f Lock file maintenance
[18:00:57] <MacFanBot>	 [02TSPortal] 07renovate[bot] 04deleted 03renovate/lock-file-maintenance at 03f28646f 13https://api.github.com/repos/miraheze/TSPortal/commit/f28646f
[18:02:32] <MacFanBot>	 [02TSPortal] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/TSPortal/commit/a33589ccd80d6a33fda2efa4231256689240afa2
[18:02:32] <MacFanBot>	 02TSPortal/03main 07CosmicAlpha 03a33589c Update schedule
[18:05:23] <MacFanBot>	 [02TSPortal] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/TSPortal/commit/9fb0a0d879d2271089df27501561634bf8b5c678
[18:05:23] <MacFanBot>	 02TSPortal/03main 07CosmicAlpha 039fb0a0d Fix schedule
[18:09:49] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.78, 19.82, 13.02
[18:11:49] <icinga-miraheze>	 RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 10.98, 16.61, 12.70
[18:24:35] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.34, 20.03, 16.21
[18:26:30] <icinga-miraheze>	 RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 17.97, 19.14, 16.34
[18:33:16] <MacFanBot>	 [02TSPortal] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/TSPortal/commit/efe1a659c4893932306cf66d8c3d06683f7f38ba
[18:33:16] <MacFanBot>	 02TSPortal/03main 07CosmicAlpha 03efe1a65 Update index.php to new structure
[18:33:43] <MacFanBot>	 [02TSPortal] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/TSPortal/commit/cd6398c7ff8c38f17656269533c0a7cf3c97c234
[18:33:43] <MacFanBot>	 02TSPortal/03main 07CosmicAlpha 03cd6398c Spacing
[18:33:50] <icinga-miraheze>	 PROBLEM - cp171 Disk Space on cp171 is WARNING: DISK WARNING - free space: / 45313MiB (10.0% inode=100%);
[18:34:11] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 20.72, 20.96, 18.30
[18:36:06] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.26, 22.12, 19.03
[18:39:56] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 19.06, 22.05, 19.77
[18:41:51] <icinga-miraheze>	 RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 12.62, 19.12, 19.00
[19:07:21] <icinga-miraheze>	 PROBLEM - cp191 Disk Space on cp191 is WARNING: DISK WARNING - free space: / 45366MiB (10.0% inode=100%);
[19:12:26] <icinga-miraheze>	 [Grafana] FIRING: The mediawiki JobQueue backlog is increasing by more than 100 jobs a minute over an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1769710310000&orgId=1&to=1769713946679
[19:17:24] <icinga-miraheze>	 PROBLEM - cp201 Disk Space on cp201 is WARNING: DISK WARNING - free space: / 45285MiB (10.0% inode=100%);
[19:21:49] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 20.54, 18.29, 14.09
[19:22:26] <icinga-miraheze>	 [Grafana] RESOLVED: MediaWiki Jobs are rapidly increasing https://grafana.wikitide.net/d/GtxbP1Xnk?from=1769710850000&orgId=1&to=1769714450000
[19:27:49] <icinga-miraheze>	 RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 19.51, 20.29, 16.48
[19:31:50] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.96, 22.32, 18.15
[19:33:49] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.60, 22.74, 18.84
[19:39:49] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.85, 23.16, 20.19
[19:42:32] <MacFanBot>	 [02mw-config] 07pskyechology pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/16189b0ea045eb39abd5dd5868e3499af7c80346
[19:42:32] <MacFanBot>	 02mw-config/03main 07Skye 0316189b0 Ratelimit edits on loginwiki to 5 per hour…
[19:42:44] <MacFanBot>	 [02.github] 07dependabot[bot] created 03dependabot/github_actions/actions/cache-5.0.3 (+1 new commit) 13https://github.com/miraheze/.github/commit/6afe66e4ad53
[19:42:44] <MacFanBot>	 02.github/03dependabot/github_actions/actions/cache-5.0.3 07dependabot[bot] 036afe66e Bump actions/cache from 5.0.2 to 5.0.3…
[19:42:46] <MacFanBot>	 [02.github] 07dependabot[bot] added the label 'github_actions' to pull request #88 (Bump actions/cache from 5.0.2 to 5.0.3) 13https://github.com/miraheze/.github/pull/88
[19:42:48] <MacFanBot>	 [02.github] 07dependabot[bot] added the label 'dependencies' to pull request #88 (Bump actions/cache from 5.0.2 to 5.0.3) 13https://github.com/miraheze/.github/pull/88
[19:42:50] <MacFanBot>	 [02.github] 07dependabot[bot] opened pull request #88: Bump actions/cache from 5.0.2 to 5.0.3 (03main...03dependabot/github_actions/actions/cache-5.0.3) 13https://github.com/miraheze/.github/pull/88
[19:42:52] <MacFanBot>	 [02.github] 07dependabot[bot] added the label 'dependencies' to pull request #88 (Bump actions/cache from 5.0.2 to 5.0.3) 13https://github.com/miraheze/.github/pull/88
[19:42:54] <MacFanBot>	 [02.github] 07dependabot[bot] added the label 'github_actions' to pull request #88 (Bump actions/cache from 5.0.2 to 5.0.3) 13https://github.com/miraheze/.github/pull/88
[19:42:58] <MacFanBot>	 [02.github] 07coderabbitai[bot] commented on pull request #88: - [ ] <!-- {"checkboxId": "fdf320d5-c1c1-4cbc-8d68-43fb5055f35f"} --> 🔍 Trigger a full review […] 13https://github.com/miraheze/.github/pull/88#issuecomment-3819896204
[19:43:22] <MirahezeLSBot>	 !log [skye@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all
[19:43:23] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[19:43:24] <miraheze-github>	 miraheze/mw-config - pskyechology the build passed.
[19:43:54] <MirahezeLSBot>	 !log [skye@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 32s
[19:43:56] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[19:57:20] <icinga-miraheze>	 [Grafana] FIRING: The mediawiki job queue has more than 500,000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?from=1769713010000&orgId=1&to=1769716640012
[20:13:00] <MirahezeRelay>	 <justanotherdarkmodeuser, replying to icinga-miraheze> pretty sure that was me micro-editing in the login wiki earlier lol, sorry about that...skye told me to use previews next time and they also have added a ratelimit for it https://github.com/miraheze/mw-config/commit/16189b0ea045eb39abd5dd5868e3499af7c80346
[20:13:00] <MacFanBot>	 [GitHub] [miraheze/mw-config] pskyechology: Ratelimit edits on loginwiki to 5 per hour… | 6 changes in 1 file | Authored 30 minutes, 42 seconds ago | Committed 30 minutes, 42 seconds ago
[20:19:49] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 20.04, 21.72, 23.89
[20:32:20] <icinga-miraheze>	 [Grafana] RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?from=1769713010000&orgId=1&to=1769718710000
[20:59:49] <icinga-miraheze>	 RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 5.86, 14.69, 19.25
[21:05:19] <MacFanBot>	 [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/902a0c0e19cdc692885b8346488b4b799a2b2f97
[21:05:19] <MacFanBot>	 02ssl/03main 07WikiTideBot 03902a0c0 Bot: Auto-update domain lists
[21:07:21] <MacFanBot>	 [02puppet] 07paladox merged 07Universal-Omega's pull request #4723: matomo: upgrade to 5.7.0 (03main...03matomo-5.7.0) 13https://github.com/miraheze/puppet/pull/4723
[21:07:21] <MacFanBot>	 [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/7d791c99fd77171aea22b19464cc1552918c5a94
[21:07:21] <MacFanBot>	 02puppet/03main 07CosmicAlpha 037d791c9 matomo: upgrade to 5.7.0 (#4723)
[21:07:23] <MacFanBot>	 [02puppet] 07paladox 04deleted 03matomo-5.7.0 at 0396ac357 13https://api.github.com/repos/miraheze/puppet/commit/96ac357
[22:06:20] <MacFanBot>	 [02TSPortal] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/TSPortal/commit/254852ce3a97d3bae16fd00e683c99fc276dc483
[22:06:20] <MacFanBot>	 02TSPortal/03main 07CosmicAlpha 03254852c Add CHANGELOG for v27
[22:07:17] <MacFanBot>	 [02TSPortal] 07Universal-Omega released Version 27 13https://github.com/miraheze/TSPortal/releases/tag/v27
[22:07:18] <MacFanBot>	 [02TSPortal] 07Universal-Omega tagged 03v27 at 03main 13https://github.com/miraheze/TSPortal/compare/v27
[22:11:12] <MacFanBot>	 [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/21808ce4191e0f598797326acb985e6366775001
[22:11:12] <MacFanBot>	 02puppet/03main 07CosmicAlpha 0321808ce Update TSPortal to v27
[22:14:41] <MacFanBot>	 [02TSPortal] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/TSPortal/commit/1691447c7bdbe938b3fd50823e471fbd5fd190b8
[22:14:41] <MacFanBot>	 02TSPortal/03main 07CosmicAlpha 031691447 Bump version
[22:15:22] <MacFanBot>	 [02TSPortal] 07Universal-Omega 04deleted 03v27 at 03254852c 13https://api.github.com/repos/miraheze/TSPortal/commit/254852c
[22:15:54] <MacFanBot>	 [02TSPortal] 07Universal-Omega released Version 27 13https://github.com/miraheze/TSPortal/releases/tag/v27
[22:15:54] <MacFanBot>	 [02TSPortal] 07Universal-Omega tagged 03v27 at 03main 13https://github.com/miraheze/TSPortal/compare/v27
[23:14:54] <MirahezeLSBot>	 !log [void@bots171] Update package python3-pyasn1
[23:14:56] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[23:16:52] <icinga-miraheze>	 RECOVERY - bots171 APT on bots171 is OK: APT OK: 1 packages available for upgrade (0 critical updates).
[23:35:18] <MacFanBot>	 [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/87f9ff38b7ceeebf1b35bff3ed3fbdd61d19b9fd
[23:35:19] <MacFanBot>	 02ssl/03main 07WikiTideBot 0387f9ff3 Bot: Auto-update domain lists