[00:01:20] [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/abccc02e5c1a918e83e3fc1daee34df07f8a289d [00:01:20] 02statichelp/03main 07WikiTideBot 03abccc02 Bot: Auto-update Tech namespace pages 2025-12-05 00:01:17 [00:05:31] !log [somerandomdeveloper@mwtask181] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php CreateWiki:PopulateMainPage --wiki=killgoesewiki (END - exit=0) [00:05:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:07:06] !log [somerandomdeveloper@mwtask181] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php createAndPromote --wiki=killgoesewiki KILLGOESE --force --bureaucrat --interface-admin --sysop (END - exit=0) [00:07:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:12:00] RECOVERY - cp171 Disk Space on cp171 is OK: DISK OK - free space: / 72538MiB (15% inode=99%); [00:13:56] RECOVERY - cp191 Disk Space on cp191 is OK: DISK OK - free space: / 72557MiB (15% inode=99%); [00:14:37] RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 71695MiB (15% inode=99%); [03:05:00] [02puppet] 07paladox created 03paladox-patch-6 (+1 new commit) 13https://github.com/miraheze/puppet/commit/9e64091ef0cf [03:05:00] 02puppet/03paladox-patch-6 07paladox 039e64091 dns: Fix Recursor file ending [03:05:04] [02puppet] 07paladox opened pull request #4622: dns: Fix Recursor file ending (03main...03paladox-patch-6) 13https://github.com/miraheze/puppet/pull/4622 [03:05:09] [02puppet] 07coderabbitai[bot] commented on pull request #4622:

[…] 13https://github.com/miraheze/puppet/pull/4622#issuecomment-3615121377 [03:05:22] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/d9b9503be7d0bf15363040a6daa685f8c9cbf334 [03:05:23] 02ssl/03main 07WikiTideBot 03d9b9503 Bot: Auto-update domain lists [03:05:25] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-6 13https://github.com/miraheze/puppet/commit/3597fe7916e7912fa1f0bf1957f38e5e900796a4 [03:05:26] 02puppet/03paladox-patch-6 07paladox 033597fe7 Rename recursor.yaml.erb to recursor.yml.erb [03:05:31] [02puppet] 07paladox merged pull request #4622: dns: Fix Recursor file ending (03main...03paladox-patch-6) 13https://github.com/miraheze/puppet/pull/4622 [03:05:32] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/0389c732c0e856a6825b9c75fe189850d0a76d40 [03:05:32] 02puppet/03main 07paladox 030389c73 dns: Fix Recursor file ending (#4622) [03:05:33] [02puppet] 07paladox 04deleted 03paladox-patch-6 at 033597fe7 13https://api.github.com/repos/miraheze/puppet/commit/3597fe7 [03:07:01] RECOVERY - db201 PowerDNS Recursor on db201 is OK: DNS OK: 0.031 seconds response time. db201.fsslc.wtnet returns 10.0.20.169 [03:07:54] [02mw-config] 07paladox merged pull request #6197: Use db201 (10.0.20.169) for parsercache (03main...03paladox-patch-4) 13https://github.com/miraheze/mw-config/pull/6197 [03:07:54] [02mw-config] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/d5c5b60d71143e545baa5dac49873ad5f86e60fb [03:07:55] 02mw-config/03main 07paladox 03d5c5b60 Use db201 (10.0.20.169) for parsercache (#6197) [03:07:56] [02mw-config] 07paladox 04deleted 03paladox-patch-4 at 038cd3c9a 13https://api.github.com/repos/miraheze/mw-config/commit/8cd3c9a [03:08:26] !log [paladox@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [03:08:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:08:49] !log [paladox@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 23s [03:08:54] miraheze/mw-config - paladox the build passed. [03:08:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:35:30] !log [blankeclair@mwtask181] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php MirahezeMagic:RenameDatabase --wiki=loginwiki --rename --old=venturepunkwiki --new=clusterpunkwiki --user='BlankEclair_(Miraheze)' (END - exit=0) [03:35:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:37:59] !log [blankeclair@mwtask181] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php MirahezeMagic:RenameDatabase --wiki=loginwiki --rename --old=wingliscwiki --new=angelnischwiki --user='BlankEclair_(Miraheze)' (END - exit=0) [03:38:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:57:42] PROBLEM - puppet181 Check unit status of listdomains_github_push on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit listdomains_github_push [04:04:04] PROBLEM - mw183 MediaWiki Rendering on mw183 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.016 second response time [04:04:06] PROBLEM - mw171 MediaWiki Rendering on mw171 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [04:04:10] PROBLEM - mw202 MediaWiki Rendering on mw202 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.023 second response time [04:04:12] PROBLEM - mw163 MediaWiki Rendering on mw163 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [04:04:12] PROBLEM - mw152 MediaWiki Rendering on mw152 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [04:04:14] PROBLEM - mw161 HTTPS on mw161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:15] PROBLEM - mw203 HTTPS on mw203 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:15] PROBLEM - mw173 MediaWiki Rendering on mw173 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.012 second response time [04:04:15] PROBLEM - mw181 MediaWiki Rendering on mw181 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.012 second response time [04:04:16] PROBLEM - mw193 HTTPS on mw193 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:16] PROBLEM - mw192 MediaWiki Rendering on mw192 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.012 second response time [04:04:16] PROBLEM - mw151 HTTPS on mw151 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:17] PROBLEM - mw201 HTTPS on mw201 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:19] PROBLEM - mw182 MediaWiki Rendering on mw182 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.013 second response time [04:04:20] PROBLEM - cp191 HTTPS on cp191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:20] PROBLEM - mw201 MediaWiki Rendering on mw201 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.012 second response time [04:04:20] PROBLEM - mw173 HTTPS on mw173 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:21] PROBLEM - mw202 HTTPS on mw202 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:21] PROBLEM - mw162 HTTPS on mw162 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:21] PROBLEM - mw192 HTTPS on mw192 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:22] PROBLEM - mw171 HTTPS on mw171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:23] PROBLEM - mw203 MediaWiki Rendering on mw203 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.012 second response time [04:04:24] PROBLEM - mw152 HTTPS on mw152 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:25] PROBLEM - mw191 HTTPS on mw191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:25] PROBLEM - mw172 HTTPS on mw172 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:27] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 11 backends are down. mw151 mw161 mw162 mw181 mw153 mw163 mw173 mw191 mw192 mw193 mw201 [04:04:29] PROBLEM - mw191 MediaWiki Rendering on mw191 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.010 second response time [04:04:29] PROBLEM - cp161 HTTPS on cp161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:30] PROBLEM - mw153 HTTPS on mw153 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:31] PROBLEM - mw163 HTTPS on mw163 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:31] PROBLEM - cp171 HTTPS on cp171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:34] PROBLEM - mw172 MediaWiki Rendering on mw172 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.017 second response time [04:04:37] PROBLEM - mw182 HTTPS on mw182 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:37] PROBLEM - db151 Current Load on db151 is CRITICAL: LOAD CRITICAL - total load average: 196.32, 90.93, 36.38 [04:04:38] PROBLEM - mw181 HTTPS on mw181 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:39] PROBLEM - mw183 HTTPS on mw183 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:44] PROBLEM - cp191 Varnish Backends on cp191 is CRITICAL: 11 backends are down. mw151 mw161 mw162 mw181 mw153 mw163 mw173 mw191 mw192 mw193 mw201 [04:04:45] PROBLEM - mw153 MediaWiki Rendering on mw153 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.012 second response time [04:04:45] PROBLEM - mw162 MediaWiki Rendering on mw162 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [04:04:52] PROBLEM - mw193 MediaWiki Rendering on mw193 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [04:04:56] PROBLEM - mw151 MediaWiki Rendering on mw151 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.012 second response time [04:05:07] god damn [04:05:18] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 11 backends are down. mw151 mw161 mw162 mw181 mw153 mw163 mw173 mw191 mw192 mw193 mw201 [04:05:32] @Infrastructure Specialists @MediaWiki Specialists [04:05:42] RECOVERY - puppet181 Check unit status of listdomains_github_push on puppet181 is OK: OK: Status of the systemd unit listdomains_github_push [04:05:45] PROBLEM - cp201 HTTPS on cp201 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:06:04] PROBLEM - mw161 MediaWiki Rendering on mw161 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:06:05] ah fuck [04:06:57] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is CRITICAL: CRITICAL - NGINX Error Rate is 86% [04:07:01] it's db151 [04:08:12] PROBLEM - db151 MariaDB Connections on db151 is UNKNOWN: [04:08:26] PROBLEM - db151 APT on db151 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:08:28] i assume you'll deal w everything? [04:08:30] !log [paladox@mwtask181] starting deploy of {'config': True} to all [04:08:32] RECOVERY - mw151 HTTPS on mw151 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 7.730 second response time [04:08:33] RECOVERY - mw191 HTTPS on mw191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 4.586 second response time [04:08:33] RECOVERY - mw201 MediaWiki Rendering on mw201 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 8.277 second response time [04:08:33] RECOVERY - mw162 HTTPS on mw162 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 5.745 second response time [04:08:34] I am [04:08:35] RECOVERY - mw192 HTTPS on mw192 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 6.015 second response time [04:08:35] RECOVERY - mw153 HTTPS on mw153 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.041 second response time [04:08:38] RECOVERY - mw191 MediaWiki Rendering on mw191 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.190 second response time [04:08:39] RECOVERY - mw172 MediaWiki Rendering on mw172 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 4.768 second response time [04:08:40] RECOVERY - mw163 HTTPS on mw163 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.060 second response time [04:08:40] RECOVERY - mw183 HTTPS on mw183 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 3.011 second response time [04:08:41] oki thx [04:08:41] RECOVERY - mw182 HTTPS on mw182 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 3.881 second response time [04:08:42] RECOVERY - cp161 HTTPS on cp161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4328 bytes in 3.860 second response time [04:08:42] I'm restarting mariadb on db151 [04:08:46] RECOVERY - mw181 HTTPS on mw181 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.069 second response time [04:08:50] i was having a psychological discussion w/ someone else [04:08:52] RECOVERY - mw153 MediaWiki Rendering on mw153 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.260 second response time [04:08:53] RECOVERY - mw162 MediaWiki Rendering on mw162 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.259 second response time [04:08:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:08:57] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is WARNING: WARNING - NGINX Error Rate is 54% [04:09:00] RECOVERY - mw193 MediaWiki Rendering on mw193 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.194 second response time [04:09:02] RECOVERY - mw151 MediaWiki Rendering on mw151 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.308 second response time [04:09:06] !log [paladox@mwtask181] finished deploy of {'config': True} to all - SUCCESS in 36s [04:09:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:09:18] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [04:09:27] PROBLEM - db151 Puppet on db151 is CRITICAL: connect to address 10.0.15.110 port 5666: Connection refusedconnect to host 10.0.15.110 port 5666: Connection refused [04:09:42] PROBLEM - puppet181 Check unit status of listdomains_github_push on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit listdomains_github_push [04:09:45] RECOVERY - cp201 HTTPS on cp201 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4273 bytes in 0.066 second response time [04:09:47] RECOVERY - mw183 MediaWiki Rendering on mw183 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.210 second response time [04:09:49] RECOVERY - mw171 MediaWiki Rendering on mw171 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.280 second response time [04:09:54] !log [paladox@mwtask181] starting deploy of {'config': True} to all [04:09:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:10:01] RECOVERY - mw161 MediaWiki Rendering on mw161 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.229 second response time [04:10:02] RECOVERY - mw202 MediaWiki Rendering on mw202 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.222 second response time [04:10:04] RECOVERY - mw202 HTTPS on mw202 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.068 second response time [04:10:04] RECOVERY - mw203 HTTPS on mw203 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.074 second response time [04:10:07] RECOVERY - mw152 MediaWiki Rendering on mw152 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.222 second response time [04:10:07] RECOVERY - mw203 MediaWiki Rendering on mw203 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.216 second response time [04:10:13] RECOVERY - cp191 HTTPS on cp191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4273 bytes in 0.068 second response time [04:10:15] RECOVERY - mw152 HTTPS on mw152 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.061 second response time [04:10:15] RECOVERY - mw182 MediaWiki Rendering on mw182 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.215 second response time [04:10:16] RECOVERY - mw192 MediaWiki Rendering on mw192 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.197 second response time [04:10:17] RECOVERY - mw173 MediaWiki Rendering on mw173 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.202 second response time [04:10:17] RECOVERY - mw172 HTTPS on mw172 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.063 second response time [04:10:17] !log [paladox@mwtask181] finished deploy of {'config': True} to all - SUCCESS in 23s [04:10:19] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [04:10:20] RECOVERY - mw193 HTTPS on mw193 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.071 second response time [04:10:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:10:22] RECOVERY - mw171 HTTPS on mw171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.062 second response time [04:10:23] RECOVERY - mw161 HTTPS on mw161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.048 second response time [04:10:25] RECOVERY - mw163 MediaWiki Rendering on mw163 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.181 second response time [04:10:25] RECOVERY - mw173 HTTPS on mw173 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.058 second response time [04:10:26] PROBLEM - db151 conntrack_table_size on db151 is CRITICAL: connect to address 10.0.15.110 port 5666: Connection refusedconnect to host 10.0.15.110 port 5666: Connection refused [04:10:26] RECOVERY - cp171 HTTPS on cp171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4273 bytes in 0.064 second response time [04:10:30] RECOVERY - mw181 MediaWiki Rendering on mw181 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.221 second response time [04:10:31] RECOVERY - mw201 HTTPS on mw201 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.069 second response time [04:10:33] PROBLEM - db151 Disk Space on db151 is CRITICAL: connect to address 10.0.15.110 port 5666: Connection refusedconnect to host 10.0.15.110 port 5666: Connection refused [04:10:44] RECOVERY - cp191 Varnish Backends on cp191 is OK: All 31 backends are healthy [04:10:57] RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 12% [04:11:15] PROBLEM - db151 Check unit status of sql-backup on db151 is CRITICAL: connect to address 10.0.15.110 port 5666: Connection refusedconnect to host 10.0.15.110 port 5666: Connection refused [04:11:21] PROBLEM - db151 NTP time on db151 is CRITICAL: connect to address 10.0.15.110 port 5666: Connection refusedconnect to host 10.0.15.110 port 5666: Connection refused [04:11:24] PROBLEM - db151 SSH on db151 is CRITICAL: connect to address 10.0.15.110 and port 22: Connection refused [04:12:13] PROBLEM - db151 PowerDNS Recursor on db151 is CRITICAL: connect to address 10.0.15.110 port 5666: Connection refusedconnect to host 10.0.15.110 port 5666: Connection refused [04:12:21] PROBLEM - db151 MariaDB on db151 is CRITICAL: Can't connect to server on 'db151.fsslc.wtnet' (115) [04:12:22] PROBLEM - db151 ferm_active on db151 is CRITICAL: connect to address 10.0.15.110 port 5666: Connection refusedconnect to host 10.0.15.110 port 5666: Connection refused [04:14:22] RECOVERY - db151 ferm_active on db151 is OK: OK ferm input default policy is set [04:14:26] RECOVERY - db151 conntrack_table_size on db151 is OK: OK: nf_conntrack is 0 % full [04:14:33] RECOVERY - db151 Disk Space on db151 is OK: DISK OK - free space: / 410680MiB (46% inode=97%); [04:14:36] RECOVERY - db151 Current Load on db151 is OK: LOAD OK - total load average: 0.48, 0.11, 0.04 [04:15:17] RECOVERY - db151 Check unit status of sql-backup on db151 is OK: OK: Status of the systemd unit sql-backup [04:15:23] RECOVERY - db151 NTP time on db151 is OK: NTP OK: Offset 0.01953214407 secs [04:15:24] RECOVERY - db151 SSH on db151 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u7 (protocol 2.0) [04:15:42] RECOVERY - puppet181 Check unit status of listdomains_github_push on puppet181 is OK: OK: Status of the systemd unit listdomains_github_push [04:15:49] RECOVERY - db151 MariaDB Connections on db151 is OK: OK connection usage: 48%Current connections: 480 [04:16:12] RECOVERY - db151 MariaDB on db151 is OK: Uptime: 116 Threads: 477 Questions: 3089 Slow queries: 1 Opens: 273 Open tables: 267 Queries per second avg: 26.629 [04:16:17] RECOVERY - db151 PowerDNS Recursor on db151 is OK: DNS OK: 2.126 seconds response time. db151.fsslc.wtnet returns 10.0.15.110 [04:16:20] RECOVERY - db151 Puppet on db151 is OK: OK: Puppet is currently enabled, last run 16 minutes ago with 0 failures [04:16:57] i think i misread this as disaster [04:17:27] PROBLEM - mw171 MediaWiki Rendering on mw171 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [04:17:34] PROBLEM - mw183 MediaWiki Rendering on mw183 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [04:17:39] i mean, deny a suicidal trans person hrt and you get a psychological disaster lol [04:17:43] PROBLEM - mw202 HTTPS on mw202 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:17:45] PROBLEM - cp201 HTTPS on cp201 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:17:47] PROBLEM - mw203 MediaWiki Rendering on mw203 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [04:17:50] PROBLEM - mw202 MediaWiki Rendering on mw202 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [04:17:52] PROBLEM - mw173 MediaWiki Rendering on mw173 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.010 second response time [04:17:52] PROBLEM - mw203 HTTPS on mw203 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:17:55] can someone look at this [04:17:57] PROBLEM - mw161 MediaWiki Rendering on mw161 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [04:17:57] bed too cozy [04:17:59] PROBLEM - mw152 HTTPS on mw152 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:18:00] PROBLEM - mw192 MediaWiki Rendering on mw192 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.010 second response time [04:18:00] PROBLEM - mw193 HTTPS on mw193 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:18:02] PROBLEM - cp191 HTTPS on cp191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:18:03] PROBLEM - mw172 HTTPS on mw172 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:18:06] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 11 backends are down. mw151 mw161 mw162 mw181 mw153 mw163 mw173 mw191 mw192 mw193 mw201 [04:18:07] PROBLEM - mw152 MediaWiki Rendering on mw152 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [04:18:12] PROBLEM - mw161 HTTPS on mw161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:18:13] PROBLEM - mw173 HTTPS on mw173 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:18:14] PROBLEM - mw163 MediaWiki Rendering on mw163 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.012 second response time [04:18:15] PROBLEM - mw182 MediaWiki Rendering on mw182 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [04:18:15] PROBLEM - mw191 HTTPS on mw191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:18:16] PROBLEM - mw181 MediaWiki Rendering on mw181 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [04:18:17] PROBLEM - cp171 HTTPS on cp171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:18:21] PROBLEM - mw162 HTTPS on mw162 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:18:21] PROBLEM - mw192 HTTPS on mw192 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:18:22] PROBLEM - mw171 HTTPS on mw171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:18:22] PROBLEM - mw151 HTTPS on mw151 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:18:26] PROBLEM - mw201 MediaWiki Rendering on mw201 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [04:18:34] PROBLEM - mw172 MediaWiki Rendering on mw172 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.012 second response time [04:18:37] PROBLEM - mw183 HTTPS on mw183 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:18:37] PROBLEM - mw182 HTTPS on mw182 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:18:39] PROBLEM - db151 Current Load on db151 is CRITICAL: LOAD CRITICAL - total load average: 474.45, 230.39, 91.49 [04:18:40] PROBLEM - mw191 MediaWiki Rendering on mw191 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:18:40] PROBLEM - mw201 HTTPS on mw201 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [04:18:41] PROBLEM - mw153 HTTPS on mw153 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [04:18:44] PROBLEM - cp191 Varnish Backends on cp191 is CRITICAL: 11 backends are down. mw151 mw161 mw162 mw181 mw153 mw163 mw173 mw191 mw192 mw193 mw201 [04:18:45] PROBLEM - mw163 HTTPS on mw163 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [04:18:46] PROBLEM - cp161 HTTPS on cp161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [04:18:52] PROBLEM - mw181 HTTPS on mw181 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10001 milliseconds with 0 bytes received [04:18:55] PROBLEM - mw153 MediaWiki Rendering on mw153 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:18:55] PROBLEM - mw162 MediaWiki Rendering on mw162 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:18:57] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is CRITICAL: CRITICAL - NGINX Error Rate is 81% [04:19:02] PROBLEM - mw193 MediaWiki Rendering on mw193 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:19:06] PROBLEM - mw151 MediaWiki Rendering on mw151 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:19:11] !log [paladox@mwtask181] starting deploy of {'config': True} to all [04:19:18] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 11 backends are down. mw151 mw161 mw162 mw181 mw153 mw163 mw173 mw191 mw192 mw193 mw201 [04:19:20] !log [paladox@mwtask181] starting deploy of {'config': True, 'force': True} to all [04:19:42] PROBLEM - puppet181 Check unit status of listdomains_github_push on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit listdomains_github_push [04:20:16] RECOVERY - mw191 HTTPS on mw191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 3.825 second response time [04:20:17] RECOVERY - mw163 MediaWiki Rendering on mw163 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 5.303 second response time [04:20:21] RECOVERY - mw151 HTTPS on mw151 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.063 second response time [04:20:35] RECOVERY - mw153 HTTPS on mw153 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.064 second response time [04:20:38] RECOVERY - mw191 MediaWiki Rendering on mw191 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.178 second response time [04:20:45] RECOVERY - mw163 HTTPS on mw163 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.065 second response time [04:20:53] RECOVERY - mw162 MediaWiki Rendering on mw162 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.260 second response time [04:20:55] RECOVERY - mw153 MediaWiki Rendering on mw153 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 3.443 second response time [04:20:55] RECOVERY - mw181 HTTPS on mw181 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 4.092 second response time [04:20:57] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is WARNING: WARNING - NGINX Error Rate is 52% [04:21:01] RECOVERY - mw151 MediaWiki Rendering on mw151 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.222 second response time [04:21:21] RECOVERY - mw171 MediaWiki Rendering on mw171 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 5.870 second response time [04:21:30] RECOVERY - mw183 MediaWiki Rendering on mw183 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 7.434 second response time [04:21:43] RECOVERY - mw202 HTTPS on mw202 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.903 second response time [04:21:45] RECOVERY - mw202 MediaWiki Rendering on mw202 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.216 second response time [04:21:50] RECOVERY - mw173 MediaWiki Rendering on mw173 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.239 second response time [04:21:50] RECOVERY - cp201 HTTPS on cp201 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4273 bytes in 5.118 second response time [04:21:50] RECOVERY - mw203 HTTPS on mw203 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 4.823 second response time [04:21:51] !log [paladox@mwtask181] finished deploy of {'config': True, 'force': True} to all - SUCCESS in 150s [04:21:53] RECOVERY - mw152 HTTPS on mw152 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.064 second response time [04:21:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:21:57] RECOVERY - mw172 HTTPS on mw172 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.063 second response time [04:21:58] RECOVERY - mw161 MediaWiki Rendering on mw161 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.222 second response time [04:21:58] RECOVERY - cp191 HTTPS on cp191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4273 bytes in 0.075 second response time [04:22:00] RECOVERY - mw193 HTTPS on mw193 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.072 second response time [04:22:01] RECOVERY - db151 APT on db151 is OK: APT OK: 127 packages available for upgrade (0 critical updates). [04:22:04] what was it this time [04:22:04] RECOVERY - mw192 MediaWiki Rendering on mw192 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.227 second response time [04:22:07] RECOVERY - mw152 MediaWiki Rendering on mw152 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.202 second response time [04:22:14] RECOVERY - cp171 HTTPS on cp171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4273 bytes in 0.064 second response time [04:22:15] RECOVERY - mw161 HTTPS on mw161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.067 second response time [04:22:19] RECOVERY - mw173 HTTPS on mw173 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.052 second response time [04:22:22] RECOVERY - mw182 MediaWiki Rendering on mw182 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.229 second response time [04:22:22] RECOVERY - mw171 HTTPS on mw171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.064 second response time [04:22:23] RECOVERY - mw181 MediaWiki Rendering on mw181 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.237 second response time [04:22:28] RECOVERY - mw201 MediaWiki Rendering on mw201 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.222 second response time [04:22:28] RECOVERY - mw162 HTTPS on mw162 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.066 second response time [04:22:29] RECOVERY - mw192 HTTPS on mw192 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.070 second response time [04:22:35] RECOVERY - mw172 MediaWiki Rendering on mw172 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.210 second response time [04:22:36] RECOVERY - mw183 HTTPS on mw183 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.056 second response time [04:22:37] RECOVERY - mw182 HTTPS on mw182 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.074 second response time [04:22:43] RECOVERY - cp161 HTTPS on cp161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4328 bytes in 0.083 second response time [04:22:44] RECOVERY - cp191 Varnish Backends on cp191 is OK: All 31 backends are healthy [04:22:46] RECOVERY - mw201 HTTPS on mw201 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.073 second response time [04:22:57] RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 12% [04:23:07] RECOVERY - mw193 MediaWiki Rendering on mw193 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.221 second response time [04:23:18] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [04:23:43] RECOVERY - mw203 MediaWiki Rendering on mw203 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.220 second response time [04:23:58] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [04:25:42] RECOVERY - puppet181 Check unit status of listdomains_github_push on puppet181 is OK: OK: Status of the systemd unit listdomains_github_push [05:06:36] PROBLEM - db151 Current Load on db151 is WARNING: LOAD WARNING - total load average: 0.60, 0.70, 11.35 [05:08:36] RECOVERY - db151 Current Load on db151 is OK: LOAD OK - total load average: 0.70, 0.71, 10.06 [08:54:22] are the servers so screwed they're not even making logs anymore [08:55:24] grafana's also dead [08:57:42] PROBLEM - puppet181 Check unit status of listdomains_github_push on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit listdomains_github_push [09:15:42] RECOVERY - puppet181 Check unit status of listdomains_github_push on puppet181 is OK: OK: Status of the systemd unit listdomains_github_push [09:40:38] !log test [09:40:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:58:37] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.69, 17.63, 12.69 [12:15:23] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/e93f71b81cbc7f8a63db352c6fb84bf418e8b5f6 [12:15:23] 02ssl/03main 07WikiTideBot 03e93f71b Bot: Auto-update domain lists [13:12:37] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 8.73, 19.29, 23.56 [13:16:37] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 5.93, 12.46, 19.83 [13:30:30] !log [somerandomdeveloper@mwtask181] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php SemanticMediaWiki:setupStore --wiki=emiliabearwiki (END - exit=0) [13:30:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:47:56] PROBLEM - cp191 Disk Space on cp191 is WARNING: DISK WARNING - free space: / 49897MiB (10% inode=99%); [14:02:04] [02mw-config] 07dependabot[bot] created 03dependabot/composer/phpunit/phpunit-12.5.0 (+1 new commit) 13https://github.com/miraheze/mw-config/commit/d04554b48b7b [14:02:04] 02mw-config/03dependabot/composer/phpunit/phpunit-12.5.0 07dependabot[bot] 03d04554b Update phpunit/phpunit requirement from 12.4.1 to 12.5.0… [14:02:06] [02mw-config] 07dependabot[bot] added the label 'php' to pull request #6198 (Update phpunit/phpunit requirement from 12.4.1 to 12.5.0) 13https://github.com/miraheze/mw-config/pull/6198 [14:02:08] [02mw-config] 07dependabot[bot] added the label 'dependencies' to pull request #6198 (Update phpunit/phpunit requirement from 12.4.1 to 12.5.0) 13https://github.com/miraheze/mw-config/pull/6198 [14:02:10] [02mw-config] 07dependabot[bot] opened pull request #6198: Update phpunit/phpunit requirement from 12.4.1 to 12.5.0 (03main...03dependabot/composer/phpunit/phpunit-12.5.0) 13https://github.com/miraheze/mw-config/pull/6198 [14:02:12] [02mw-config] 07dependabot[bot] added the label 'dependencies' to pull request #6198 (Update phpunit/phpunit requirement from 12.4.1 to 12.5.0) 13https://github.com/miraheze/mw-config/pull/6198 [14:02:14] [02mw-config] 07dependabot[bot] added the label 'php' to pull request #6198 (Update phpunit/phpunit requirement from 12.4.1 to 12.5.0) 13https://github.com/miraheze/mw-config/pull/6198 [14:02:16] [02mw-config] 07dependabot[bot] commented on pull request #6196: Superseded by #6198. 13https://github.com/miraheze/mw-config/pull/6196#issuecomment-3617050356 [14:02:18] [02mw-config] 07dependabot[bot] closed pull request #6196: Update phpunit/phpunit requirement from 12.4.1 to 12.4.5 (03main...03dependabot/composer/phpunit/phpunit-12.4.5) 13https://github.com/miraheze/mw-config/pull/6196 [14:02:20] [02mw-config] 07dependabot[bot] 04deleted 03dependabot/composer/phpunit/phpunit-12.4.5 at 033a2a6ae 13https://api.github.com/repos/miraheze/mw-config/commit/3a2a6ae [14:02:53] miraheze/mw-config - dependabot[bot] the build passed. [14:22:37] PROBLEM - cp201 Disk Space on cp201 is WARNING: DISK WARNING - free space: / 49794MiB (10% inode=99%); [14:26:00] PROBLEM - cp171 Disk Space on cp171 is WARNING: DISK WARNING - free space: / 49792MiB (10% inode=99%); [15:27:54] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php CirrusSearch:UpdateSearchIndexConfig --wiki=superblox64wiki --startOver (END - exit=0) [15:27:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:28:23] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php CirrusSearch:ForceSearchIndex --wiki=superblox64wiki (END - exit=0) [15:28:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:41:49] [02puppet] 07AgentIsai pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/d6a4793a4dc263aebcc8f44200062f64e0e89f6d [17:41:49] 02puppet/03main 07Agent Isai 03d6a4793 Provision db192 [17:45:03] [02dns] 07AgentIsai pushed 1 new commit to 03main 13https://github.com/miraheze/dns/commit/5ebe7058ef11654eb8cab55c7d4137feb6dc1daf [17:45:03] 02dns/03main 07Agent Isai 035ebe705 Add db192 to dns [17:45:59] miraheze/dns - AgentIsai the build passed. [19:06:05] PROBLEM - db192 MariaDB on db192 is CRITICAL: Host '10.0.18.109' is not allowed to connect to this MariaDB server [19:06:55] PROBLEM - db192 MariaDB Connections on db192 is UNKNOWN: PHP Fatal error: Uncaught mysqli_sql_exception: Host '10.0.18.109' is not allowed to connect to this MariaDB server in /usr/lib/nagios/plugins/check_mysql_connections.php:66Stack trace:#0 /usr/lib/nagios/plugins/check_mysql_connections.php(66): mysqli_real_connect(Object(mysqli), 'db192.fsslc.wtn...', 'icinga', Object(SensitiveParameterValue), NULL, NULL, NULL, true)#1 {main} thro [19:06:55] /lib/nagios/plugins/check_mysql_connections.php on line 66Fatal error: Uncaught mysqli_sql_exception: Host '10.0.18.109' is not allowed to connect to this MariaDB server in /usr/lib/nagios/plugins/check_mysql_connections.php:66Stack trace:#0 /usr/lib/nagios/plugins/check_mysql_connections.php(66): mysqli_real_connect(Object(mysqli), 'db192.fsslc.wtn...', 'icinga', Object(SensitiveParameterValue), NULL, NULL, NULL, true)#1 {main} thrown [19:06:55] b/nagios/plugins/check_mysql_connections.php on line 66 [19:23:12] !log drop parsercache db on db161 [19:23:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:24:36] [02mw-config] 07paladox created 03paladox-patch-4 (+1 new commit) 13https://github.com/miraheze/mw-config/commit/4bc8bed89103 [19:24:36] 02mw-config/03paladox-patch-4 07paladox 034bc8bed Remove c2 (db161) from wgCreateWikiDatabaseClustersInactive [19:24:42] [02mw-config] 07paladox opened pull request #6199: Remove c2 (db161) from wgCreateWikiDatabaseClustersInactive (03main...03paladox-patch-4) 13https://github.com/miraheze/mw-config/pull/6199 [19:25:45] miraheze/mw-config - paladox the build passed. [19:36:38] [02ssl] 07pskyechology pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/8b887b5d227c55ac7a5ec582c04ec3560cab5398 [19:36:39] 02ssl/03main 07Skye 038b887b5 T14618: Add redirect for larnwiki [19:45:22] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/0eb106b825ce163a2c5841d7b612d83d9a332ca1 [19:45:22] 02ssl/03main 07WikiTideBot 030eb106b Bot: Auto-update domain lists [22:28:27] [02mw-config] 07SomeMWDev approved pull request #6199 13https://github.com/miraheze/mw-config/pull/6199#pullrequestreview-3546632216 [22:28:36] [02mw-config] 07SomeMWDev merged 07paladox's pull request #6199: Remove c2 (db161) from wgCreateWikiDatabaseClustersInactive (03main...03paladox-patch-4) 13https://github.com/miraheze/mw-config/pull/6199 [22:28:38] [02mw-config] 07SomeMWDev pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/a3f763b2c516f113681658ff643a171d9bfd99e1 [22:28:40] [02mw-config] 07SomeMWDev 04deleted 03paladox-patch-4 at 034bc8bed 13https://api.github.com/repos/miraheze/mw-config/commit/4bc8bed [22:28:42] 02mw-config/03main 07paladox 03a3f763b Remove c2 (db161) from wgCreateWikiDatabaseClustersInactive (#6199) [22:28:43] !log [somerandomdeveloper@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [22:28:44] !log [somerandomdeveloper@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 1s [22:28:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:28:48] !log [somerandomdeveloper@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [22:28:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:28:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:29:13] !log [somerandomdeveloper@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 25s [22:29:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:29:29] miraheze/mw-config - SomeMWDev the build passed.