[00:01:16] [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/d34c3dc59cc7741191b14764ce5adfec366f8c8c [00:01:16] 02statichelp/03main 07WikiTideBot 03d34c3dc Bot: Auto-update Tech namespace pages 2025-10-18 00:01:14 [00:11:44] RECOVERY - cp171 Disk Space on cp171 is OK: DISK OK - free space: / 60549MiB (13% inode=99%); [00:13:32] RECOVERY - cp191 Disk Space on cp191 is OK: DISK OK - free space: / 75169MiB (16% inode=99%); [00:13:54] RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 74044MiB (16% inode=99%); [00:55:26] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/dc0ddd91987c5ae1292b2bb7851d748ace130b88 [00:55:26] 02ssl/03main 07WikiTideBot 03dc0ddd9 Bot: Auto-update domain lists [10:08:34] PROBLEM - puppet181 Check unit status of listdomains_github_push on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit listdomains_github_push [10:10:17] [02ssl] 07SomeMWDev created 03T14414 (+1 new commit) 13https://github.com/miraheze/ssl/commit/4277bed519d1 [10:10:17] 02ssl/03T14414 07SomeRandomDeveloper 034277bed T14414: Add redirect for fortfortdroguewiki [10:10:28] [02ssl] 07SomeMWDev opened pull request #889: T14414: Add redirect for fortfortdroguewiki (03main...03T14414) 13https://github.com/miraheze/ssl/pull/889 [10:11:16] [02ssl] 07SomeMWDev merged pull request #889: T14414: Add redirect for fortfortdroguewiki (03main...03T14414) 13https://github.com/miraheze/ssl/pull/889 [10:11:18] [02ssl] 07SomeMWDev pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/0aab010ca932a4b877020617cd96ab32a5ed5cfc [10:11:20] [02ssl] 07SomeMWDev 04deleted 03T14414 at 034277bed 13https://api.github.com/repos/miraheze/ssl/commit/4277bed [10:11:22] 02ssl/03main 07SomeRandomDeveloper 030aab010 T14414: Add redirect for fortfortdroguewiki (#889) [10:16:34] RECOVERY - puppet181 Check unit status of listdomains_github_push on puppet181 is OK: OK: Status of the systemd unit listdomains_github_push [10:33:19] [02ssl] 07coderabbitai[bot] commented on pull request #889: A new redirect entry named `fortfortdroguewiki` is added to `redirects.yaml` with configuration mapping `fortfortdrogue.miraheze.org` to `fortfortdrogue.com` using Cloudflare certificate authentication and strict HSTS policy. […] 13https://github.com/miraheze/ssl/pull/889#issuecomment-3418192886 [10:40:40] PROBLEM - mattermost1 Puppet on mattermost1 is CRITICAL: CRITICAL: Puppet last ran 12 hours ago [10:44:45] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 18.21, 11.10, 5.56 [10:46:44] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 10.78, 10.68, 6.05 [10:48:44] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 3.40, 7.86, 5.58 [11:30:44] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 10.07, 11.99, 6.80 [11:32:44] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 3.30, 8.72, 6.23 [12:10:47] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 9.19, 10.56, 6.39 [12:12:44] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 2.58, 7.59, 5.79 [12:45:27] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/aaa8b98ce2ab72607402032266ccd48be9ae7e80 [12:45:27] 02ssl/03main 07WikiTideBot 03aaa8b98 Bot: Auto-update domain lists [13:12:44] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 32.17, 16.55, 8.40 [13:20:44] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 3.53, 9.21, 8.37 [13:38:44] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 20.36, 11.85, 7.89 [13:40:44] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 3.76, 8.33, 7.08 [14:55:26] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/870ad5d073aa961c6278bc8ef0bacec1a4464fd1 [14:55:27] 02ssl/03main 07WikiTideBot 03870ad5d Bot: Auto-update domain lists [15:04:47] PROBLEM - cp201 Disk Space on cp201 is WARNING: DISK WARNING - free space: / 49750MiB (10% inode=99%); [15:17:37] PROBLEM - cp171 Disk Space on cp171 is WARNING: DISK WARNING - free space: / 49785MiB (10% inode=99%); [15:55:26] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/34d6b504e0e056e891229338bbe6749c55a53bb8 [15:55:26] 02ssl/03main 07WikiTideBot 0334d6b50 Bot: Auto-update domain lists [16:01:05] PROBLEM - cp191 Disk Space on cp191 is WARNING: DISK WARNING - free space: / 49823MiB (10% inode=99%); [16:55:52] [02mw-config] 07pixDeVl commented on pull request #6145: This PR evidently did not add the prefix correctly (see https://discord.com/channels/407504499280707585/407537962553966603/1429140144455422133). It needs to somehow be added as a multi level prefix(see mh: and fandom: prefixes), which I’m fairly sure […] 13https://github.com/miraheze/mw-config/pull/6145#issuecomment-3418658281 [17:09:49] [02mw-config] 07SomeMWDev commented on pull request #6145: The PR hasn't been merged yet 13https://github.com/miraheze/mw-config/pull/6145#issuecomment-3418668976 [17:11:17] miraheze/mw-config - NotAracham the build passed. [17:13:59] [02mw-config] 07pskyechology merged 07NotAracham's pull request #6145: Adding IWD entry for Wiki.gg (07miraheze:03main...07NotAracham:03WikiGG_IWD) 13https://github.com/miraheze/mw-config/pull/6145 [17:14:00] [02mw-config] 07pskyechology pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/f50da034b6512d23ab3c596feda973a05a4f692e [17:14:00] 02mw-config/03main 07NotAracham 03f50da03 Adding IWD entry for Wiki.gg (#6145)… [17:15:01] miraheze/mw-config - pskyechology the build passed. [17:17:37] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 17.81, 9.46, 5.50 [17:19:33] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 4.19, 7.08, 5.09 [17:20:30] !log [skye@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [17:20:31] !log [skye@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 0s [17:20:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:20:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:35:10] !log [skye@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [17:35:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:35:34] !log [skye@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 24s [17:35:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:26:17] !log puppetserver ca --sign mattermost1.miraheze.org [18:26:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:26:39] RECOVERY - mattermost1 Puppet on mattermost1 is OK: OK: Puppet is currently enabled, last run 24 seconds ago with 0 failures [18:29:05] PROBLEM - mattermost1 HTTPS on mattermost1 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Connection timed out after 10003 milliseconds [18:35:10] PROBLEM - mon181 Puppet on mon181 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [18:42:00] [02puppet] 07RhinosF1 pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/577d2cb6a24d41bf0b02727daa586a4fea31b89f [18:42:01] 02puppet/03main 07RhinosF1 03577d2cb fix mattermost1 dns? [18:43:38] PROBLEM - eventgate181 Puppet on eventgate181 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 4 minutes ago with 0 failures [18:43:41] PROBLEM - mw192 Puppet on mw192 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 17 minutes ago with 0 failures [18:43:41] PROBLEM - mw171 Puppet on mw171 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 23 minutes ago with 0 failures [18:43:44] PROBLEM - reports171 Puppet on reports171 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 13 minutes ago with 0 failures [18:43:47] PROBLEM - changeprop201 Puppet on changeprop201 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 16 minutes ago with 0 failures [18:43:51] PROBLEM - mw163 Puppet on mw163 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 15 minutes ago with 0 failures [18:43:53] PROBLEM - cp191 Puppet on cp191 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 18 minutes ago with 0 failures [18:43:53] PROBLEM - mem151 Puppet on mem151 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 27 minutes ago with 0 failures [18:43:53] PROBLEM - os191 Puppet on os191 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 13 minutes ago with 0 failures [18:43:54] PROBLEM - mw162 Puppet on mw162 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 4 minutes ago with 0 failures [18:43:56] PROBLEM - swiftobject181 Puppet on swiftobject181 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 30 minutes ago with 0 failures [18:43:58] PROBLEM - os162 Puppet on os162 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 17 minutes ago with 0 failures [18:43:59] PROBLEM - swiftproxy171 Puppet on swiftproxy171 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 4 minutes ago with 0 failures [18:44:02] PROBLEM - ns1 Puppet on ns1 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 27 minutes ago with 0 failures [18:44:03] PROBLEM - cp171 Puppet on cp171 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 3 minutes ago with 0 failures [18:44:03] PROBLEM - mwtask181 Puppet on mwtask181 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 10 minutes ago with 0 failures [18:44:11] PROBLEM - cloud15 Puppet on cloud15 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 28 minutes ago with 0 failures [18:44:12] PROBLEM - swiftproxy161 Puppet on swiftproxy161 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 6 minutes ago with 0 failures [18:44:15] PROBLEM - rdb151 Puppet on rdb151 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 19 minutes ago with 0 failures [18:44:15] PROBLEM - mwtask171 Puppet on mwtask171 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 2 minutes ago with 0 failures [18:44:16] PROBLEM - graylog161 Puppet on graylog161 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 18 minutes ago with 0 failures [18:44:17] PROBLEM - mem191 Puppet on mem191 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 18 minutes ago with 0 failures [18:44:18] PROBLEM - mw201 Puppet on mw201 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 10 minutes ago with 0 failures [18:44:20] PROBLEM - cloud19 Puppet on cloud19 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 23 minutes ago with 0 failures [18:44:21] PROBLEM - mem161 Puppet on mem161 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 2 minutes ago with 0 failures [18:44:23] PROBLEM - cp161 Puppet on cp161 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 18 minutes ago with 0 failures [18:44:24] PROBLEM - cp201 Puppet on cp201 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 6 minutes ago with 0 failures [18:44:29] PROBLEM - prometheus151 Puppet on prometheus151 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 8 minutes ago with 0 failures [18:44:29] PROBLEM - db171 Puppet on db171 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 26 minutes ago with 0 failures [18:44:29] PROBLEM - mwtask151 Puppet on mwtask151 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 30 minutes ago with 0 failures [18:44:29] PROBLEM - mw173 Puppet on mw173 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 31 minutes ago with 0 failures [18:44:30] PROBLEM - mw203 Puppet on mw203 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 3 minutes ago with 0 failures [18:44:35] PROBLEM - swiftobject161 Puppet on swiftobject161 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 13 minutes ago with 0 failures [18:44:35] PROBLEM - matomo151 Puppet on matomo151 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 7 minutes ago with 0 failures [18:44:37] PROBLEM - db151 Puppet on db151 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 14 minutes ago with 0 failures [18:44:37] PROBLEM - ldap171 Puppet on ldap171 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 10 minutes ago with 0 failures [18:44:39] PROBLEM - kafka181 Puppet on kafka181 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 26 minutes ago with 0 failures [18:44:41] PROBLEM - mw151 Puppet on mw151 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 16 minutes ago with 0 failures [18:44:41] PROBLEM - llm191 Puppet on llm191 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 3 minutes ago with 0 failures [18:44:43] PROBLEM - swiftac171 Puppet on swiftac171 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 31 minutes ago with 0 failures [18:44:45] PROBLEM - mw193 Puppet on mw193 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 27 minutes ago with 0 failures [18:44:47] PROBLEM - os202 Puppet on os202 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 16 minutes ago with 0 failures [18:44:50] PROBLEM - cloud20 Puppet on cloud20 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 13 minutes ago with 0 failures [18:44:52] PROBLEM - os151 Puppet on os151 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 28 minutes ago with 0 failures [18:44:52] PROBLEM - cloud17 Puppet on cloud17 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 23 minutes ago with 0 failures [18:44:55] PROBLEM - bast181 Puppet on bast181 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 11 minutes ago with 0 failures [18:44:58] PROBLEM - swiftobject201 Puppet on swiftobject201 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 3 minutes ago with 0 failures [18:44:58] PROBLEM - phorge171 Puppet on phorge171 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 31 minutes ago with 0 failures [18:44:59] PROBLEM - mw172 Puppet on mw172 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 4 minutes ago with 0 failures [18:44:59] PROBLEM - db161 Puppet on db161 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 30 minutes ago with 0 failures [18:44:59] PROBLEM - mw182 Puppet on mw182 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 9 minutes ago with 0 failures [18:45:02] PROBLEM - swiftobject191 Puppet on swiftobject191 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 3 minutes ago with 0 failures [18:45:04] RECOVERY - mattermost1 HTTPS on mattermost1 is OK: HTTP OK: HTTP/2 200 - 505 bytes in 0.103 second response time [18:45:08] PROBLEM - mw202 Puppet on mw202 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 5 minutes ago with 0 failures [18:45:09] PROBLEM - swiftobject171 Puppet on swiftobject171 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 19 minutes ago with 0 failures [18:45:09] PROBLEM - mw181 Puppet on mw181 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 13 minutes ago with 0 failures [18:45:12] PROBLEM - mw152 Puppet on mw152 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 11 minutes ago with 0 failures [18:45:13] PROBLEM - cloud16 Puppet on cloud16 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 29 minutes ago with 0 failures [18:45:14] PROBLEM - mw153 Puppet on mw153 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 24 minutes ago with 0 failures [18:45:15] PROBLEM - ns2 Puppet on ns2 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 15 minutes ago with 0 failures [18:45:15] PROBLEM - mw161 Puppet on mw161 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 4 minutes ago with 0 failures [18:45:18] PROBLEM - db172 Puppet on db172 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 7 minutes ago with 0 failures [18:45:18] PROBLEM - bast161 Puppet on bast161 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 5 minutes ago with 0 failures [18:45:19] PROBLEM - db181 Puppet on db181 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 17 minutes ago with 0 failures [18:45:20] PROBLEM - bots171 Puppet on bots171 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 22 minutes ago with 0 failures [18:45:20] PROBLEM - cloud18 Puppet on cloud18 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 22 minutes ago with 0 failures [18:45:21] PROBLEM - mem201 Puppet on mem201 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 9 minutes ago with 0 failures [18:45:23] PROBLEM - mw183 Puppet on mw183 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 10 minutes ago with 0 failures [18:45:24] !log disabled puppet globally due to incident [18:45:27] PROBLEM - os161 Puppet on os161 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 13 minutes ago with 0 failures [18:45:29] PROBLEM - mwtask161 Puppet on mwtask161 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 3 minutes ago with 0 failures [18:45:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:45:32] PROBLEM - os201 Puppet on os201 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 29 minutes ago with 0 failures [18:45:33] PROBLEM - db182 Puppet on db182 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 25 minutes ago with 0 failures [18:45:34] PROBLEM - mw191 Puppet on mw191 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 24 minutes ago with 0 failures [18:45:42] PROBLEM - test151 Puppet on test151 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 12 minutes ago with 0 failures [18:45:42] PROBLEM - swiftobject151 Puppet on swiftobject151 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 14 minutes ago with 0 failures [18:47:42] RECOVERY - test151 Puppet on test151 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:03] RECOVERY - cp171 Puppet on cp171 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:10] RECOVERY - cloud15 Puppet on cloud15 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:15] RECOVERY - mwtask171 Puppet on mwtask171 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:16] RECOVERY - mem191 Puppet on mem191 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:16] RECOVERY - mw201 Puppet on mw201 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:17] RECOVERY - rdb151 Puppet on rdb151 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:17] RECOVERY - graylog161 Puppet on graylog161 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:20] RECOVERY - cloud19 Puppet on cloud19 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:22] RECOVERY - mem161 Puppet on mem161 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:23] RECOVERY - cp161 Puppet on cp161 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:24] RECOVERY - cp201 Puppet on cp201 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:28] RECOVERY - mwtask151 Puppet on mwtask151 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:28] RECOVERY - prometheus151 Puppet on prometheus151 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:29] RECOVERY - db171 Puppet on db171 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:29] RECOVERY - mw173 Puppet on mw173 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:32] RECOVERY - mw203 Puppet on mw203 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:34] RECOVERY - matomo151 Puppet on matomo151 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:35] RECOVERY - swiftobject161 Puppet on swiftobject161 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:38] RECOVERY - ldap171 Puppet on ldap171 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:39] RECOVERY - kafka181 Puppet on kafka181 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:39] RECOVERY - db151 Puppet on db151 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:41] RECOVERY - llm191 Puppet on llm191 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:43] RECOVERY - mw151 Puppet on mw151 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:44] RECOVERY - swiftac171 Puppet on swiftac171 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:45] RECOVERY - mw193 Puppet on mw193 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:47] RECOVERY - os202 Puppet on os202 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:50] RECOVERY - cloud20 Puppet on cloud20 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:51] RECOVERY - cloud17 Puppet on cloud17 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:52] RECOVERY - bast181 Puppet on bast181 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:54] RECOVERY - os151 Puppet on os151 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:57] RECOVERY - swiftobject201 Puppet on swiftobject201 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:58] RECOVERY - mw172 Puppet on mw172 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:59] RECOVERY - phorge171 Puppet on phorge171 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:59] RECOVERY - db161 Puppet on db161 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:50:59] RECOVERY - mw182 Puppet on mw182 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:02] RECOVERY - swiftobject191 Puppet on swiftobject191 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:08] RECOVERY - mw202 Puppet on mw202 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:09] RECOVERY - swiftobject171 Puppet on swiftobject171 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:09] RECOVERY - mw181 Puppet on mw181 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:12] RECOVERY - mw152 Puppet on mw152 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:14] RECOVERY - mw153 Puppet on mw153 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:17] RECOVERY - mw161 Puppet on mw161 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:17] RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:17] RECOVERY - cloud16 Puppet on cloud16 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:20] RECOVERY - bast161 Puppet on bast161 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:20] RECOVERY - bots171 Puppet on bots171 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:21] RECOVERY - db172 Puppet on db172 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:21] RECOVERY - mem201 Puppet on mem201 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:22] RECOVERY - mw183 Puppet on mw183 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:22] RECOVERY - db181 Puppet on db181 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:23] RECOVERY - cloud18 Puppet on cloud18 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:26] RECOVERY - os161 Puppet on os161 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:51:30] RECOVERY - mwtask161 Puppet on mwtask161 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:32] RECOVERY - os201 Puppet on os201 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:35] RECOVERY - db182 Puppet on db182 is OK: OK: Puppet is currently enabled, last run 50 seconds ago with 0 failures [18:51:36] RECOVERY - mw191 Puppet on mw191 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:38] RECOVERY - eventgate181 Puppet on eventgate181 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:41] RECOVERY - mw171 Puppet on mw171 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:42] RECOVERY - swiftobject151 Puppet on swiftobject151 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:44] RECOVERY - reports171 Puppet on reports171 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:45] RECOVERY - mw192 Puppet on mw192 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:47] RECOVERY - changeprop201 Puppet on changeprop201 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:51] RECOVERY - mw163 Puppet on mw163 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:52] RECOVERY - mem151 Puppet on mem151 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:52] RECOVERY - mw162 Puppet on mw162 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:53] RECOVERY - cp191 Puppet on cp191 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:51:53] RECOVERY - os191 Puppet on os191 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:51:56] RECOVERY - swiftobject181 Puppet on swiftobject181 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:52:02] RECOVERY - swiftproxy171 Puppet on swiftproxy171 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:52:03] RECOVERY - os162 Puppet on os162 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:52:03] RECOVERY - mwtask181 Puppet on mwtask181 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:52:06] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:52:10] RECOVERY - swiftproxy161 Puppet on swiftproxy161 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:58:41] PROBLEM - llm191 Puppet on llm191 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[openwebui_pip_install] [19:00:41] RECOVERY - llm191 Puppet on llm191 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [19:01:45] !log puppet node clean mattermost1.miraheze.org [19:01:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:20:25] !log sudo puppet node deactivate mattermost1.miraheze.org [19:20:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:23:33] RECOVERY - mon181 Puppet on mon181 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [19:27:12] !log puppet is now restored everywhere; incident over [19:27:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:45:36] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php MirahezeMagic:ResetWiki --wiki=loginwiki --dbname=starwarswiki --requester=Darkrai18 (END - exit=0) [20:45:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:50:00] !log Clarification for the above/below reset: steward action as requested by Raidarr in #tech-stewards. [20:50:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:50:07] (unncessary but might as well) [20:55:31] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/ee8bfafc5aaa9b0812e7b99d22bdf57de8f1c9a3 [20:55:31] 02ssl/03main 07WikiTideBot 03ee8bfaf Bot: Auto-update domain lists [21:18:18] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 7.08, 13.05, 9.30 [21:20:13] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 4.83, 11.77, 9.40 [21:24:04] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 2.70, 7.86, 8.36 [21:27:35] [02mw-config] 07pskyechology pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/cd5f0300cd958e22c9fa54379cbdb419332eb87e [21:27:35] 02mw-config/03main 07Skye 03cd5f030 Add 2 protection levels for ultimatelevelbuilderwiki… [21:28:37] miraheze/mw-config - pskyechology the build passed. [21:36:50] !log [skye@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [21:36:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:37:13] !log [skye@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 23s [21:37:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:17:59] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php MirahezeMagic:RenameDatabase --wiki=loginwiki --rename --old=upstormwiki --new=imagipediawiki --user=Skye (END - exit=0) [22:18:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:18:40] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php CreateWiki:SetContainersAccess --wiki=imagipediawiki (END - exit=0) [22:18:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:21:18] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php MirahezeMagic:ResetWiki --wiki=loginwiki --dbname=wingliscwiki --requester=Fewdark (END - exit=0) [22:21:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:23:14] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 7.80, 11.39, 7.66 [22:25:10] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 2.94, 8.47, 7.03 [22:27:46] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php MirahezeMagic:RenameDatabase --wiki=loginwiki --rename --old=kakwewwiki --new=nilsoulwiki --user=Skye (END - exit=0) [22:27:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:33:58] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php CreateWiki:SetContainersAccess --wiki=nilsoulwiki (END - exit=0) [22:34:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:39:42] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php MirahezeMagic:RenameDatabase --wiki=loginwiki --rename --old=brojogontsqwiki --new=brojogonwiki --user=Skye (END - exit=0) [22:39:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:40:40] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php CreateWiki:SetContainersAccess --wiki=brojogonwiki (END - exit=0) [22:40:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:09:30] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 19.19, 10.18, 5.81 [23:11:26] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 3.73, 7.30, 5.29 [23:13:37] RECOVERY - cp171 Disk Space on cp171 is OK: DISK OK - free space: / 72852MiB (16% inode=99%);