[00:16:12] <icinga-miraheze>	 RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 69114MiB (15% inode=99%);
[00:36:50] <miraheze-github>	 miraheze/mw-config - MacFan4000 the build passed.
[00:37:49] <MirahezeLSBot>	 !log [macfan@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all
[00:37:54] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[00:38:13] <MirahezeLSBot>	 !log [macfan@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 24s
[00:38:47] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[00:40:37] <MirahezeLSBot>	 !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php ManageWiki:ResetWikiCaches --wiki=metawiki (END - exit=0)
[00:41:38] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:44:00] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 26.90, 20.98, 13.43
[01:45:54] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 22.29, 21.64, 14.59
[01:47:48] <icinga-miraheze>	 RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 13.34, 18.75, 14.38
[01:58:06] <icinga-miraheze>	 PROBLEM - db192 MariaDB on db192 is UNKNOWN: <Timeout exceeded.><Terminated by signal 15 (Terminated).><Terminated with exit code 128 (0x80).>
[02:00:00] <icinga-miraheze>	 PROBLEM - db192 MariaDB on db192 is CRITICAL: Access denied for user 'icinga'@'10.0.18.109' (using password: YES)
[02:00:36] <miraheze-github>	 miraheze/mw-config - AgentIsai the build passed.
[02:01:07] <MirahezeLSBot>	 !log [agent@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all
[02:01:31] <MirahezeLSBot>	 !log [agent@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 23s
[02:03:19] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[02:03:19] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[02:03:46] <MirahezeLSBot>	 !log [agent@mwtask181] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php /srv/mediawiki/1.44/maintenance/sql.php --wiki=metawiki --wikidb mhglobal (END - exit=0)
[02:03:51] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[02:27:27] <icinga-miraheze>	 PROBLEM - puppet181 Check unit status of listdomains_github_push on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit listdomains_github_push
[02:29:31] <miraheze-github>	 miraheze/mw-config - AgentIsai the build passed.
[02:35:27] <icinga-miraheze>	 RECOVERY - puppet181 Check unit status of listdomains_github_push on puppet181 is OK: OK: Status of the systemd unit listdomains_github_push
[02:45:31] <icinga-miraheze>	 RECOVERY - db192 MariaDB on db192 is OK: Uptime: 2795 Threads: 41 Questions: 1696882 Slow queries: 0 Opens: 276 Open tables: 199 Queries per second avg: 607.113
[02:45:36] <icinga-miraheze>	 RECOVERY - db192 MariaDB Connections on db192 is OK: OK connection usage: 3.9%Current connections: 39
[02:49:15] <miraheze-github>	 miraheze/mw-config - AgentIsai the build passed.
[02:49:59] <icinga-miraheze>	 [Grafana] FIRING: The mediawiki JobQueue backlog is increasing by more than 100 jobs a minute over an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1765158560000&orgId=1&to=1765162199275
[02:54:59] <icinga-miraheze>	 [Grafana] RESOLVED: DatasourceError https://grafana.wikitide.net/d/GtxbP1Xnk?from=1765158560000&orgId=1&to=1765162220000
[03:29:59] <icinga-miraheze>	 [Grafana] FIRING: The mediawiki JobQueue backlog is increasing by more than 100 jobs a minute over an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1765160960000&orgId=1&to=1765164599339
[03:34:59] <icinga-miraheze>	 [Grafana] RESOLVED: DatasourceError https://grafana.wikitide.net/d/GtxbP1Xnk?from=1765161170000&orgId=1&to=1765164830000
[03:43:08] <icinga-miraheze>	 PROBLEM - bast161 Current Load on bast161 is CRITICAL: LOAD CRITICAL - total load average: 5.81, 3.29, 1.36
[03:45:19] <icinga-miraheze>	 PROBLEM - mwtask161 Puppet on mwtask161 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 2 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[git_pull_mathoid],Exec[git_pull_femiwiki-deploy-1.44],Exec[git_pull_3d2png]
[03:45:24] <icinga-miraheze>	 PROBLEM - phorge171 Puppet on phorge171 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_phorge-extensions]
[03:45:32] <icinga-miraheze>	 PROBLEM - mw161 Puppet on mw161 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_3d2png]
[03:45:33] <icinga-miraheze>	 PROBLEM - mwtask151 Puppet on mwtask151 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 2 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[git_pull_mathoid],Exec[git_pull_femiwiki-deploy-1.44],Exec[git_pull_3d2png]
[03:46:04] <icinga-miraheze>	 PROBLEM - mw172 Puppet on mw172 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_3d2png]
[03:46:04] <icinga-miraheze>	 PROBLEM - mwtask171 Puppet on mwtask171 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 3 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[git_pull_femiwiki-deploy-1.44],Exec[git_pull_3d2png]
[03:46:36] <icinga-miraheze>	 PROBLEM - mw203 Puppet on mw203 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 3 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[git_pull_femiwiki-deploy-1.44],Exec[git_pull_3d2png]
[03:47:04] <icinga-miraheze>	 RECOVERY - bast161 Current Load on bast161 is OK: LOAD OK - total load average: 0.22, 1.57, 1.08
[04:11:50] <icinga-miraheze>	 RECOVERY - mw161 Puppet on mw161 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[04:12:06] <icinga-miraheze>	 RECOVERY - mw172 Puppet on mw172 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[04:12:34] <icinga-miraheze>	 RECOVERY - mw203 Puppet on mw203 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[04:13:18] <icinga-miraheze>	 RECOVERY - mwtask161 Puppet on mwtask161 is OK: OK: Puppet is currently enabled, last run 38 seconds ago with 0 failures
[04:13:32] <icinga-miraheze>	 RECOVERY - mwtask171 Puppet on mwtask171 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[04:14:57] <icinga-miraheze>	 RECOVERY - phorge171 Puppet on phorge171 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[04:15:29] <icinga-miraheze>	 RECOVERY - mwtask151 Puppet on mwtask151 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[04:16:26] <miraheze-github>	 miraheze/mw-config - AgentIsai the build passed.
[04:37:17] <icinga-miraheze>	 PROBLEM - mwtask181 Puppet on mwtask181 is CRITICAL: CRITICAL: Puppet has 6 failures. Last run 4 minutes ago with 6 failures. Failed resources (up to 3 shown): Package[gnupg],Package[dirmngr],Package[djvulibre-bin],Package[poppler-utils]
[05:05:31] <icinga-miraheze>	 RECOVERY - mwtask181 Puppet on mwtask181 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[06:38:54] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 25.24, 16.52, 10.34
[06:48:54] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.73, 23.32, 17.13
[07:08:54] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.52, 23.99, 21.85
[07:08:56] <icinga-miraheze>	 [Grafana] FIRING: The mediawiki JobQueue backlog is increasing by more than 100 jobs a minute over an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1765174100000&orgId=1&to=1765177736682
[07:10:54] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.60, 23.98, 22.12
[07:12:54] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.28, 24.09, 22.38
[07:13:56] <icinga-miraheze>	 [Grafana] RESOLVED: MediaWiki Jobs are rapidly increasing https://grafana.wikitide.net/d/GtxbP1Xnk?from=1765174160000&orgId=1&to=1765177760000
[08:08:54] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 8.03, 18.99, 23.55
[08:12:54] <icinga-miraheze>	 RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 8.27, 12.48, 19.77
[12:11:57] <MacFanBot>	 [02DiscordNotifications] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/DiscordNotifications/commit/3bb1965929070ec249817d2f534d82338a1b1c9f
[12:11:57] <MacFanBot>	 02DiscordNotifications/03main 07translatewiki.net 033bb1965 Localisation updates from https://translatewiki.net.
[12:20:28] <miraheze-github>	 miraheze/DiscordNotifications - translatewiki the build passed.
[12:32:52] <icinga-miraheze>	 PROBLEM - cp201 Disk Space on cp201 is WARNING: DISK WARNING - free space: / 49786MiB (10% inode=99%);
[14:02:56] <MacFanBot>	 [02mw-config] 07dependabot[bot] created 03dependabot/composer/phpunit/phpunit-12.5.2 (+1 new commit) 13https://github.com/miraheze/mw-config/commit/100e8672d39f
[14:02:56] <MacFanBot>	 02mw-config/03dependabot/composer/phpunit/phpunit-12.5.2 07dependabot[bot] 03100e867 Update phpunit/phpunit requirement from 12.4.1 to 12.5.2…
[14:02:57] <MacFanBot>	 [02mw-config] 07dependabot[bot] added the label 'dependencies' to pull request #6203 (Update phpunit/phpunit requirement from 12.4.1 to 12.5.2) 13https://github.com/miraheze/mw-config/pull/6203
[14:02:59] <MacFanBot>	 [02mw-config] 07dependabot[bot] added the label 'php' to pull request #6203 (Update phpunit/phpunit requirement from 12.4.1 to 12.5.2) 13https://github.com/miraheze/mw-config/pull/6203
[14:03:01] <MacFanBot>	 [02mw-config] 07dependabot[bot] opened pull request #6203: Update phpunit/phpunit requirement from 12.4.1 to 12.5.2 (03main...03dependabot/composer/phpunit/phpunit-12.5.2) 13https://github.com/miraheze/mw-config/pull/6203
[14:03:03] <MacFanBot>	 [02mw-config] 07dependabot[bot] added the label 'dependencies' to pull request #6203 (Update phpunit/phpunit requirement from 12.4.1 to 12.5.2) 13https://github.com/miraheze/mw-config/pull/6203
[14:03:05] <MacFanBot>	 [02mw-config] 07dependabot[bot] added the label 'php' to pull request #6203 (Update phpunit/phpunit requirement from 12.4.1 to 12.5.2) 13https://github.com/miraheze/mw-config/pull/6203
[14:03:07] <MacFanBot>	 [02mw-config] 07dependabot[bot] closed pull request #6198: Update phpunit/phpunit requirement from 12.4.1 to 12.5.0 (03main...03dependabot/composer/phpunit/phpunit-12.5.0) 13https://github.com/miraheze/mw-config/pull/6198
[14:03:09] <MacFanBot>	 [02mw-config] 07dependabot[bot] commented on pull request #6198: Superseded by #6203. 13https://github.com/miraheze/mw-config/pull/6198#issuecomment-3627090740
[14:03:11] <MacFanBot>	 [02mw-config] 07dependabot[bot] 04deleted 03dependabot/composer/phpunit/phpunit-12.5.0 at 03d04554b 13https://api.github.com/repos/miraheze/mw-config/commit/d04554b
[14:04:04] <miraheze-github>	 miraheze/mw-config - dependabot[bot] the build passed.
[16:06:34] <icinga-miraheze>	 PROBLEM - cp191 Disk Space on cp191 is WARNING: DISK WARNING - free space: / 49855MiB (10% inode=99%);
[16:08:46] <icinga-miraheze>	 PROBLEM - cp171 Disk Space on cp171 is WARNING: DISK WARNING - free space: / 49812MiB (10% inode=99%);
[20:34:15] <MirahezeRelay>	 <paladox> !log upgrade bast181 to debian trxie
[20:34:20] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[20:45:24] <icinga-miraheze>	 PROBLEM - cp171 NTP time on cp171 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:25] <icinga-miraheze>	 PROBLEM - swiftobject171 NTP time on swiftobject171 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:31] <icinga-miraheze>	 PROBLEM - mw201 NTP time on mw201 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:33] <icinga-miraheze>	 PROBLEM - swiftobject151 NTP time on swiftobject151 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:35] <icinga-miraheze>	 PROBLEM - cloud17 NTP time on cloud17 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:36] <icinga-miraheze>	 PROBLEM - db181 NTP time on db181 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:36] <icinga-miraheze>	 PROBLEM - swiftproxy171 NTP time on swiftproxy171 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:36] <icinga-miraheze>	 PROBLEM - swiftobject161 NTP time on swiftobject161 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:37] <icinga-miraheze>	 PROBLEM - bots171 NTP time on bots171 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:39] <icinga-miraheze>	 PROBLEM - mw171 NTP time on mw171 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:39] <icinga-miraheze>	 PROBLEM - cloud19 NTP time on cloud19 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:40] <icinga-miraheze>	 PROBLEM - reports171 NTP time on reports171 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:40] <icinga-miraheze>	 PROBLEM - mon181 NTP time on mon181 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:42] <icinga-miraheze>	 PROBLEM - swiftobject191 NTP time on swiftobject191 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:44] <icinga-miraheze>	 PROBLEM - mw151 NTP time on mw151 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:45] <icinga-miraheze>	 PROBLEM - cp201 NTP time on cp201 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:48] <icinga-miraheze>	 PROBLEM - mw202 NTP time on mw202 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:48] <icinga-miraheze>	 PROBLEM - mwtask171 NTP time on mwtask171 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:48] <icinga-miraheze>	 PROBLEM - mem191 NTP time on mem191 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:48] <icinga-miraheze>	 PROBLEM - mw173 NTP time on mw173 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:49] <icinga-miraheze>	 PROBLEM - cloud18 NTP time on cloud18 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:52] <icinga-miraheze>	 PROBLEM - db182 NTP time on db182 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:53] <icinga-miraheze>	 PROBLEM - prometheus151 NTP time on prometheus151 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:54] <icinga-miraheze>	 PROBLEM - mw181 NTP time on mw181 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:54] <icinga-miraheze>	 PROBLEM - puppet181 NTP time on puppet181 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:55] <icinga-miraheze>	 PROBLEM - mem151 NTP time on mem151 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:56] <icinga-miraheze>	 PROBLEM - cloud15 NTP time on cloud15 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:57] <icinga-miraheze>	 PROBLEM - swiftobject181 NTP time on swiftobject181 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:57] <icinga-miraheze>	 PROBLEM - os201 NTP time on os201 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:57] <icinga-miraheze>	 PROBLEM - mw193 NTP time on mw193 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:45:59] <icinga-miraheze>	 PROBLEM - db151 NTP time on db151 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:00] <icinga-miraheze>	 PROBLEM - llm191 NTP time on llm191 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:03] <icinga-miraheze>	 PROBLEM - mw183 NTP time on mw183 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:03] <icinga-miraheze>	 PROBLEM - db172 NTP time on db172 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:04] <RhinosF1>	 Sigh
[20:46:08] <icinga-miraheze>	 PROBLEM - mw203 NTP time on mw203 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:09] <icinga-miraheze>	 PROBLEM - mw191 NTP time on mw191 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:09] <icinga-miraheze>	 PROBLEM - changeprop201 NTP time on changeprop201 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:09] <icinga-miraheze>	 PROBLEM - mw153 NTP time on mw153 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:11] <icinga-miraheze>	 PROBLEM - swiftproxy161 NTP time on swiftproxy161 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:12] <icinga-miraheze>	 PROBLEM - db171 NTP time on db171 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:21] <icinga-miraheze>	 PROBLEM - mw152 NTP time on mw152 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:23] <icinga-miraheze>	 PROBLEM - mwtask151 NTP time on mwtask151 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:24] <icinga-miraheze>	 PROBLEM - mwtask161 NTP time on mwtask161 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:25] <icinga-miraheze>	 PROBLEM - mw172 NTP time on mw172 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:28] <icinga-miraheze>	 PROBLEM - mw162 NTP time on mw162 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:31] <icinga-miraheze>	 PROBLEM - cloud16 NTP time on cloud16 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:31] <icinga-miraheze>	 PROBLEM - os162 NTP time on os162 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:32] <icinga-miraheze>	 PROBLEM - mem161 NTP time on mem161 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:37] <icinga-miraheze>	 PROBLEM - test151 NTP time on test151 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:39] <icinga-miraheze>	 PROBLEM - phorge171 NTP time on phorge171 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:39] <icinga-miraheze>	 PROBLEM - os151 NTP time on os151 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:39] <icinga-miraheze>	 PROBLEM - db161 NTP time on db161 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:40] <icinga-miraheze>	 PROBLEM - mw161 NTP time on mw161 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:40] <icinga-miraheze>	 PROBLEM - os161 NTP time on os161 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:40] <icinga-miraheze>	 PROBLEM - mw192 NTP time on mw192 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:41] <icinga-miraheze>	 PROBLEM - eventgate181 NTP time on eventgate181 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:43] <icinga-miraheze>	 PROBLEM - mem201 NTP time on mem201 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:54] <icinga-miraheze>	 PROBLEM - ldap171 NTP time on ldap171 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:56] <icinga-miraheze>	 PROBLEM - cloud20 NTP time on cloud20 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:46:57] <icinga-miraheze>	 PROBLEM - cp191 NTP time on cp191 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:47:03] <icinga-miraheze>	 PROBLEM - matomo151 NTP time on matomo151 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:47:04] <icinga-miraheze>	 PROBLEM - mw182 NTP time on mw182 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:47:04] <icinga-miraheze>	 PROBLEM - ns1 NTP time on ns1 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:47:05] <icinga-miraheze>	 PROBLEM - kafka181 NTP time on kafka181 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:47:06] <icinga-miraheze>	 PROBLEM - mw163 NTP time on mw163 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:47:09] <icinga-miraheze>	 PROBLEM - os191 NTP time on os191 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:47:10] <icinga-miraheze>	 PROBLEM - mwtask181 NTP time on mwtask181 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:47:11] <icinga-miraheze>	 PROBLEM - graylog161 NTP time on graylog161 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:47:14] <icinga-miraheze>	 PROBLEM - swiftobject201 NTP time on swiftobject201 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:47:18] <icinga-miraheze>	 PROBLEM - rdb151 NTP time on rdb151 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:47:18] <icinga-miraheze>	 PROBLEM - os202 NTP time on os202 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:47:19] <icinga-miraheze>	 PROBLEM - db192 NTP time on db192 is UNKNOWN: NTP UNKNOWN: Offset unknown|
[20:47:19] <icinga-miraheze>	 PROBLEM - cp161 NTP time on cp161 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:47:20] <icinga-miraheze>	 PROBLEM - db201 NTP time on db201 is UNKNOWN: NTP UNKNOWN: Offset unknown|
[20:47:21] <icinga-miraheze>	 PROBLEM - swiftac171 NTP time on swiftac171 is CRITICAL: NTP CRITICAL: Offset unknown|
[20:48:32] <MacFanBot>	 [02dns] 07paladox created 03paladox-patch-1 (+1 new commit) 13https://github.com/miraheze/dns/commit/479aca546944
[20:48:32] <MacFanBot>	 02dns/03paladox-patch-1 07paladox 03479aca5 Use bast161 ntp for now
[20:48:41] <MacFanBot>	 [02dns] 07paladox opened pull request #698: Use bast161 ntp for now (03main...03paladox-patch-1) 13https://github.com/miraheze/dns/pull/698
[20:48:46] <MacFanBot>	 [02dns] 07paladox merged pull request #698: Use bast161 ntp for now (03main...03paladox-patch-1) 13https://github.com/miraheze/dns/pull/698
[20:48:48] <MacFanBot>	 [02dns] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/dns/commit/9159897abe90d2968a82ca301f0cd5f7888429b7
[20:48:50] <MacFanBot>	 [02dns] 07paladox 04deleted 03paladox-patch-1 at 03479aca5 13https://api.github.com/repos/miraheze/dns/commit/479aca5
[20:48:51] <MacFanBot>	 02dns/03main 07paladox 039159897 Use bast161 ntp for now (#698)
[20:49:38] <miraheze-github>	 miraheze/dns - paladox the build passed.
[20:49:40] <miraheze-github_>	 miraheze/dns - paladox the build passed.
[20:49:42] <icinga-miraheze>	 RECOVERY - swiftobject191 NTP time on swiftobject191 is OK: NTP OK: Offset -2.014636993e-05 secs
[20:49:46] <icinga-miraheze>	 RECOVERY - mwtask171 NTP time on mwtask171 is OK: NTP OK: Offset -1.013278961e-05 secs
[20:49:46] <icinga-miraheze>	 RECOVERY - mem191 NTP time on mem191 is OK: NTP OK: Offset -1.192092896e-05 secs
[20:49:47] <icinga-miraheze>	 RECOVERY - cloud18 NTP time on cloud18 is OK: NTP OK: Offset -3.454089165e-05 secs
[20:49:50] <icinga-miraheze>	 RECOVERY - db182 NTP time on db182 is OK: NTP OK: Offset -3.805756569e-05 secs
[20:49:50] <icinga-miraheze>	 RECOVERY - prometheus151 NTP time on prometheus151 is OK: NTP OK: Offset -0.00021058321 secs
[20:49:54] <icinga-miraheze>	 RECOVERY - cloud15 NTP time on cloud15 is OK: NTP OK: Offset -0.0001602172852 secs
[20:49:55] <icinga-miraheze>	 RECOVERY - mem151 NTP time on mem151 is OK: NTP OK: Offset -0.0001799464226 secs
[20:49:55] <icinga-miraheze>	 RECOVERY - swiftobject181 NTP time on swiftobject181 is OK: NTP OK: Offset 0.0001150667667 secs
[20:49:57] <icinga-miraheze>	 RECOVERY - os201 NTP time on os201 is OK: NTP OK: Offset -5.000829697e-05 secs
[20:50:03] <icinga-miraheze>	 RECOVERY - db172 NTP time on db172 is OK: NTP OK: Offset 2.59578228e-05 secs
[20:50:07] <icinga-miraheze>	 RECOVERY - mw191 NTP time on mw191 is OK: NTP OK: Offset -6.198883057e-05 secs
[20:50:07] <icinga-miraheze>	 RECOVERY - mw203 NTP time on mw203 is OK: NTP OK: Offset 3.266334534e-05 secs
[20:50:09] <icinga-miraheze>	 RECOVERY - mw153 NTP time on mw153 is OK: NTP OK: Offset -0.0001480281353 secs
[20:50:21] <icinga-miraheze>	 RECOVERY - mw152 NTP time on mw152 is OK: NTP OK: Offset -2.726912498e-05 secs
[20:50:23] <icinga-miraheze>	 RECOVERY - mwtask151 NTP time on mwtask151 is OK: NTP OK: Offset -0.0001863539219 secs
[20:50:25] <icinga-miraheze>	 RECOVERY - mw172 NTP time on mw172 is OK: NTP OK: Offset 6.61611557e-06 secs
[20:50:28] <icinga-miraheze>	 RECOVERY - mw162 NTP time on mw162 is OK: NTP OK: Offset -3.233551979e-05 secs
[20:50:31] <icinga-miraheze>	 RECOVERY - cloud16 NTP time on cloud16 is OK: NTP OK: Offset 0.0001333653927 secs
[20:50:32] <icinga-miraheze>	 RECOVERY - mem161 NTP time on mem161 is OK: NTP OK: Offset 9.208917618e-06 secs
[20:50:39] <icinga-miraheze>	 RECOVERY - os151 NTP time on os151 is OK: NTP OK: Offset -0.0001602470875 secs
[20:50:40] <icinga-miraheze>	 RECOVERY - mw161 NTP time on mw161 is OK: NTP OK: Offset -1.430511475e-06 secs
[20:50:40] <icinga-miraheze>	 RECOVERY - os161 NTP time on os161 is OK: NTP OK: Offset -1.4513731e-05 secs
[20:50:40] <icinga-miraheze>	 RECOVERY - mw192 NTP time on mw192 is OK: NTP OK: Offset 6.118416786e-05 secs
[20:50:42] <icinga-miraheze>	 RECOVERY - eventgate181 NTP time on eventgate181 is OK: NTP OK: Offset -4.377961159e-05 secs
[20:50:54] <icinga-miraheze>	 RECOVERY - ldap171 NTP time on ldap171 is OK: NTP OK: Offset -4.24683094e-05 secs
[20:50:56] <icinga-miraheze>	 RECOVERY - cloud20 NTP time on cloud20 is OK: NTP OK: Offset -3.212690353e-05 secs
[20:50:57] <icinga-miraheze>	 RECOVERY - cp191 NTP time on cp191 is OK: NTP OK: Offset -3.072619438e-05 secs
[20:51:03] <icinga-miraheze>	 RECOVERY - matomo151 NTP time on matomo151 is OK: NTP OK: Offset -0.0001568198204 secs
[20:51:04] <icinga-miraheze>	 RECOVERY - mw182 NTP time on mw182 is OK: NTP OK: Offset -3.555417061e-05 secs
[20:51:04] <icinga-miraheze>	 RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset -0.0001140832901 secs
[20:51:06] <icinga-miraheze>	 RECOVERY - mw163 NTP time on mw163 is OK: NTP OK: Offset 0.000115275383 secs
[20:51:09] <icinga-miraheze>	 RECOVERY - os191 NTP time on os191 is OK: NTP OK: Offset -3.391504288e-05 secs
[20:51:15] <icinga-miraheze>	 RECOVERY - swiftobject201 NTP time on swiftobject201 is OK: NTP OK: Offset 9.655952454e-06 secs
[20:51:18] <icinga-miraheze>	 RECOVERY - rdb151 NTP time on rdb151 is OK: NTP OK: Offset -0.0002301931381 secs
[20:51:18] <icinga-miraheze>	 RECOVERY - os202 NTP time on os202 is OK: NTP OK: Offset -2.42292881e-05 secs
[20:51:20] <icinga-miraheze>	 RECOVERY - db201 NTP time on db201 is OK: NTP OK: Offset -7.227063179e-05 secs
[20:51:31] <icinga-miraheze>	 RECOVERY - swiftobject151 NTP time on swiftobject151 is OK: NTP OK: Offset -0.0001885592937 secs
[20:51:34] <icinga-miraheze>	 RECOVERY - swiftproxy171 NTP time on swiftproxy171 is OK: NTP OK: Offset 2.253055573e-05 secs
[20:51:36] <icinga-miraheze>	 RECOVERY - swiftobject161 NTP time on swiftobject161 is OK: NTP OK: Offset -3.24845314e-06 secs
[20:51:43] <icinga-miraheze>	 RECOVERY - mw151 NTP time on mw151 is OK: NTP OK: Offset -0.0001667737961 secs
[20:51:54] <icinga-miraheze>	 RECOVERY - mw181 NTP time on mw181 is OK: NTP OK: Offset -0.0001158416271 secs
[20:51:55] <icinga-miraheze>	 RECOVERY - mw193 NTP time on mw193 is OK: NTP OK: Offset -2.208352089e-05 secs
[20:51:56] <icinga-miraheze>	 RECOVERY - db151 NTP time on db151 is OK: NTP OK: Offset -0.0001694262028 secs
[20:52:09] <icinga-miraheze>	 RECOVERY - changeprop201 NTP time on changeprop201 is OK: NTP OK: Offset -1.87754631e-05 secs
[20:52:43] <icinga-miraheze>	 RECOVERY - graylog161 NTP time on graylog161 is OK: NTP OK: Offset 3.036856651e-05 secs
[20:53:19] <icinga-miraheze>	 RECOVERY - db192 NTP time on db192 is OK: NTP OK: Offset -0.0001287460327 secs
[20:53:24] <icinga-miraheze>	 RECOVERY - db181 NTP time on db181 is OK: NTP OK: Offset -7.009506226e-05 secs
[20:53:24] <icinga-miraheze>	 RECOVERY - cloud19 NTP time on cloud19 is OK: NTP OK: Offset -2.381205559e-05 secs
[20:53:31] <icinga-miraheze>	 RECOVERY - reports171 NTP time on reports171 is OK: NTP OK: Offset -1.341104507e-05 secs
[20:53:34] <icinga-miraheze>	 RECOVERY - cloud17 NTP time on cloud17 is OK: NTP OK: Offset -5.07235527e-05 secs
[20:53:45] <icinga-miraheze>	 RECOVERY - cp201 NTP time on cp201 is OK: NTP OK: Offset 5.811452866e-06 secs
[20:53:46] <icinga-miraheze>	 RECOVERY - mw202 NTP time on mw202 is OK: NTP OK: Offset 2.91466713e-05 secs
[20:53:47] <icinga-miraheze>	 RECOVERY - mw173 NTP time on mw173 is OK: NTP OK: Offset 4.017353058e-05 secs
[20:53:54] <icinga-miraheze>	 RECOVERY - puppet181 NTP time on puppet181 is OK: NTP OK: Offset -5.894899368e-05 secs
[20:53:56] <icinga-miraheze>	 RECOVERY - llm191 NTP time on llm191 is OK: NTP OK: Offset -6.297230721e-05 secs
[20:53:59] <icinga-miraheze>	 RECOVERY - mw183 NTP time on mw183 is OK: NTP OK: Offset -5.587935448e-05 secs
[20:54:10] <icinga-miraheze>	 RECOVERY - swiftproxy161 NTP time on swiftproxy161 is OK: NTP OK: Offset 3.135204315e-05 secs
[20:54:10] <icinga-miraheze>	 RECOVERY - db171 NTP time on db171 is OK: NTP OK: Offset -1.937150955e-06 secs
[20:54:24] <icinga-miraheze>	 RECOVERY - mwtask161 NTP time on mwtask161 is OK: NTP OK: Offset 1.519918442e-06 secs
[20:54:31] <icinga-miraheze>	 RECOVERY - os162 NTP time on os162 is OK: NTP OK: Offset 0.0003262162209 secs
[20:54:36] <icinga-miraheze>	 RECOVERY - cp161 NTP time on cp161 is OK: NTP OK: Offset 1.341104507e-05 secs
[20:54:37] <icinga-miraheze>	 RECOVERY - test151 NTP time on test151 is OK: NTP OK: Offset -0.0001825392246 secs
[20:54:39] <icinga-miraheze>	 RECOVERY - phorge171 NTP time on phorge171 is OK: NTP OK: Offset -4.318356514e-05 secs
[20:54:39] <icinga-miraheze>	 RECOVERY - db161 NTP time on db161 is OK: NTP OK: Offset 1.993775368e-05 secs
[20:54:43] <icinga-miraheze>	 RECOVERY - mwtask181 NTP time on mwtask181 is OK: NTP OK: Offset -5.972385406e-05 secs
[20:54:43] <icinga-miraheze>	 RECOVERY - mem201 NTP time on mem201 is OK: NTP OK: Offset 6.476044655e-05 secs
[20:54:46] <icinga-miraheze>	 RECOVERY - kafka181 NTP time on kafka181 is OK: NTP OK: Offset -0.0001138746738 secs
[20:54:52] <icinga-miraheze>	 RECOVERY - swiftac171 NTP time on swiftac171 is OK: NTP OK: Offset -7.01546669e-05 secs
[20:55:20] <icinga-miraheze>	 RECOVERY - mw171 NTP time on mw171 is OK: NTP OK: Offset 6.529688835e-05 secs
[20:55:23] <icinga-miraheze>	 RECOVERY - swiftobject171 NTP time on swiftobject171 is OK: NTP OK: Offset 6.884336472e-05 secs
[20:55:24] <icinga-miraheze>	 RECOVERY - cp171 NTP time on cp171 is OK: NTP OK: Offset 0.0003788769245 secs
[20:55:26] <MacFanBot>	 [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/9f6e5b3d4d2097a8d68c668c3394d5c289ca42a9
[20:55:26] <MacFanBot>	 02puppet/03main 07paladox 039f6e5b3 Fix recursion.yml.erb
[20:55:30] <icinga-miraheze>	 RECOVERY - mon181 NTP time on mon181 is OK: NTP OK: Offset -5.459785461e-05 secs
[20:55:31] <icinga-miraheze>	 RECOVERY - mw201 NTP time on mw201 is OK: NTP OK: Offset -4.991889e-05 secs
[20:55:33] <icinga-miraheze>	 RECOVERY - bots171 NTP time on bots171 is OK: NTP OK: Offset -5.900859833e-06 secs
[21:00:35] <MacFanBot>	 [02dns] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/dns/commit/3af92f01fca66b93e313eb14a3f5c7579b9f45aa
[21:00:35] <MacFanBot>	 02dns/03main 07paladox 033af92f0 Revert "Use bast161 ntp for now (#698)"…
[21:01:31] <miraheze-github>	 miraheze/dns - paladox the build passed.
[21:04:03] <MacFanBot>	 [02dns] 07paladox created 03paladox-patch-1 (+1 new commit) 13https://github.com/miraheze/dns/commit/d773043976ac
[21:04:03] <MacFanBot>	 02dns/03paladox-patch-1 07paladox 03d773043 Switch bastion to use bast181 temporarily
[21:04:07] <MacFanBot>	 [02dns] 07paladox opened pull request #699: Switch bastion to use bast181 temporarily (03main...03paladox-patch-1) 13https://github.com/miraheze/dns/pull/699
[21:04:13] <MacFanBot>	 [02dns] 07paladox merged pull request #699: Switch bastion to use bast181 temporarily (03main...03paladox-patch-1) 13https://github.com/miraheze/dns/pull/699
[21:04:15] <MacFanBot>	 [02dns] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/dns/commit/127b209e1071b937169501338318091a0c35d229
[21:04:17] <MacFanBot>	 [02dns] 07paladox 04deleted 03paladox-patch-1 at 03d773043 13https://api.github.com/repos/miraheze/dns/commit/d773043
[21:04:18] <MacFanBot>	 02dns/03main 07paladox 03127b209 Switch bastion to use bast181 temporarily (#699)
[21:04:58] <miraheze-github>	 miraheze/dns - paladox the build passed.
[21:05:15] <miraheze-github>	 miraheze/dns - paladox the build passed.
[21:17:49] <MirahezeRelay>	 <paladox> !log late logging but upgraded bast161 to trixie
[21:17:53] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[21:25:59] <MacFanBot>	 [02dns] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/dns/commit/700073eb3478bd5843651b8fd93e154ae72465e1
[21:25:59] <MacFanBot>	 02dns/03main 07paladox 03700073e Revert "Switch bastion to use bast181 temporarily (#699)"…
[21:26:56] <miraheze-github>	 miraheze/dns - paladox the build passed.
[23:31:12] <MacFanBot>	 [02mw-config] 07SomeMWDev created 03T14530 (+1 new commit) 13https://github.com/miraheze/mw-config/commit/fcfac2b5bb60
[23:31:12] <MacFanBot>	 02mw-config/03T14530 07SomeRandomDeveloper 03fcfac2b T14530: Enable vector night mode by default on zenithwiki
[23:31:35] <MacFanBot>	 [02mw-config] 07SomeMWDev opened pull request #6204: T14530: Enable vector night mode by default on zenithwiki (03main...03T14530) 13https://github.com/miraheze/mw-config/pull/6204
[23:32:31] <miraheze-github>	 miraheze/mw-config - SomeMWDev the build passed.
[23:32:46] <MacFanBot>	 [02mw-config] 07SomeMWDev merged pull request #6204: T14530: Enable vector night mode by default on zenithwiki (03main...03T14530) 13https://github.com/miraheze/mw-config/pull/6204
[23:32:46] <MacFanBot>	 [02mw-config] 07SomeMWDev pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/df1171596c0c59c9009bd4f078dd11a206bc8a3c
[23:32:46] <MacFanBot>	 02mw-config/03main 07SomeRandomDeveloper 03df11715 T14530: Enable vector night mode by default on zenithwiki (#6204)
[23:32:48] <MacFanBot>	 [02mw-config] 07SomeMWDev 04deleted 03T14530 at 03fcfac2b 13https://api.github.com/repos/miraheze/mw-config/commit/fcfac2b
[23:32:48] <MirahezeLSBot>	 !log [somerandomdeveloper@test151] starting deploy of {'pull': 'config', 'config': True} to test151
[23:32:49] <MirahezeLSBot>	 !log [somerandomdeveloper@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 0s
[23:32:52] <MirahezeLSBot>	 !log [somerandomdeveloper@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all
[23:32:53] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[23:32:57] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[23:33:02] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[23:33:15] <MirahezeLSBot>	 !log [somerandomdeveloper@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 23s
[23:33:19] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[23:33:42] <miraheze-github>	 miraheze/mw-config - SomeMWDev the build passed.
[23:34:22] <MacFanBot>	 [02mw-config] 07BlankEclair opened pull request #6205: Add editmoderatorprotected for damnationwiki (07miraheze:03main...07BlankEclair:03damnationwiki-editmoderatorprotected) 13https://github.com/miraheze/mw-config/pull/6205
[23:34:58] <MacFanBot>	 [02mw-config] 07BlankEclair merged pull request #6205: Add editmoderatorprotected for damnationwiki (07miraheze:03main...07BlankEclair:03damnationwiki-editmoderatorprotected) 13https://github.com/miraheze/mw-config/pull/6205
[23:34:58] <MacFanBot>	 [02mw-config] 07BlankEclair pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/018222daf3b49cddc717f481993fbe71daf6809c
[23:34:59] <MacFanBot>	 02mw-config/03main 07Claire Elaina 03018222d Add editmoderatorprotected for damnationwiki (#6205)…
[23:35:00] <MirahezeLSBot>	 !log [blankeclair@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all
[23:35:05] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[23:35:18] <miraheze-github>	 miraheze/mw-config - BlankEclair the build passed.
[23:35:23] <MirahezeLSBot>	 !log [blankeclair@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 22s
[23:35:27] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[23:36:05] <miraheze-github>	 miraheze/mw-config - BlankEclair the build passed.