[00:09:57] <icinga-miraheze>	 PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 0.94, 1.83, 1.51
[00:11:51] <icinga-miraheze>	 RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 0.45, 1.37, 1.38
[00:12:00] <icinga-miraheze>	 RECOVERY - wiki.mlpwiki.net - reverse DNS on sslhost is OK: rDNS OK - wiki.mlpwiki.net reverse DNS resolves to cp10.miraheze.org
[00:44:18] <icinga-miraheze>	 PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 206s
[00:46:17] <icinga-miraheze>	 RECOVERY - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 0s
[01:16:16] <icinga-miraheze>	 PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 223s
[01:18:15] <icinga-miraheze>	 PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is WARNING: MariaDB replication - both - WARNING - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 155s
[01:20:15] <icinga-miraheze>	 RECOVERY - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 0s
[02:02:31] <icinga-miraheze>	 PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is WARNING: MariaDB replication - both - WARNING - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 129s
[02:04:31] <icinga-miraheze>	 PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 238s
[02:08:26] <icinga-miraheze>	 RECOVERY - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 0s
[02:23:08] <icinga-miraheze>	 PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 317s
[02:40:47] <icinga-miraheze>	 PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is WARNING: MariaDB replication - both - WARNING - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 106s
[02:42:45] <icinga-miraheze>	 RECOVERY - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 77s
[03:30:00] <icinga-miraheze>	 PROBLEM - wiki.mlpwiki.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mlpwiki.net reverse DNS resolves to 192-185-16-85.unifiedlayer.com
[04:14:17] <icinga-miraheze>	 PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 224s
[06:23:49] <icinga-miraheze>	 RECOVERY - phab2 APT on phab2 is OK: APT OK: 22 packages available for upgrade (0 critical updates).
[06:30:00] <icinga-miraheze>	 RECOVERY - wiki.mlpwiki.net - reverse DNS on sslhost is OK: rDNS OK - wiki.mlpwiki.net reverse DNS resolves to cp10.miraheze.org
[06:35:27] <icinga-miraheze>	 RECOVERY - mail2 APT on mail2 is OK: APT OK: 22 packages available for upgrade (0 critical updates).
[09:16:20] <RhinosF1>	 !log > UPDATE recentchanges SET rc_bot = 0 where rc_actor = 2 and rc_bot = 1; via rhinos@jobrunner3:~$ sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php --wiki=abysmalhardwarewiki for T7031
[09:16:23] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[11:52:16] <icinga-miraheze>	 RECOVERY - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 0s
[12:08:56] <Not-818c>	 [02MirahezeMagic] 07R4356th opened pull request 03#240: Remove Meta-specific messages - 13https://git.io/JYZjd
[12:10:02] <miraheze-github>	 miraheze/MirahezeMagic - R4356th the build passed.
[12:36:00] <icinga-miraheze>	 PROBLEM - wiki.mlpwiki.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mlpwiki.net reverse DNS resolves to 192-185-16-85.unifiedlayer.com
[13:05:01] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.01, 5.96, 4.54
[13:07:01] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.60, 5.34, 4.48
[14:23:45] <Not-818c>	 [02MirahezeMagic] 07dmehus commented on pull request 03#240: Remove Meta-specific messages - 13https://git.io/JYnsr
[14:24:18] <icinga-miraheze>	 PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is WARNING: MariaDB replication - both - WARNING - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 124s
[14:26:03] <dmehus>	 Reception123, or RhinosF1, jobrunner seems to be backlogged on Meta. Would either of you mind checking the type of jobs in the queue (for the data John asked about), then running jobs, being sure to specify a minium max memory allocation to the jobrunner?
[14:26:17] <icinga-miraheze>	 PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 235s
[14:26:49] <dmehus>	 Reception123, or RhinosF1, oh too late, wikis have already been created and CW AI scoring done so wouldn't give us the data we wanted
[14:30:54] <RhinosF1>	 Ack
[14:38:17] <icinga-miraheze>	 RECOVERY - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 0s
[14:46:58] <icinga-miraheze>	 PROBLEM - cp11 Current Load on cp11 is CRITICAL: CRITICAL - load average: 9.75, 9.88, 4.56
[14:47:22] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.52, 5.85, 4.64
[14:49:20] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.28, 6.06, 4.87
[14:49:29] * Reception123 hopes we can get those jobs stats soon
[14:52:56] <icinga-miraheze>	 RECOVERY - cp11 Current Load on cp11 is OK: OK - load average: 0.28, 3.29, 3.24
[14:59:02] <MH-Discord>	 <R4356th> Are translations of https://translatewiki.net/w/i.php?title=MediaWiki:Miraheze-dberr-again/en actually used anywhere?
[14:59:03] <MirahezeBot>	 [ MediaWiki:Miraheze-dberr-again/en - translatewiki.net ] - translatewiki.net
[15:04:04] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.99, 7.05, 5.97
[15:04:22] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.75, 6.70, 5.75
[15:06:03] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.15, 6.25, 5.81
[15:06:23] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.15, 6.13, 5.67
[15:18:07] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.25, 6.55, 6.04
[15:22:03] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.20, 6.56, 6.16
[15:23:38] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.58, 7.33, 6.37
[15:24:50] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 4.42, 4.30, 2.15
[15:25:34] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.44, 6.23, 6.08
[15:26:49] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 1.78, 3.25, 2.02
[15:35:15] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.81, 6.92, 6.37
[15:36:02] <icinga-miraheze>	 RECOVERY - wiki.mlpwiki.net - reverse DNS on sslhost is OK: rDNS OK - wiki.mlpwiki.net reverse DNS resolves to cp10.miraheze.org
[15:37:10] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.37, 6.69, 6.34
[15:45:27] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 9.63, 7.42, 6.51
[15:46:53] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.89, 6.69, 6.17
[15:47:23] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 5.81, 6.86, 6.43
[15:48:52] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.65, 6.67, 6.23
[15:49:19] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.86, 6.23, 6.25
[15:51:02] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.74, 7.68, 6.63
[15:53:02] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.31, 7.30, 6.61
[15:53:06] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.99, 7.30, 6.57
[15:55:06] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.66, 6.73, 6.45
[15:56:51] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.47, 6.83, 6.44
[15:57:11] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.12, 7.40, 6.76
[15:58:59] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.52, 6.99, 6.65
[15:59:03] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.17, 7.50, 6.92
[16:00:57] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.46, 6.60, 6.54
[16:01:06] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.91, 7.85, 7.09
[16:04:49] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 10.06, 8.35, 7.23
[16:05:02] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.25, 7.82, 7.22
[16:06:48] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.55, 7.53, 7.06
[16:06:58] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.73, 7.83, 7.30
[16:07:03] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.08, 7.99, 7.61
[16:10:42] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.32, 6.17, 6.62
[16:10:53] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.43, 6.71, 6.78
[16:12:51] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.60, 5.95, 6.66
[16:13:03] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.22, 5.52, 6.61
[16:17:01] <Not-818c>	 [02miraheze/dns] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JYnDi
[16:17:03] <Not-818c>	 [02miraheze/dns] 07paladox 03254af72 - Experiment with round robin dns (mediawiki-backends.m.org)
[16:17:04] <Not-818c>	 [02dns] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbQXl
[16:17:06] <Not-818c>	 [02dns] 07paladox opened pull request 03#201: Experiment with round robin dns (mediawiki-backends.m.org) - 13https://git.io/JYnD1
[17:33:53] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 12.04, 7.14, 5.08
[17:35:48] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.08, 6.85, 5.24
[17:37:46] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 3.31, 5.63, 4.98
[18:00:49] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 8.32, 3.37, 1.59
[18:01:36] <icinga-miraheze>	 PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.06, 18.00, 14.56
[18:02:49] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 2.04, 2.77, 1.59
[18:03:36] <icinga-miraheze>	 RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 15.38, 16.81, 14.55
[18:16:16] <icinga-miraheze>	 PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 216s
[18:21:02] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.81, 6.43, 5.93
[18:21:36] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.75, 6.60, 5.46
[18:22:02] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.70, 6.55, 5.57
[18:22:33] <RhinosF1>	 Afternoon Universal_Omega
[18:23:03] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.36, 5.91, 5.82
[18:23:34] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.38, 5.71, 5.28
[18:24:02] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.04, 6.10, 5.53
[18:30:15] <icinga-miraheze>	 PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is WARNING: MariaDB replication - both - WARNING - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 194s
[18:32:15] <icinga-miraheze>	 RECOVERY - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 0s
[18:35:02] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.92, 6.69, 5.92
[18:37:01] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 3.78, 5.85, 5.73
[18:44:05] <icinga-miraheze>	 PROBLEM - test3 MediaWiki Rendering on test3 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 324 bytes in 0.015 second response time
[18:45:38] <RhinosF1>	 paladox: is test3 you?
[18:45:49] <paladox>	 yes
[18:46:08] <RhinosF1>	 Okay
[18:46:17] * RhinosF1 goes back to his Sunday
[18:53:04] <Not-818c>	 [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JYc3g
[18:53:06] <Not-818c>	 [02miraheze/dns] 07paladox 03c4033bd - Update miraheze.org
[18:53:09] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.92, 5.96, 5.55
[18:53:38] <Not-818c>	 [02miraheze/dns] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JYc3r
[18:53:40] <Not-818c>	 [02miraheze/dns] 07paladox 035bd1cb7 - Update config
[18:53:41] <Not-818c>	 [02dns] 07paladox synchronize pull request 03#201: Experiment with round robin dns (mediawiki-backends.m.org) - 13https://git.io/JYnD1
[18:53:44] <Not-818c>	 [02dns] 07paladox closed pull request 03#201: Experiment with round robin dns (mediawiki-backends.m.org) - 13https://git.io/JYnD1
[18:53:46] <Not-818c>	 [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JYc3K
[18:53:47] <Not-818c>	 [02miraheze/dns] 07paladox 0347ae176 - Experiment with round robin dns (mediawiki-backends.m.org) (#201)
[18:53:49] <Not-818c>	 [02dns] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbQXl
[18:53:50] <Not-818c>	 [02miraheze/dns] 07paladox deleted branch 03paladox-patch-1
[18:57:04] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.83, 5.96, 5.68
[19:03:50] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.76, 7.95, 6.45
[19:04:48] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.27, 7.42, 6.23
[19:05:45] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.32, 7.54, 6.50
[19:06:44] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.38, 6.34, 5.99
[19:07:41] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 3.97, 6.40, 6.22
[19:16:20] <Not-818c>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://git.io/JYcnn
[19:16:22] <Not-818c>	 [02miraheze/puppet] 07paladox 035ed818e - Monitor test4 in blackbox_https_mediawiki_cp
[19:16:23] <Not-818c>	 [02puppet] 07paladox created branch 03paladox-patch-3 - 13https://git.io/vbiAS
[19:16:25] <Not-818c>	 [02puppet] 07paladox opened pull request 03#1723: Monitor test4 in blackbox_https_mediawiki_cp - 13https://git.io/JYcnc
[19:16:48] <Not-818c>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://git.io/JYcn4
[19:16:50] <Not-818c>	 [02miraheze/puppet] 07paladox 03fe36f05 - Update prometheus.yml.erb
[19:16:51] <Not-818c>	 [02puppet] 07paladox synchronize pull request 03#1723: Monitor test4 in blackbox_https_mediawiki_cp - 13https://git.io/JYcnc
[19:16:59] <Not-818c>	 [02puppet] 07paladox closed pull request 03#1723: Monitor test4 in blackbox_https_mediawiki_cp - 13https://git.io/JYcnc
[19:17:01] <Not-818c>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JYcn0
[19:17:02] <Not-818c>	 [02miraheze/puppet] 07paladox 032225c89 - Monitor test4 in blackbox_https_mediawiki_cp (#1723)
[19:17:04] <Not-818c>	 [02puppet] 07paladox deleted branch 03paladox-patch-3 - 13https://git.io/vbiAS
[19:17:05] <Not-818c>	 [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-3
[19:17:39] <Not-818c>	 [02miraheze/dns] 07paladox pushed 032 commits to 03master [+0/-0/±2] 13https://git.io/JYcng
[19:17:41] <Not-818c>	 [02miraheze/dns] 07paladox 03a13862f - Revert "Experiment with round robin dns (mediawiki-backends.m.org) (#201)"
[19:17:42] <Not-818c>	 [02miraheze/dns] 07paladox 033a1f68d - Revert "Update miraheze.org"
[19:18:31] <Not-818c>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JYcnK
[19:18:33] <Not-818c>	 [02miraheze/puppet] 07paladox 0306e936f - fix
[19:19:52] <RhinosF1>	 JohnLewis: ack, I'll get Reception123 to create a db then when needed.
[19:19:56] <icinga-miraheze>	 PROBLEM - cloud3 Puppet on cloud3 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[19:20:01] <JohnLewis>	 Reception123: https://phabricator.miraheze.org/H63 triggers on all tasks that don't have SRE as a project
[19:20:03] <MirahezeBot>	 [ ☿ Add MediaWiki (SRE) tag ] - phabricator.miraheze.org
[19:20:28] <icinga-miraheze>	 PROBLEM - test4 Puppet on test4 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[19:20:31] <RhinosF1>	 JohnLewis: that should be all not any
[19:20:38] <icinga-miraheze>	 PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[19:20:43] <icinga-miraheze>	 PROBLEM - bacula2 Puppet on bacula2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[19:20:54] <RhinosF1>	 Was broke yesterday
[19:21:33] <JohnLewis>	 Changed it
[19:21:49] <icinga-miraheze>	 PROBLEM - gluster3 Puppet on gluster3 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[19:23:24] <RhinosF1>	 JohnLewis: ty
[19:26:34] <Not-818c>	 [02miraheze/MWExt-SkinUpdateTracker] 07RhinosF1 pushed 031 commit to 03main [+3/-0/±0] 13https://git.io/JYcc9
[19:26:36] <Not-818c>	 [02miraheze/MWExt-SkinUpdateTracker] 07RhinosF1 035f571a1 - Initial commit
[19:26:37] <Not-818c>	 [02MWExt-SkinUpdateTracker] 07RhinosF1 created branch 03main - 13https://git.io/JYccH
[19:27:53] <RhinosF1>	 !log create https://github.com/miraheze/MWExt-SkinUpdateTracker for T7052
[19:27:54] <MirahezeBot>	 [ Page not found · GitHub · GitHub ] - github.com
[19:27:59] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[19:30:28] <icinga-miraheze>	 RECOVERY - test4 Puppet on test4 is OK: OK: Puppet is currently enabled, last run 32 seconds ago with 0 failures
[19:32:09] <Reception123>	 JohnLewis: oh whoops, thanks for fixing
[19:33:03] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.84, 7.06, 6.30
[19:34:31] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.43, 7.16, 6.34
[19:35:10] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.81, 7.33, 6.16
[19:36:06] <icinga-miraheze>	 RECOVERY - test3 MediaWiki Rendering on test3 is OK: HTTP OK: HTTP/1.1 200 OK - 20764 bytes in 0.232 second response time
[19:36:27] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.80, 6.79, 6.31
[19:37:07] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.92, 7.73, 6.46
[19:39:02] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.77, 6.71, 6.52
[19:42:58] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.30, 7.70, 6.81
[19:44:11] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.31, 6.82, 6.48
[19:44:55] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.93, 7.65, 6.89
[19:46:10] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.46, 6.41, 6.38
[19:46:52] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.69, 6.72, 6.66
[19:47:56] <icinga-miraheze>	 RECOVERY - cloud3 Puppet on cloud3 is OK: OK: Puppet is currently enabled, last run 31 seconds ago with 0 failures
[19:48:37] <icinga-miraheze>	 RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 57 seconds ago with 0 failures
[19:48:43] <icinga-miraheze>	 RECOVERY - bacula2 Puppet on bacula2 is OK: OK: Puppet is currently enabled, last run 16 seconds ago with 0 failures
[19:49:49] <icinga-miraheze>	 RECOVERY - gluster3 Puppet on gluster3 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures
[19:54:37] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.56, 6.92, 6.75
[19:55:09] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.30, 7.19, 6.62
[19:56:34] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.81, 6.58, 6.65
[19:57:09] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.56, 6.82, 6.55
[19:59:08] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.16, 6.46, 6.47
[20:05:16] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.32, 6.95, 6.40
[20:05:21] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.11, 6.96, 6.74
[20:07:12] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.83, 6.22, 6.20
[20:07:18] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.16, 6.33, 6.51
[20:08:43] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 9.65, 7.56, 6.68
[20:10:00] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.29, 7.07, 6.63
[20:11:13] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.60, 7.14, 6.82
[20:12:37] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.64, 7.41, 6.85
[20:13:58] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.25, 6.49, 6.53
[20:15:10] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.14, 6.71, 6.74
[20:16:29] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 6.02, 6.75, 6.72
[20:17:55] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.96, 7.00, 6.76
[20:21:52] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.02, 6.62, 6.65
[20:36:21] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.10, 6.79, 6.45
[20:38:17] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.01, 6.55, 6.41
[20:53:00] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 16.11, 14.14, 6.34
[20:53:36] <icinga-miraheze>	 PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 18.26, 21.93, 17.71
[20:55:36] <icinga-miraheze>	 RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 15.92, 19.59, 17.34
[20:59:24] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.81, 7.46, 6.86
[21:00:51] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 0.47, 3.23, 3.96
[21:01:22] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.92, 7.33, 6.87
[21:03:21] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 11.19, 8.83, 7.48
[21:04:21] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 10.93, 7.60, 6.31
[21:04:52] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 0.27, 1.62, 3.12
[21:05:12] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.34, 6.60, 6.05
[21:05:34] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.24, 7.58, 6.64
[21:06:21] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.71, 6.68, 6.13
[21:07:12] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 10.17, 7.73, 6.53
[21:09:10] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.66, 7.42, 6.55
[21:09:25] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.13, 6.41, 6.39
[21:13:07] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.51, 6.43, 6.37
[21:19:21] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 5.35, 6.88, 7.95
[21:22:21] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.09, 6.79, 6.15
[21:24:21] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.70, 5.97, 5.93
[21:32:49] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.98, 7.32, 6.63
[21:36:21] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.35, 7.88, 6.81
[21:36:54] <paladox>	 !log upgrade dbbackup2 to debian 10.9 and install security updates
[21:36:58] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[21:38:21] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.42, 7.30, 6.73
[21:38:37] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.97, 7.47, 7.05
[21:41:19] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.71, 7.47, 7.31
[21:41:29] <paladox>	 !log upgrade dbbackup1 to debian 10.9 and install security updates
[21:41:34] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[21:41:45] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.37, 7.72, 6.82
[21:42:21] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.71, 6.60, 6.57
[21:43:28] <icinga-miraheze>	 PROBLEM - dbbackup2 MariaDB c3 on dbbackup2 is CRITICAL: Can't connect to MySQL server on 'dbbackup2.miraheze.org' (111)
[21:43:28] <icinga-miraheze>	 PROBLEM - dbbackup2 Disk Space on dbbackup2 is CRITICAL: connect to address 107.161.22.249 port 5666: Connection refusedconnect to host 107.161.22.249 port 5666: Connection refused
[21:43:32] <icinga-miraheze>	 PROBLEM - dbbackup2 Puppet on dbbackup2 is CRITICAL: connect to address 107.161.22.249 port 5666: Connection refusedconnect to host 107.161.22.249 port 5666: Connection refused
[21:43:41] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.27, 7.45, 6.82
[21:44:16] <icinga-miraheze>	 PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is CRITICAL: connect to address 107.161.22.249 port 5666: Connection refusedconnect to host 107.161.22.249 port 5666: Connection refused
[21:44:17] <icinga-miraheze>	 PROBLEM - dbbackup2 APT on dbbackup2 is CRITICAL: connect to address 107.161.22.249 port 5666: Connection refusedconnect to host 107.161.22.249 port 5666: Connection refused
[21:44:30] <icinga-miraheze>	 PROBLEM - dbbackup2 SSH on dbbackup2 is CRITICAL: connect to address 107.161.22.249 and port 22: Connection refused
[21:44:32] <icinga-miraheze>	 PROBLEM - dbbackup2 NTP time on dbbackup2 is CRITICAL: connect to address 107.161.22.249 port 5666: Connection refusedconnect to host 107.161.22.249 port 5666: Connection refused
[21:44:59] <icinga-miraheze>	 PROBLEM - dbbackup2 PowerDNS Recursor on dbbackup2 is CRITICAL: connect to address 107.161.22.249 port 5666: Connection refusedconnect to host 107.161.22.249 port 5666: Connection refused
[21:45:16] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.69, 7.59, 7.45
[21:45:38] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.56, 6.28, 6.46
[21:47:46] <icinga-miraheze>	 PROBLEM - dbbackup1 Puppet on dbbackup1 is CRITICAL: connect to address 107.161.20.211 port 5666: Connection refusedconnect to host 107.161.20.211 port 5666: Connection refused
[21:47:48] <icinga-miraheze>	 PROBLEM - dbbackup1 MariaDB c4 on dbbackup1 is CRITICAL: Can't connect to MySQL server on 'dbbackup1.miraheze.org' (111)
[21:47:54] <icinga-miraheze>	 PROBLEM - dbbackup1 NTP time on dbbackup1 is CRITICAL: connect to address 107.161.20.211 port 5666: Connection refusedconnect to host 107.161.20.211 port 5666: Connection refused
[21:47:55] <icinga-miraheze>	 PROBLEM - dbbackup1 Disk Space on dbbackup1 is CRITICAL: connect to address 107.161.20.211 port 5666: Connection refusedconnect to host 107.161.20.211 port 5666: Connection refused
[21:48:27] <icinga-miraheze>	 PROBLEM - dbbackup1 MariaDB c2 on dbbackup1 is CRITICAL: Can't connect to MySQL server on 'dbbackup1.miraheze.org' (111)
[21:48:36] <icinga-miraheze>	 PROBLEM - dbbackup1 PowerDNS Recursor on dbbackup1 is CRITICAL: connect to address 107.161.20.211 port 5666: Connection refusedconnect to host 107.161.20.211 port 5666: Connection refused
[21:48:45] <icinga-miraheze>	 PROBLEM - dbbackup1 SSH on dbbackup1 is CRITICAL: connect to address 107.161.20.211 and port 22: Connection refused
[21:48:58] <icinga-miraheze>	 PROBLEM - dbbackup1 APT on dbbackup1 is CRITICAL: connect to address 107.161.20.211 port 5666: Connection refusedconnect to host 107.161.20.211 port 5666: Connection refused
[21:52:05] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.55, 7.39, 7.07
[21:54:01] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.31, 7.05, 6.99
[21:56:57] <icinga-miraheze>	 RECOVERY - dbbackup2 PowerDNS Recursor on dbbackup2 is OK: DNS OK: 0.990 seconds response time. miraheze.org returns 2607:5300:205:200::1c30,51.222.25.132
[21:59:37] <icinga-miraheze>	 RECOVERY - dbbackup2 Disk Space on dbbackup2 is OK: DISK OK - free space: / 426888 MB (67% inode=98%);
[21:59:40] <icinga-miraheze>	 RECOVERY - dbbackup2 Puppet on dbbackup2 is OK: OK: Puppet is currently enabled, last run 46 minutes ago with 0 failures
[22:00:08] <icinga-miraheze>	 RECOVERY - dbbackup2 APT on dbbackup2 is OK: APT OK: 0 packages available for upgrade (0 critical updates).
[22:00:36] <icinga-miraheze>	 RECOVERY - dbbackup2 SSH on dbbackup2 is OK: SSH OK - OpenSSH_7.9p1 Debian-10+deb10u2 (protocol 2.0)
[22:00:38] <icinga-miraheze>	 PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is UNKNOWN: NRPE: Unable to read output
[22:00:43] <icinga-miraheze>	 RECOVERY - dbbackup2 NTP time on dbbackup2 is OK: NTP OK: Offset -5.796551704e-05 secs
[22:01:01] <icinga-miraheze>	 RECOVERY - dbbackup1 APT on dbbackup1 is OK: APT OK: 0 packages available for upgrade (0 critical updates).
[22:01:11] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.60, 5.89, 6.63
[22:01:16] <icinga-miraheze>	 PROBLEM - dbbackup1 Check MariaDB Replication c4 on dbbackup1 is UNKNOWN: NRPE: Unable to read output
[22:01:16] <icinga-miraheze>	 PROBLEM - dbbackup1 Check MariaDB Replication c2 on dbbackup1 is UNKNOWN: NRPE: Unable to read output
[22:01:56] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.88, 7.08, 6.98
[22:02:56] <icinga-miraheze>	 PROBLEM - dbbackup1 MariaDB c4 on dbbackup1 is UNKNOWN: <Timeout exceeded.><Terminated by signal 9 (Killed).>
[22:03:59] <icinga-miraheze>	 RECOVERY - dbbackup1 Puppet on dbbackup1 is OK: OK: Puppet is currently enabled, last run 35 minutes ago with 0 failures
[22:04:25] <icinga-miraheze>	 PROBLEM - ping6 on dbbackup1 is CRITICAL: PING CRITICAL - Packet loss = 100%
[22:04:29] <icinga-miraheze>	 RECOVERY - dbbackup1 NTP time on dbbackup1 is OK: NTP OK: Offset 7.191300392e-05 secs
[22:04:29] <icinga-miraheze>	 RECOVERY - dbbackup1 Disk Space on dbbackup1 is OK: DISK OK - free space: / 103830 MB (16% inode=98%);
[22:04:42] <icinga-miraheze>	 RECOVERY - dbbackup1 PowerDNS Recursor on dbbackup1 is OK: DNS OK: 1.617 second response time. miraheze.org returns 2607:5300:205:200::1c30,51.222.25.132
[22:04:45] <icinga-miraheze>	 PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 1410s
[22:04:51] <icinga-miraheze>	 RECOVERY - dbbackup1 SSH on dbbackup1 is OK: SSH OK - OpenSSH_7.9p1 Debian-10+deb10u2 (protocol 2.0)
[22:05:13] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.55, 7.36, 6.66
[22:05:27] <icinga-miraheze>	 PROBLEM - dbbackup1 MariaDB c4 on dbbackup1 is CRITICAL: Can't connect to MySQL server on 'dbbackup1.miraheze.org' (111)
[22:05:31] <icinga-miraheze>	 RECOVERY - dbbackup2 MariaDB c3 on dbbackup2 is OK: Uptime: 289 Threads: 9 Questions: 11958 Slow queries: 3 Opens: 26 Flush tables: 1 Open tables: 20 Queries per second avg: 41.377
[22:07:09] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.19, 6.44, 6.40
[22:07:20] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.76, 7.40, 7.00
[22:07:49] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.55, 7.54, 7.39
[22:08:29] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.40, 7.22, 6.79
[22:09:18] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.29, 7.02, 6.92
[22:10:42] <icinga-miraheze>	 RECOVERY - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 0s
[22:11:05] <icinga-miraheze>	 RECOVERY - dbbackup1 MariaDB c2 on dbbackup1 is OK: Uptime: 317 Threads: 7 Questions: 10 Slow queries: 1 Opens: 16 Flush tables: 1 Open tables: 10 Queries per second avg: 0.031
[22:11:19] <icinga-miraheze>	 PROBLEM - dbbackup1 Check MariaDB Replication c2 on dbbackup1 is CRITICAL: MariaDB replication - both - CRITICAL - Slave IO State not correct, slave stopped or replication broken!
[22:11:40] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.28, 5.86, 6.73
[22:12:22] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.30, 6.05, 6.45
[22:13:27] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.00, 6.12, 6.61
[22:15:27] <icinga-miraheze>	 PROBLEM - dbbackup1 Check MariaDB Replication c4 on dbbackup1 is CRITICAL: MariaDB replication - both - CRITICAL - Slave IO State not correct, slave stopped or replication broken!
[22:15:31] <icinga-miraheze>	 RECOVERY - dbbackup1 MariaDB c4 on dbbackup1 is OK: Uptime: 282 Threads: 8 Questions: 34 Slow queries: 3 Opens: 16 Flush tables: 1 Open tables: 10 Queries per second avg: 0.120
[22:15:48] <SPF|Cloud>	 !log start c2 replication
[22:15:52] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[22:23:43] <SPF|Cloud>	 paladox: dbbackup1 doesn't get its v6 address from dhcp, is that normal for RamNode?
[22:25:15] <paladox>	 well dbbackup2 does it from interfaces.d/50-cloud-init.cfg
[22:25:27] <paladox>	 seems like its normal
[22:25:28] <SPF|Cloud>	 dbbackup1 has the same config though
[22:26:07] <paladox>	 looks like its normal as we have to define ipv6 for bacula2 too
[22:26:56] <SPF|Cloud>	 https://clientarea.ramnode.com/knowledgebase/4180/Static-IP-Configuration-Cloud.html The cloud images that we provide use cloud-init and DHCP to configure your instance network interfaces
[22:26:56] <MirahezeBot>	 [ Static IP Configuration (Cloud) - Knowledgebase - RamNode ] - clientarea.ramnode.com
[22:35:47] <SPF|Cloud>	 if you have no idea, I'll open a task
[22:36:08] <SPF|Cloud>	 tried renewing the lease using dhclient, but it doesn't look like RamNode offers an address?
[23:01:23] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.40, 6.53, 5.99
[23:03:25] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.43, 6.81, 6.13
[23:05:26] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.76, 6.88, 6.26
[23:05:52] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 10.52, 7.41, 6.07
[23:06:23] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.64, 7.38, 6.26
[23:07:50] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.39, 7.15, 6.16
[23:08:20] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.37, 6.32, 6.01
[23:09:25] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.27, 5.88, 6.03
[23:09:50] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.33, 6.39, 6.02