[00:03:36] <icinga-miraheze>	 PROBLEM - graylog2 Puppet on graylog2 is CRITICAL: CRITICAL: Puppet has 6 failures. Last run 3 minutes ago with 6 failures. Failed resources (up to 3 shown)
[00:03:57] <icinga-miraheze>	 PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 25.02, 18.71, 12.98
[00:04:32] <icinga-miraheze>	 PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 1.42, 5.88, 7.52
[00:04:47] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.52, 6.25, 4.22
[00:05:47] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.50, 6.32, 4.27
[00:06:33] <icinga-miraheze>	 RECOVERY - db11 Current Load on db11 is OK: OK - load average: 1.02, 4.21, 6.70
[00:06:52] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.03, 6.12, 4.43
[00:07:47] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.35, 6.15, 4.48
[00:09:44] <Not-80da>	 [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JsGw3
[00:09:45] <Not-80da>	 [02miraheze/mw-config] 07Universal-Omega 03306cb25 - Set DPL3 maxResultCount to 999
[00:09:57] <icinga-miraheze>	 PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 17.43, 22.60, 16.95
[00:10:43] <miraheze-github>	 miraheze/mw-config - Universal-Omega the build passed.
[00:11:56] <icinga-miraheze>	 RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 9.90, 18.36, 16.10
[00:31:22] <icinga-miraheze>	 PROBLEM - miraheze.ga - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for miraheze.ga could not be found
[00:31:24] <icinga-miraheze>	 PROBLEM - wiki.worldsofweary.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.worldsofweary.com could not be found
[00:31:32] <icinga-miraheze>	 PROBLEM - wiki.caker18.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.caker18.com could not be found
[00:31:38] <icinga-miraheze>	 RECOVERY - graylog2 Puppet on graylog2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[00:31:47] <icinga-miraheze>	 PROBLEM - wiki.joust.ro - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.joust.ro could not be found
[00:38:12] <icinga-miraheze>	 RECOVERY - miraheze.ga - reverse DNS on sslhost is OK: rDNS OK - miraheze.ga reverse DNS resolves to cp11.miraheze.org
[00:38:13] <icinga-miraheze>	 RECOVERY - wiki.worldsofweary.com - reverse DNS on sslhost is OK: rDNS OK - wiki.worldsofweary.com reverse DNS resolves to cp11.miraheze.org
[00:38:31] <icinga-miraheze>	 RECOVERY - wiki.joust.ro - reverse DNS on sslhost is OK: rDNS OK - wiki.joust.ro reverse DNS resolves to cp11.miraheze.org
[00:38:31] <icinga-miraheze>	 RECOVERY - wiki.caker18.com - reverse DNS on sslhost is OK: rDNS OK - wiki.caker18.com reverse DNS resolves to cp11.miraheze.org
[00:42:38] <icinga-miraheze>	 PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 2.10, 2.25, 1.55
[00:44:38] <icinga-miraheze>	 PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.09, 1.79, 1.46
[00:46:38] <icinga-miraheze>	 RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 0.95, 1.53, 1.41
[00:56:48] <Not-80da>	 [02dns] 07Southparkfan commented on commit 0366bd191fe8913d791b91ed00d2a799279549df4d - 13https://git.io/JsGPt
[00:57:00] <SPF|Cloud>	 ^ paladox (I'll depool later this day)
[00:58:02] <SPF|Cloud>	 repool*
[00:58:14] <SPF|Cloud>	 .in 10h Repool cp10
[00:58:14] <MirahezeBot>	 SPF|Cloud: Okay, will remind at 2021-05-15 - 10:58:13UTC
[00:58:23] <SPF|Cloud>	 night
[01:21:54] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[01:35:31] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[02:04:39] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[02:18:15] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[02:25:49] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 11.96, 7.22, 4.77
[02:26:24] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.35, 6.57, 4.68
[02:26:54] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 12.17, 7.84, 5.30
[02:27:08] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.02, 6.02, 4.78
[02:28:11] <icinga-miraheze>	 PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:800:1bbd::4/cpweb, 2607:5300:205:200::1c30/cpweb
[02:28:15] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 3 datacenters are down: 2400:6180:0:d0::403:f001/cpweb, 2001:41d0:800:1bbd::4/cpweb, 51.222.25.132/cpweb
[02:28:27] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.91, 6.89, 5.07
[02:28:29] <icinga-miraheze>	 PROBLEM - cp10 Stunnel Http for mw11 on cp10 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[02:28:40] <icinga-miraheze>	 PROBLEM - wiki.consid.vn - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.consid.vn could not be found
[02:28:40] <icinga-miraheze>	 PROBLEM - guia.cineastas.pt - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for guia.cineastas.pt could not be found
[02:28:41] <icinga-miraheze>	 PROBLEM - wiki.sdbx.dev - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.sdbx.dev could not be found
[02:29:08] <icinga-miraheze>	 PROBLEM - mw11 MediaWiki Rendering on mw11 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[02:29:09] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.97, 7.87, 5.64
[02:29:21] <icinga-miraheze>	 PROBLEM - cp12 Stunnel Http for mw11 on cp12 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[02:29:47] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 3.68, 6.48, 5.15
[02:30:07] <icinga-miraheze>	 PROBLEM - isv.orain.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for isv.orain.org could not be found
[02:30:08] <icinga-miraheze>	 PROBLEM - www.petrawiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.petrawiki.org could not be found
[02:30:12] <icinga-miraheze>	 PROBLEM - sims.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for sims.miraheze.org could not be found
[02:30:17] <icinga-miraheze>	 PROBLEM - cp3 Stunnel Http for mw11 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[02:30:18] <icinga-miraheze>	 PROBLEM - en.famepedia.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for en.famepedia.org could not be found
[02:30:26] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 3.92, 5.75, 4.86
[02:30:34] <icinga-miraheze>	 RECOVERY - cp10 Stunnel Http for mw11 on cp10 is OK: HTTP OK: HTTP/1.1 200 OK - 15285 bytes in 7.468 second response time
[02:31:04] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.10, 7.84, 5.91
[02:31:07] <icinga-miraheze>	 RECOVERY - mw11 MediaWiki Rendering on mw11 is OK: HTTP OK: HTTP/1.1 200 OK - 20903 bytes in 0.983 second response time
[02:31:18] <icinga-miraheze>	 RECOVERY - cp12 Stunnel Http for mw11 on cp12 is OK: HTTP OK: HTTP/1.1 200 OK - 15285 bytes in 0.546 second response time
[02:32:07] <icinga-miraheze>	 RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online
[02:32:13] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[02:32:19] <icinga-miraheze>	 RECOVERY - cp3 Stunnel Http for mw11 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15285 bytes in 1.223 second response time
[02:33:03] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.53, 8.19, 6.26
[02:33:45] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[02:33:47] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 12.25, 8.67, 6.27
[02:35:32] <icinga-miraheze>	 RECOVERY - guia.cineastas.pt - reverse DNS on sslhost is OK: rDNS OK - guia.cineastas.pt reverse DNS resolves to cp11.miraheze.org
[02:35:33] <icinga-miraheze>	 RECOVERY - wiki.consid.vn - reverse DNS on sslhost is OK: rDNS OK - wiki.consid.vn reverse DNS resolves to cp11.miraheze.org
[02:35:34] <icinga-miraheze>	 RECOVERY - wiki.sdbx.dev - reverse DNS on sslhost is OK: rDNS OK - wiki.sdbx.dev reverse DNS resolves to cp11.miraheze.org
[02:37:02] <icinga-miraheze>	 RECOVERY - isv.orain.org - reverse DNS on sslhost is OK: rDNS OK - isv.orain.org reverse DNS resolves to cp11.miraheze.org
[02:37:04] <icinga-miraheze>	 RECOVERY - www.petrawiki.org - reverse DNS on sslhost is OK: rDNS OK - www.petrawiki.org reverse DNS resolves to cp11.miraheze.org
[02:37:07] <icinga-miraheze>	 RECOVERY - sims.miraheze.org - reverse DNS on sslhost is OK: rDNS OK - sims.miraheze.org reverse DNS resolves to cp11.miraheze.org
[02:37:17] <icinga-miraheze>	 RECOVERY - en.famepedia.org - reverse DNS on sslhost is OK: rDNS OK - en.famepedia.org reverse DNS resolves to cp11.miraheze.org
[02:37:47] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.35, 7.32, 6.30
[02:39:04] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 5.53, 7.11, 6.50
[02:39:47] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 3.80, 6.30, 6.06
[02:40:49] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 4.92, 7.72, 7.38
[02:41:03] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.18, 6.09, 6.20
[02:44:50] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.27, 5.89, 6.70
[02:47:22] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[02:55:02] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.96, 6.95, 5.86
[02:56:17] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[02:57:02] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.42, 7.41, 6.19
[02:59:02] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 3.51, 5.88, 5.77
[03:04:23] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.21, 6.36, 4.91
[03:05:02] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.11, 6.35, 5.89
[03:06:24] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 3.54, 5.17, 4.65
[03:07:02] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 3.86, 5.38, 5.59
[03:09:54] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[03:18:36] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[03:32:12] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[04:08:05] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[04:42:05] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[04:50:47] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[04:57:35] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[05:19:55] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[05:33:31] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[05:33:51] <Not-80da>	 [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JsZCc
[05:33:53] <Not-80da>	 [02miraheze/CreateWiki] 07Universal-Omega 03baf6bf3 - Fix edit-language style
[05:33:54] <Not-80da>	 [02CreateWiki] 07Universal-Omega created branch 03Universal-Omega-patch-2 - 13https://git.io/vpJTL
[05:33:56] <Not-80da>	 [02CreateWiki] 07Universal-Omega opened pull request 03#217: Fix edit-language style - 13https://git.io/JsZCW
[05:35:00] <miraheze-github>	 miraheze/CreateWiki - Universal-Omega the build passed.
[05:41:55] <Not-80da>	 [02dns] 07revi opened pull request 03#204: DNS purge 2021-05 - 13https://git.io/JsZlU
[05:42:15] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[05:56:15] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[06:12:15] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[06:34:18] <icinga-miraheze>	 PROBLEM - cp11 Current Load on cp11 is CRITICAL: CRITICAL - load average: 8.18, 10.84, 5.17
[06:40:18] <icinga-miraheze>	 PROBLEM - cp11 Current Load on cp11 is WARNING: WARNING - load average: 0.28, 3.50, 3.62
[06:42:18] <icinga-miraheze>	 RECOVERY - cp11 Current Load on cp11 is OK: OK - load average: 0.87, 2.56, 3.25
[07:06:41] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[07:43:15] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[07:56:51] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[08:12:19] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[08:25:55] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[08:34:38] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[09:08:40] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[09:17:22] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[09:26:18] <icinga-miraheze>	 PROBLEM - cp11 Current Load on cp11 is CRITICAL: CRITICAL - load average: 5.68, 3.86, 2.29
[09:28:18] <icinga-miraheze>	 RECOVERY - cp11 Current Load on cp11 is OK: OK - load average: 1.62, 2.90, 2.12
[09:30:58] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[09:39:40] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[10:05:07] <Not-80da>	 [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JsnIh
[10:05:09] <Not-80da>	 [02miraheze/mw-config] 07Universal-Omega 03646008c - Update LocalSettings.php
[10:05:10] <Not-80da>	 [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/vbvb3
[10:05:41] <Not-80da>	 [02mw-config] 07Universal-Omega opened pull request 03#3906: T7315: Move GlobalWatchlist to loginwiki - 13https://git.io/JsnLU
[10:06:26] <Not-80da>	 [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JsnLn
[10:06:27] <Not-80da>	 [02miraheze/mw-config] 07Universal-Omega 03c22031b - Update LocalSettings.php
[10:06:29] <Not-80da>	 [02mw-config] 07Universal-Omega synchronize pull request 03#3906: T7315: Move GlobalWatchlist to loginwiki - 13https://git.io/JsnLU
[10:06:42] <miraheze-github>	 miraheze/mw-config - Universal-Omega the build passed.
[10:06:55] <Not-80da>	 [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JsnL4
[10:06:56] <Not-80da>	 [02miraheze/mw-config] 07Universal-Omega 03173b6a9 - Update LocalSettings.php
[10:06:58] <Not-80da>	 [02mw-config] 07Universal-Omega synchronize pull request 03#3906: T7315: Move GlobalWatchlist to loginwiki - 13https://git.io/JsnLU
[10:07:27] <miraheze-github>	 miraheze/mw-config - Universal-Omega the build passed.
[10:07:58] <miraheze-github>	 miraheze/mw-config - Universal-Omega the build passed.
[10:11:23] <icinga-miraheze>	 PROBLEM - wiki.starship.digital - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.starship.digital could not be found
[10:11:23] <icinga-miraheze>	 PROBLEM - wiki.azimuth.navy - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.azimuth.navy could not be found
[10:13:40] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[10:18:16] <icinga-miraheze>	 RECOVERY - wiki.starship.digital - reverse DNS on sslhost is OK: rDNS OK - wiki.starship.digital reverse DNS resolves to cp11.miraheze.org
[10:18:16] <icinga-miraheze>	 RECOVERY - wiki.azimuth.navy - reverse DNS on sslhost is OK: rDNS OK - wiki.azimuth.navy reverse DNS resolves to cp11.miraheze.org
[10:35:57] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[10:49:33] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[10:58:14] <MirahezeBot>	 SPF|Cloud: Repool cp10
[10:58:15] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[11:11:51] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[11:13:02] <LeftTheNegotiato>	 so, for some reason I can't log into my account on google chrome
[11:24:07] <MH-Discord>	 <Left> ok
[11:24:09] <MH-Discord>	 <Left> so uh
[11:24:13] <MH-Discord>	 <Left> i cant stay in the relay
[11:24:27] <MH-Discord>	 <Left> but i got in, and i have another error as well
[11:25:14] <MH-Discord>	 <Left> on my main page, it says [f7911b53820a4b51e004ebf3] 2021-05-15 11:23:47: Fatal exception of type "TypeError"
[11:27:23] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[11:40:59] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[11:45:35] <Not-80da>	 [02miraheze/dns] 07Southparkfan pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JsnRI
[11:45:36] <Not-80da>	 [02miraheze/dns] 07Southparkfan 036612519 - Revert "Depool cp10 for performance test"
[11:49:43] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[11:56:27] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 5.00, 4.35, 2.03
[11:58:27] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 1.52, 3.21, 1.89
[12:06:26] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 6.12, 4.43, 2.91
[12:08:27] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 1.45, 3.31, 2.69
[12:37:19] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 10.97, 6.35, 3.88
[12:38:32] <icinga-miraheze>	 PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.39, 17.56, 14.58
[12:40:28] <icinga-miraheze>	 RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 19.43, 17.85, 15.04
[12:43:08] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 1.31, 3.81, 3.64
[12:44:07] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org
[12:47:05] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 0.91, 2.51, 3.16
[12:52:17] <icinga-miraheze>	 PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.47, 18.36, 15.37
[12:52:30] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.29, 5.74, 3.95
[12:52:55] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[12:53:03] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 4.38, 4.39, 3.78
[12:53:59] <icinga-miraheze>	 PROBLEM - mw8 Check Gluster Clients on mw8 is CRITICAL: PROCS CRITICAL: 0 processes with args '/usr/sbin/glusterfs'
[12:54:12] <icinga-miraheze>	 PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.52, 20.62, 16.58
[12:54:29] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.13, 5.46, 4.07
[12:55:00] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 1.46, 3.34, 3.47
[12:56:07] <icinga-miraheze>	 RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.83, 19.09, 16.50
[12:56:56] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 2.31, 3.10, 3.37
[13:05:56] <icinga-miraheze>	 PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 29.92, 23.94, 18.74
[13:06:31] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[13:06:41] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 3.06, 5.02, 4.16
[13:07:56] <icinga-miraheze>	 PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 17.60, 21.30, 18.40
[13:08:02] <icinga-miraheze>	 PROBLEM - mw8 Puppet on mw8 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/mnt/mediawiki-static]
[13:08:37] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 0.74, 3.58, 3.73
[13:09:57] <icinga-miraheze>	 RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 15.11, 18.62, 17.73
[13:12:30] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 0.58, 2.00, 3.06
[13:15:15] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[13:28:18] <icinga-miraheze>	 PROBLEM - cp11 Current Load on cp11 is WARNING: WARNING - load average: 2.17, 3.64, 2.16
[13:30:19] <icinga-miraheze>	 RECOVERY - cp11 Current Load on cp11 is OK: OK - load average: 0.44, 2.47, 1.91
[14:02:31] <paladox>	 SPF|Cloud: around?
[14:09:39] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[14:56:49] <Not-80da>	 [02miraheze/puppet] 07paladox pushed 036 commits to 03master [+2/-2/±31] 13https://git.io/JscvE
[14:56:50] <Not-80da>	 [02miraheze/puppet] 07paladox 0391053f3 - Disable xdebug on test4
[14:56:52] <Not-80da>	 [02miraheze/puppet] 07paladox 03bdfd1e6 - Remove unused hiera value default_lua_script
[14:56:53] <Not-80da>	 [02miraheze/puppet] 07paladox 03f9cd9f1 - Remove trafficserver::get_paths
[14:56:55] <Not-80da>	 [02miraheze/puppet] ... and 3 more commits.
[14:59:16] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[15:08:52] <Not-80da>	 [02mw-config] 07Universal-Omega closed pull request 03#3906: T7315: Move GlobalWatchlist to loginwiki - 13https://git.io/JsnLU
[15:08:53] <Not-80da>	 [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JscJK
[15:08:55] <Not-80da>	 [02miraheze/mw-config] 07Universal-Omega 03d783a2d - T7315: Move GlobalWatchlist to loginwiki (#3906)
[15:08:56] <Not-80da>	 [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://git.io/vbvb3
[15:08:58] <Not-80da>	 [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1
[15:09:52] <miraheze-github>	 miraheze/mw-config - Universal-Omega the build passed.
[15:12:52] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[15:20:41] <icinga-miraheze>	 PROBLEM - test4 Puppet on test4 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[15:21:33] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[15:25:04] <icinga-miraheze>	 PROBLEM - wiki.azimuth.navy - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Cannot make SSL connection.23401554855744:error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error:../ssl/record/rec_layer_s3.c:1544:SSL alert number 80
[15:35:10] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org
[15:43:53] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[15:57:29] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org
[15:57:42] <Not-80da>	 [02mw-config] 07dmehus commented on pull request 03#3906: T7315: Move GlobalWatchlist to loginwiki - 13https://git.io/Jsc36
[16:17:25] <Not-80da>	 [02mw-config] 07Universal-Omega commented on pull request 03#3906: T7315: Move GlobalWatchlist to loginwiki - 13https://git.io/Jscn7
[16:20:40] <icinga-miraheze>	 RECOVERY - wiki.azimuth.navy - LetsEncrypt on sslhost is OK: OK - Certificate 'azimuth.navy' will expire on Sun 15 May 2022 23:59:59 GMT +0000.
[16:24:25] <Not-80da>	 [02mw-config] 07dmehus commented on pull request 03#3906: T7315: Move GlobalWatchlist to loginwiki - 13https://git.io/JscWt
[16:48:49] <Not-80da>	 [02dns] 07dmehus commented on pull request 03#204: DNS purge 2021-05 - 13https://git.io/Jsc0q
[16:49:06] <Not-80da>	 [02dns] 07dmehus deleted a comment on pull request 03#204: DNS purge 2021-05 - 13https://git.io/Jsc0q
[17:00:06] <Not-80da>	 [02dns] 07Reception123 commented on pull request 03#204: DNS purge 2021-05 - 13https://git.io/JsczI
[17:00:09] <Not-80da>	 [02dns] 07Reception123 closed pull request 03#204: DNS purge 2021-05 - 13https://git.io/JsZlU
[17:00:11] <Not-80da>	 [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+0/-8/±0] 13https://git.io/JsczL
[17:00:12] <Not-80da>	 [02miraheze/dns] 07revi 0349ad495 - DNS purge 2021-05 (#204)
[17:04:26] <icinga-miraheze>	 PROBLEM - wiki.azimuth.navy - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.azimuth.navy could not be found
[17:34:47] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 7.54, 3.51, 1.94
[17:36:43] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 3.13, 3.01, 1.94
[17:38:28] <Reception123>	 !log sudo umount /mnt/mediawiki-static on mw8
[17:38:31] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[17:39:58] <icinga-miraheze>	 RECOVERY - mw8 Check Gluster Clients on mw8 is OK: PROCS OK: 1 process with args '/usr/sbin/glusterfs'
[17:55:07] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[18:05:21] <icinga-miraheze>	 RECOVERY - mw8 Puppet on mw8 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[18:08:42] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org
[18:37:48] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[18:51:24] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org
[18:55:16] <SPF|Cloud>	 paladox: around now (for a bit)
[19:00:05] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[19:04:10] <paladox>	 SPF|Cloud: should we move forward with switching i/o to none on cloud4 & vms?
[19:07:10] <SPF|Cloud>	 https://meta.miraheze.org/wiki/User:Southparkfan/2021-05-15-hdd-cloud
[19:07:11] <MirahezeBot>	 [ User:Southparkfan/2021-05-15-hdd-cloud - Miraheze Meta ] - meta.miraheze.org
[19:08:19] <SPF|Cloud>	 let's see if we can compare cloud5 data day-to-day with the previous week's
[19:13:41] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org
[19:15:35] <paladox>	 SPF|Cloud: so that screenshot makes it look like the change we did has done greatness? E.g. didn't cause much of an issue.
[19:16:18] <SPF|Cloud>	 lower is better, so the spike at 05/14 20:00 is not great
[19:16:55] <SPF|Cloud>	 however, cloud5 was under greater stress than cloud4 because cp10 was depooled, which could explain the extra load
[19:17:43] <paladox>	 yeh lower is better but that shows the green/yellow proves that the change did well. You can see almost all the time it's under cloud4 even when cp10 is depooled.
[19:19:33] <SPF|Cloud>	 I feel not enough time has passed to make a good call...
[19:19:52] <paladox>	 even so shouldn't the vms have it set to none regardless?
[19:19:59] <paladox>	 i see cp12 does that
[19:20:16] <SPF|Cloud>	 likely, otherwise you're scheduling i/o twice (at the guest and at the host)
[19:20:41] <paladox>	 yeh which would increase i/o, i'll switch the vms to none
[19:20:51] <SPF|Cloud>	 since changing the i/o scheduler seems 'safe' (does not cause critical issues), I'd say keep as-is for a week 
[19:21:15] <SPF|Cloud>	 afterwards we can do a week-by-week comparison
[19:21:22] <paladox>	 ok
[19:21:40] <paladox>	 !log mw8/9: echo "none" > /sys/block/sda/queue/scheduler
[19:21:43] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[19:21:50] <SPF|Cloud>	 uh
[19:22:10] <SPF|Cloud>	 you may want to keep it as-is on cloud4 and VMs
[19:23:07] <SPF|Cloud>	 but if we were to change the scheduler on cloud5 and VMs for at least a week, we can compare data between cloud4 and cloud5
[19:23:25] <SPF|Cloud>	 that's interesting data
[19:24:03] <SPF|Cloud>	 I have found out how to make a week-by-week comparison in grafana
[19:24:12] <paladox>	 oh
[19:24:16] <paladox>	 ok, i'll revert
[19:24:55] <SPF|Cloud>	 thanks
[19:25:00] <SPF|Cloud>	 are all cloud5 VMs on the none scheduler?
[19:26:49] <paladox>	 yeh
[19:27:46] <SPF|Cloud>	 nice
[19:28:28] <SPF|Cloud>	 take a look at https://meta.miraheze.org/m/5r6
[19:28:29] <MirahezeBot>	 [ Grafana ] - meta.miraheze.org
[19:41:31] <SPF|Cloud>	 !log queue checkarray test on cloud5
[19:41:34] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[19:43:02] <SPF|Cloud>	 paladox: ^ ran sudo /usr/share/mdadm/checkarray --all --idle
[19:43:19] <paladox>	 nice
[19:43:19] <SPF|Cloud>	 according to /proc/mdstat, the check 'works'
[19:43:25] <paladox>	 nice!
[19:43:53] <SPF|Cloud>	 the next question is if ionice does its job
[19:44:28] <icinga-miraheze>	 PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 7 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.195.236.250/cpweb, 2001:41d0:800:178a::5/cpweb, 2001:41d0:800:1bbd::4/cpweb, 51.222.25.132/cpweb, 2607:5300:205:200::1c30/cpweb
[19:44:30] <icinga-miraheze>	 PROBLEM - cp10 Stunnel Http for mw11 on cp10 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[19:44:55] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 9.58, 5.22, 2.75
[19:45:20] <icinga-miraheze>	 PROBLEM - wiki.ct777.cf - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.ct777.cf could not be found
[19:45:20] <SPF|Cloud>	 huh?
[19:45:24] <icinga-miraheze>	 PROBLEM - phab-storage.bots.miraheze.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for phab-storage.bots.miraheze.wiki could not be found
[19:45:27] <icinga-miraheze>	 PROBLEM - datacrondatabase.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for datacrondatabase.com could not be found
[19:45:28] <icinga-miraheze>	 PROBLEM - wiki.xysspon.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.xysspon.com could not be found
[19:45:30] <icinga-miraheze>	 PROBLEM - dariawiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for dariawiki.org could not be found
[19:45:31] <icinga-miraheze>	 PROBLEM - wiki.starship.digital - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.starship.digital could not be found
[19:45:33] <icinga-miraheze>	 PROBLEM - marinebiodiversitymatrix.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for marinebiodiversitymatrix.org could not be found
[19:45:59] <icinga-miraheze>	 PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 23.30, 21.34, 15.78
[19:46:09] <SPF|Cloud>	 the check isn't running on cloud4
[19:46:26] <icinga-miraheze>	 RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online
[19:46:30] <icinga-miraheze>	 RECOVERY - cp10 Stunnel Http for mw11 on cp10 is OK: HTTP OK: HTTP/1.1 200 OK - 15299 bytes in 1.677 second response time
[19:46:51] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 2.25, 3.98, 2.59
[19:47:06] <icinga-miraheze>	 PROBLEM - test2.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for test2.miraheze.org could not be found
[19:47:12] <icinga-miraheze>	 PROBLEM - wikiru.wildterra2.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wikiru.wildterra2.com could not be found
[19:47:12] <icinga-miraheze>	 PROBLEM - wiki.creaturathegame.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.creaturathegame.com could not be found
[19:47:17] <icinga-miraheze>	 PROBLEM - madzebrascience.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for madzebrascience.wiki could not be found
[19:47:18] <icinga-miraheze>	 PROBLEM - rangpurpedia.xyz - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for rangpurpedia.xyz could not be found
[19:47:19] <icinga-miraheze>	 PROBLEM - wiki.s23.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.s23.org could not be found
[19:47:21] <icinga-miraheze>	 PROBLEM - pornwiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for pornwiki.org could not be found
[19:47:22] <icinga-miraheze>	 PROBLEM - wlhmn.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wlhmn.tk could not be found
[19:47:22] <icinga-miraheze>	 PROBLEM - www.zenbuddhism.info - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.zenbuddhism.info could not be found
[19:47:22] <icinga-miraheze>	 PROBLEM - wiki.seamly.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.seamly.net could not be found
[19:47:25] <icinga-miraheze>	 PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for zw.fontainebleau-avon.fr could not be found
[19:47:57] <icinga-miraheze>	 RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 13.76, 19.26, 15.73
[19:48:48] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 0.46, 2.77, 2.31
[19:49:19] <paladox>	 SPF|Cloud: but it is on cloud5 which runs mw11
[19:49:46] <SPF|Cloud>	 what's wrong with mw11?
[19:52:07] <icinga-miraheze>	 RECOVERY - wiki.ct777.cf - reverse DNS on sslhost is OK: rDNS OK - wiki.ct777.cf reverse DNS resolves to cp10.miraheze.org
[19:52:08] <paladox>	 PROBLEM - cp10 Stunnel Http for mw11 on cp10 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[19:52:11] <paladox>	 it said ^
[19:52:16] <icinga-miraheze>	 RECOVERY - phab-storage.bots.miraheze.wiki - reverse DNS on sslhost is OK: rDNS OK - phab-storage.bots.miraheze.wiki reverse DNS resolves to cp10.miraheze.org
[19:52:18] <icinga-miraheze>	 RECOVERY - datacrondatabase.com - reverse DNS on sslhost is OK: rDNS OK - datacrondatabase.com reverse DNS resolves to cp11.miraheze.org
[19:52:18] <icinga-miraheze>	 RECOVERY - wiki.xysspon.com - reverse DNS on sslhost is OK: rDNS OK - wiki.xysspon.com reverse DNS resolves to cp11.miraheze.org
[19:52:23] <icinga-miraheze>	 RECOVERY - wiki.starship.digital - reverse DNS on sslhost is OK: rDNS OK - wiki.starship.digital reverse DNS resolves to cp10.miraheze.org
[19:52:24] <icinga-miraheze>	 RECOVERY - dariawiki.org - reverse DNS on sslhost is OK: rDNS OK - dariawiki.org reverse DNS resolves to cp11.miraheze.org
[19:52:26] <paladox>	 though i guess that could have been because of highload on cp10
[19:52:27] <SPF|Cloud>	 got it
[19:52:30] <icinga-miraheze>	 RECOVERY - marinebiodiversitymatrix.org - reverse DNS on sslhost is OK: rDNS OK - marinebiodiversitymatrix.org reverse DNS resolves to cp10.miraheze.org
[19:52:45] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 3.22, 3.61, 2.76
[19:53:57] <icinga-miraheze>	 RECOVERY - test2.miraheze.org - reverse DNS on sslhost is OK: rDNS OK - test2.miraheze.org reverse DNS resolves to cp11.miraheze.org
[19:53:57] <icinga-miraheze>	 RECOVERY - madzebrascience.wiki - reverse DNS on sslhost is OK: rDNS OK - madzebrascience.wiki reverse DNS resolves to cp10.miraheze.org
[19:54:01] <icinga-miraheze>	 RECOVERY - rangpurpedia.xyz - reverse DNS on sslhost is OK: rDNS OK - rangpurpedia.xyz reverse DNS resolves to cp11.miraheze.org
[19:54:06] <icinga-miraheze>	 RECOVERY - pornwiki.org - reverse DNS on sslhost is OK: rDNS OK - pornwiki.org reverse DNS resolves to cp10.miraheze.org
[19:54:06] <icinga-miraheze>	 RECOVERY - wiki.seamly.net - reverse DNS on sslhost is OK: rDNS OK - wiki.seamly.net reverse DNS resolves to cp11.miraheze.org
[19:54:09] <icinga-miraheze>	 RECOVERY - wlhmn.tk - reverse DNS on sslhost is OK: rDNS OK - wlhmn.tk reverse DNS resolves to cp11.miraheze.org
[19:54:09] <icinga-miraheze>	 RECOVERY - www.zenbuddhism.info - reverse DNS on sslhost is OK: rDNS OK - www.zenbuddhism.info reverse DNS resolves to cp11.miraheze.org
[19:54:10] <icinga-miraheze>	 RECOVERY - wikiru.wildterra2.com - reverse DNS on sslhost is OK: rDNS OK - wikiru.wildterra2.com reverse DNS resolves to cp10.miraheze.org
[19:54:11] <icinga-miraheze>	 RECOVERY - wiki.creaturathegame.com - reverse DNS on sslhost is OK: rDNS OK - wiki.creaturathegame.com reverse DNS resolves to cp11.miraheze.org
[19:54:15] <icinga-miraheze>	 RECOVERY - zw.fontainebleau-avon.fr - reverse DNS on sslhost is OK: rDNS OK - zw.fontainebleau-avon.fr reverse DNS resolves to cp10.miraheze.org
[19:54:19] <icinga-miraheze>	 RECOVERY - wiki.s23.org - reverse DNS on sslhost is OK: rDNS OK - wiki.s23.org reverse DNS resolves to cp10.miraheze.org
[19:54:41] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 2.89, 3.19, 2.69
[19:56:23] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[19:58:35] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 2.72, 5.14, 3.76
[20:02:28] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 1.77, 3.68, 3.48
[20:06:27] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 1.03, 2.48, 3.06
[20:07:48] <Not-80da>	 [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JsCe2
[20:07:49] <Not-80da>	 [02miraheze/mw-config] 07Universal-Omega 03b8013bd - Change wgCreateWikiNotificationEmail to sre@
[20:07:51] <Not-80da>	 [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/vbvb3
[20:07:52] <Not-80da>	 [02mw-config] 07Universal-Omega opened pull request 03#3907: Change wgCreateWikiNotificationEmail to sre@ - 13https://git.io/JsCea
[20:08:18] <SPF|Cloud>	 'night' (afk)
[20:08:59] <miraheze-github>	 miraheze/mw-config - Universal-Omega the build passed.
[20:10:15] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org
[20:10:18] <icinga-miraheze>	 PROBLEM - cp11 Current Load on cp11 is WARNING: WARNING - load average: 3.36, 3.64, 2.26
[20:11:33] <Not-80da>	 [02mw-config] 07Universal-Omega closed pull request 03#3907: Change wgCreateWikiNotificationEmail to sre@ - 13https://git.io/JsCea
[20:11:34] <Not-80da>	 [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JsCej
[20:11:36] <Not-80da>	 [02miraheze/mw-config] 07Universal-Omega 03e673cfe - Change wgCreateWikiNotificationEmail to sre@ (#3907)
[20:11:37] <Not-80da>	 [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1
[20:11:39] <Not-80da>	 [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://git.io/vbvb3
[20:12:18] <icinga-miraheze>	 RECOVERY - cp11 Current Load on cp11 is OK: OK - load average: 1.52, 2.73, 2.09
[20:12:32] <miraheze-github>	 miraheze/mw-config - Universal-Omega the build passed.
[20:19:15] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[20:32:18] <icinga-miraheze>	 PROBLEM - cp11 Current Load on cp11 is CRITICAL: CRITICAL - load average: 1.37, 4.03, 3.28
[20:34:20] <icinga-miraheze>	 RECOVERY - cp11 Current Load on cp11 is OK: OK - load average: 1.80, 3.17, 3.04
[20:58:03] <Not-80da>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JsCI1
[20:58:04] <Not-80da>	 [02miraheze/puppet] 07paladox 034b65bb3 - Fix
[20:58:59] <Not-80da>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JsCIH
[20:59:01] <Not-80da>	 [02miraheze/puppet] 07paladox 03a0a7573 - Fix
[21:00:42] <icinga-miraheze>	 RECOVERY - test4 Puppet on test4 is OK: OK: Puppet is currently enabled, last run 53 seconds ago with 0 failures
[21:13:44] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org
[21:43:15] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[21:56:51] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[22:05:32] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[22:19:10] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org
[22:34:40] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[22:48:17] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org
[23:31:00] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found
[23:44:36] <icinga-miraheze>	 RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org
[23:53:19] <icinga-miraheze>	 PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found