[00:05:10] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.02, 7.28, 5.92
[00:07:09] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.61, 6.69, 5.86
[00:12:54] <icinga-miraheze>	 PROBLEM - holonet.pw - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for holonet.pw could not be found
[00:12:55] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.42, 7.08, 6.45
[00:13:04] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 5.77, 7.11, 6.36
[00:14:08] <icinga-miraheze>	 PROBLEM - holonet.pw - LetsEncrypt on sslhost is CRITICAL: connect to address holonet.pw and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket
[00:14:50] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.78, 6.65, 6.36
[00:15:03] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.64, 6.68, 6.30
[00:17:51] <icinga-miraheze>	 PROBLEM - wiki.mlpwiki.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mlpwiki.net reverse DNS resolves to 192-185-16-85.unifiedlayer.com
[00:34:59] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.12, 5.95, 5.69
[00:36:58] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.49, 5.33, 5.49
[02:26:54] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.17, 6.32, 5.30
[02:32:54] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.16, 6.38, 5.74
[02:34:28] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.12, 6.21, 4.86
[02:36:28] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.89, 5.60, 4.80
[02:36:54] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.64, 6.89, 6.14
[02:38:54] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 10.31, 7.86, 6.57
[02:39:46] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.25, 7.68, 6.43
[02:40:54] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.58, 7.42, 6.58
[02:43:41] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 5.34, 6.92, 6.44
[02:45:39] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.45, 6.20, 6.24
[02:46:54] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.73, 6.05, 6.35
[02:50:29] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.95, 6.38, 5.55
[02:51:35] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.26, 7.15, 6.60
[02:52:28] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.35, 6.41, 5.66
[02:53:36] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.94, 6.43, 6.41
[03:06:20] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.08, 7.82, 6.92
[03:06:54] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.92, 6.76, 6.26
[03:08:11] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.97, 7.28, 6.02
[03:08:18] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.61, 7.21, 6.80
[03:08:55] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 6.01, 6.51, 6.23
[03:10:06] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.78, 6.42, 5.85
[03:16:09] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.47, 6.61, 6.74
[03:17:56] <icinga-miraheze>	 RECOVERY - wiki.mlpwiki.net - reverse DNS on sslhost is OK: rDNS OK - wiki.mlpwiki.net reverse DNS resolves to cp11.miraheze.org
[04:02:35] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.16, 6.58, 5.49
[04:04:10] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.70, 6.21, 5.11
[04:04:35] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.38, 6.38, 5.54
[04:06:05] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.86, 5.90, 5.15
[06:02:20] <icinga-miraheze>	 RECOVERY - mw10 APT on mw10 is OK: APT OK: 31 packages available for upgrade (0 critical updates).
[06:02:25] <icinga-miraheze>	 RECOVERY - cp11 APT on cp11 is OK: APT OK: 22 packages available for upgrade (0 critical updates).
[06:03:09] <icinga-miraheze>	 RECOVERY - cp12 APT on cp12 is OK: APT OK: 22 packages available for upgrade (0 critical updates).
[06:03:13] <icinga-miraheze>	 RECOVERY - gluster3 APT on gluster3 is OK: APT OK: 30 packages available for upgrade (0 critical updates).
[06:03:31] <icinga-miraheze>	 RECOVERY - cloud4 APT on cloud4 is OK: APT OK: 59 packages available for upgrade (0 critical updates).
[06:05:17] <icinga-miraheze>	 RECOVERY - puppet3 APT on puppet3 is OK: APT OK: 22 packages available for upgrade (0 critical updates).
[06:05:20] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.12, 5.86, 4.77
[06:05:38] <icinga-miraheze>	 RECOVERY - test4 APT on test4 is OK: APT OK: 21 packages available for upgrade (0 critical updates).
[06:06:33] <icinga-miraheze>	 RECOVERY - mw8 APT on mw8 is OK: APT OK: 31 packages available for upgrade (0 critical updates).
[06:07:20] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 3.37, 4.87, 4.54
[06:09:32] <icinga-miraheze>	 RECOVERY - ns2 APT on ns2 is OK: APT OK: 34 packages available for upgrade (0 critical updates).
[06:10:58] <icinga-miraheze>	 RECOVERY - mon2 APT on mon2 is OK: APT OK: 28 packages available for upgrade (0 critical updates).
[06:11:03] <icinga-miraheze>	 PROBLEM - test3 MediaWiki Rendering on test3 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 324 bytes in 0.009 second response time
[06:13:04] <icinga-miraheze>	 RECOVERY - test3 MediaWiki Rendering on test3 is OK: HTTP OK: HTTP/1.1 200 OK - 20762 bytes in 0.187 second response time
[06:14:37] <Not-818c>	 [02DataDump] 07Reception123 commented on pull request 03#21: Fix T7068: Add 'useRestriction' option to 'generate' - 13https://git.io/JYPxo
[06:14:53] <icinga-miraheze>	 RECOVERY - cp3 APT on cp3 is OK: APT OK: 22 packages available for upgrade (0 critical updates).
[06:17:39] <icinga-miraheze>	 RECOVERY - phab2 APT on phab2 is OK: APT OK: 22 packages available for upgrade (0 critical updates).
[06:18:57] <icinga-miraheze>	 RECOVERY - db12 APT on db12 is OK: APT OK: 69 packages available for upgrade (0 critical updates).
[06:25:12] <icinga-miraheze>	 RECOVERY - cloud5 APT on cloud5 is OK: APT OK: 59 packages available for upgrade (0 critical updates).
[06:27:20] <icinga-miraheze>	 RECOVERY - ldap2 APT on ldap2 is OK: APT OK: 21 packages available for upgrade (0 critical updates).
[06:29:05] <icinga-miraheze>	 RECOVERY - graylog2 APT on graylog2 is OK: APT OK: 21 packages available for upgrade (0 critical updates).
[06:29:09] <icinga-miraheze>	 PROBLEM - wiki.ct777.cf - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.ct777.cf' expires in 15 day(s) (Mon 19 Apr 2021 06:20:55 GMT +0000).
[06:29:38] <icinga-miraheze>	 RECOVERY - cp10 APT on cp10 is OK: APT OK: 34 packages available for upgrade (0 critical updates).
[06:29:41] <icinga-miraheze>	 RECOVERY - jobrunner4 APT on jobrunner4 is OK: APT OK: 31 packages available for upgrade (0 critical updates).
[06:29:59] <icinga-miraheze>	 RECOVERY - services4 APT on services4 is OK: APT OK: 22 packages available for upgrade (0 critical updates).
[06:32:40] <Not-818c>	 [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JYPjV
[06:32:41] <Not-818c>	 [02miraheze/ssl] 07MirahezeSSLBot 039a78c3d - Bot: Update SSL cert for wiki.ct777.cf
[06:33:28] <icinga-miraheze>	 RECOVERY - mail2 APT on mail2 is OK: APT OK: 22 packages available for upgrade (0 critical updates).
[06:34:15] <icinga-miraheze>	 RECOVERY - test3 APT on test3 is OK: APT OK: 32 packages available for upgrade (0 critical updates).
[06:35:10] <icinga-miraheze>	 RECOVERY - jobrunner3 APT on jobrunner3 is OK: APT OK: 33 packages available for upgrade (0 critical updates).
[06:35:17] <icinga-miraheze>	 RECOVERY - mem2 APT on mem2 is OK: APT OK: 21 packages available for upgrade (0 critical updates).
[06:36:26] <icinga-miraheze>	 RECOVERY - ns1 APT on ns1 is OK: APT OK: 38 packages available for upgrade (0 critical updates).
[06:37:36] <icinga-miraheze>	 RECOVERY - mw11 APT on mw11 is OK: APT OK: 31 packages available for upgrade (0 critical updates).
[06:40:41] <icinga-miraheze>	 RECOVERY - db13 APT on db13 is OK: APT OK: 45 packages available for upgrade (0 critical updates).
[06:47:59] <icinga-miraheze>	 RECOVERY - cloud3 APT on cloud3 is OK: APT OK: 105 packages available for upgrade (0 critical updates).
[06:50:33] <icinga-miraheze>	 RECOVERY - services3 APT on services3 is OK: APT OK: 22 packages available for upgrade (0 critical updates).
[06:50:47] <icinga-miraheze>	 RECOVERY - mw9 APT on mw9 is OK: APT OK: 31 packages available for upgrade (0 critical updates).
[06:51:27] <icinga-miraheze>	 RECOVERY - mem1 APT on mem1 is OK: APT OK: 21 packages available for upgrade (0 critical updates).
[06:57:04] <Not-818c>	 [02miraheze/ssl] 07Reception123 pushed 033 commits to 03master [+3/-0/±3] 13https://git.io/JYXJh
[06:57:06] <Not-818c>	 [02miraheze/ssl] 07Reception123 03adaaae4 - add you.r-fit.cc cert
[06:57:07] <Not-818c>	 [02miraheze/ssl] 07Reception123 031eb92d2 - add wmworld.sktz.live cert
[06:57:08] <icinga-miraheze>	 RECOVERY - gluster4 APT on gluster4 is OK: APT OK: 30 packages available for upgrade (0 critical updates).
[06:57:09] <Not-818c>	 [02miraheze/ssl] 07Reception123 03de2047d - add olwest.org cert
[06:58:36] <Not-818c>	 [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/JYXUs
[06:58:38] <Not-818c>	 [02miraheze/ssl] 07Reception123 03869fd82 - add wiki.zaoace.com cert
[06:59:52] <icinga-miraheze>	 RECOVERY - db11 APT on db11 is OK: APT OK: 69 packages available for upgrade (0 critical updates).
[07:04:47] <icinga-miraheze>	 PROBLEM - rangpurpedia.xyz - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'rangpurpedia.xyz' expires in 15 day(s) (Mon 19 Apr 2021 07:02:10 GMT +0000).
[07:08:15] <Not-818c>	 [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JYXTy
[07:08:16] <Not-818c>	 [02miraheze/ssl] 07MirahezeSSLBot 034b47be5 - Bot: Update SSL cert for rangpurpedia.xyz
[07:08:24] <icinga-miraheze>	 PROBLEM - www.rangpurpedia.xyz - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'rangpurpedia.xyz' expires in 15 day(s) (Mon 19 Apr 2021 07:02:10 GMT +0000).
[07:10:21] <icinga-miraheze>	 RECOVERY - wiki.ct777.cf - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.ct777.cf' will expire on Fri 02 Jul 2021 05:32:35 GMT +0000.
[07:11:32] <Not-818c>	 [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JYXkB
[07:11:33] <Not-818c>	 [02miraheze/puppet] 07Reception123 0316b3699 - remove universalomega from mw-admins (resignation)
[07:12:10] <Not-818c>	 [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JYXkK
[07:12:12] <Not-818c>	 [02miraheze/puppet] 07Reception123 035bb8fb5 - remove universalomega from monitoring (resignation)
[07:12:46] <Reception123>	 !log removed Universal Omega from @miraheze GH
[07:12:50] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:17:12] <Reception123>	 !log removed universalomega from grafana
[07:17:25] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:18:56] <Reception123>	 !log removed universalomega from graylog
[07:19:00] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:20:08] <Reception123>	 !log removed universalomega from matomo
[07:20:13] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:23:39] <Reception123>	 !log removed universalomega from mail+groups
[07:23:42] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:25:42] <Not-818c>	 [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JYXt1
[07:25:44] <Not-818c>	 [02miraheze/mw-config] 07Reception123 03594399b - remove universalomega from staffwiki
[07:26:45] <miraheze-github>	 miraheze/mw-config - Reception123 the build passed.
[07:36:14] <icinga-miraheze>	 RECOVERY - www.rangpurpedia.xyz - LetsEncrypt on sslhost is OK: OK - Certificate 'rangpurpedia.xyz' will expire on Fri 02 Jul 2021 06:08:10 GMT +0000.
[07:39:06] <icinga-miraheze>	 RECOVERY - rangpurpedia.xyz - LetsEncrypt on sslhost is OK: OK - Certificate 'rangpurpedia.xyz' will expire on Fri 02 Jul 2021 06:08:10 GMT +0000.
[09:24:40] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 2.25, 4.03, 2.30
[09:26:41] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 2.09, 3.14, 2.17
[09:32:13] <icinga-miraheze>	 PROBLEM - wiki.mlpwiki.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mlpwiki.net reverse DNS resolves to 192-185-16-85.unifiedlayer.com
[10:34:40] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 5.25, 5.52, 2.87
[10:36:41] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 1.03, 3.80, 2.56
[10:38:39] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 1.30, 3.10, 2.46
[11:29:57] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 2.60, 5.29, 3.22
[11:31:54] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 0.54, 3.66, 2.87
[11:33:52] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 0.33, 2.61, 2.57
[12:03:30] <Not-818c>	 [02DataDump] 07paladox commented on pull request 03#21: Fix T7068: Add 'useRestriction' option to 'generate' - 13https://git.io/JYXxn
[12:04:45] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.03, 6.18, 5.01
[12:06:45] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.05, 5.45, 4.87
[12:32:00] <icinga-miraheze>	 RECOVERY - wiki.mlpwiki.net - reverse DNS on sslhost is OK: rDNS OK - wiki.mlpwiki.net reverse DNS resolves to cp10.miraheze.org
[12:51:26] <Not-818c>	 [02DataDump] 07Reception123 commented on pull request 03#21: Fix T7068: Add 'useRestriction' option to 'generate' - 13https://git.io/JY1Tp
[13:53:29] <Not-818c>	 [02DataDump] 07paladox closed pull request 03#21: Fix T7068: Add 'useRestriction' option to 'generate' - 13https://git.io/JYi4v
[13:53:30] <Not-818c>	 [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JY1Z7
[13:53:32] <Not-818c>	 [02miraheze/DataDump] 07Universal-Omega 0316a320e - Fix T7068: Add 'useRestriction' option to 'generate' (#21)
[13:53:33] <Not-818c>	 [02miraheze/DataDump] 07paladox deleted branch 03Universal-Omega-patch-1
[13:53:35] <Not-818c>	 [02DataDump] 07paladox deleted branch 03Universal-Omega-patch-1 - 13https://git.io/fhhKV
[13:54:47] <miraheze-github>	 miraheze/DataDump - paladox the build passed.
[13:54:54] <Not-818c>	 [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JY1nI
[13:54:55] <Not-818c>	 [02miraheze/mediawiki] 07paladox 0332d664a - Update DataDump
[14:00:36] <paladox>	 "error=RuntimeException: FirejailCommand does not support parameters that start with --output" still fails
[14:01:04] <RhinosF1>	 I thought the point of that was to stop it using firejail
[14:01:35] <RhinosF1>	 $restriction is checking something
[14:01:41] <RhinosF1>	 So does config need update
[14:01:47] <MH-Discord>	 <DarkMatterMan4500> Firejail?
[14:02:08] <RhinosF1>	 Sandboxes things to make them less dangerous
[14:02:13] <RhinosF1>	 But data dump doesn't need it
[14:05:46] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 9.52, 8.34, 6.63
[14:05:55] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.54, 6.97, 5.92
[14:11:40] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.11, 7.68, 6.97
[14:11:48] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.12, 6.73, 6.26
[14:13:16] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.84, 6.49, 5.18
[14:13:39] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.71, 6.44, 6.59
[14:15:17] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.22, 5.87, 5.11
[14:17:39] <Reception123>	 I wonder why that didn't work
[14:19:49] <RhinosF1>	 I said why it didn't
[14:19:59] <RhinosF1>	 Config needs updating
[14:20:17] <darkmatterman450>	 I wonder what firejail even does.
[14:20:42] <RhinosF1>	 Sandboxes stuff
[14:20:48] <RhinosF1>	 Keeps Miraheze more secure
[14:20:58] <RhinosF1>	 But it's doing too good with data dump
[14:30:00] <paladox>	 https://gerrit.wikimedia.org/r/c/mediawiki/core/+/676519
[14:30:03] <paladox>	 RhinosF1 
[14:30:42] <paladox>	 ideally we should be able to define whether we should use firejail rather than it auto detecting. As in we should be able to disable it.
[14:30:47] <RhinosF1>	 paladox: doesn't it need to hit master first
[14:30:58] <paladox>	 master seems to have been rewritten
[14:31:05] <paladox>	 it no longer uses that class
[14:31:11] <paladox>	 so i'm not sure if it has the same problem
[14:31:15] <RhinosF1>	 Ah ok
[14:31:39] <RhinosF1>	 Jerkins doesn't like it
[14:32:44] <paladox>	 yup
[14:33:10] <paladox>	 i guess proving disableFirejail() could work?
[14:33:16] <paladox>	 *providing
[14:35:14] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.00, 7.80, 6.17
[14:35:43] <paladox>	 also why are the mw*s experencing highload the passed few days
[14:36:16] <RhinosF1>	 Not sure
[14:36:48] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 46.39, 21.91, 9.23
[14:37:03] <icinga-miraheze>	 PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 44.41, 29.63, 19.50
[14:37:13] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.14, 6.69, 5.95
[14:40:58] <icinga-miraheze>	 PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 15.55, 21.87, 18.67
[14:42:56] <icinga-miraheze>	 RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 13.41, 18.86, 17.93
[14:51:33] <Not-818c>	 [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-1/±1] 13https://git.io/JY1uX
[14:51:34] <Not-818c>	 [02miraheze/ssl] 07Reception123 03a8863b3 - rm holonet.pw cert T7093
[14:52:40] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 0.80, 1.81, 3.98
[14:53:18] <Reception123>	 !log reception@jobrunner3:~$ sudo -u www-data php /srv/mediawiki/w/maintenance/updateArticleCount.php --update --wiki trollpastawiki
[14:53:21] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[14:56:07] <Reception123>	 !log sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=minecraftwiki --r "[[phab:T7066|Requested]]" /home/reception/minecraftdel.txt
[14:56:10] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[14:56:40] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 1.30, 1.50, 3.34
[15:00:40] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 1.52, 2.54, 3.44
[15:02:40] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 0.88, 2.06, 3.16
[15:06:04] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.82, 6.74, 5.74
[15:08:04] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.04, 6.04, 5.60
[15:10:01] <Reception123>	 !log renamed khatikwiki to famepediawiki
[15:10:05] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[15:10:11] <Not-818c>	 [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JY12P
[15:10:12] <Not-818c>	 [02miraheze/services] 07MirahezeSSLBot 03e9bf83d - BOT: Updating services config for wikis
[15:13:44] <icinga-miraheze>	 PROBLEM - dbbackup1 Check MariaDB Replication c2 on dbbackup1 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 223s
[15:16:09] <MH-Discord>	 <Lake> wow, it's super slow for me now
[15:16:52] <icinga-miraheze>	 PROBLEM - cp3 SSH on cp3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[15:16:52] <icinga-miraheze>	 PROBLEM - cp3 PowerDNS Recursor on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[15:17:09] <icinga-miraheze>	 PROBLEM - cp3 Stunnel Http for mon2 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[15:17:15] <icinga-miraheze>	 PROBLEM - cp3 Disk Space on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[15:17:20] <icinga-miraheze>	 PROBLEM - cp3 Current Load on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[15:17:29] <icinga-miraheze>	 PROBLEM - cp3 APT on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[15:17:53] <icinga-miraheze>	 PROBLEM - cp3 Stunnel Http for mw10 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[15:17:55] <icinga-miraheze>	 PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[15:18:26] <icinga-miraheze>	 PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb
[15:18:35] <icinga-miraheze>	 PROBLEM - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is CRITICAL: CRITICAL - NGINX Error Rate is 74%
[15:18:48] <icinga-miraheze>	 PROBLEM - ping4 on cp3 is WARNING: PING WARNING - Packet loss = 0%, RTA = 321.16 ms
[15:18:52] <icinga-miraheze>	 RECOVERY - cp3 PowerDNS Recursor on cp3 is OK: DNS OK: 0.494 seconds response time. miraheze.org returns 2001:41d0:800:178a::5,2001:41d0:800:1bbd::4,51.195.236.219,51.195.236.250
[15:18:54] <icinga-miraheze>	 RECOVERY - cp3 SSH on cp3 is OK: SSH OK - OpenSSH_7.9p1 Debian-10+deb10u2 (protocol 2.0)
[15:18:57] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb
[15:19:06] <icinga-miraheze>	 RECOVERY - cp3 Stunnel Http for mon2 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 35473 bytes in 1.325 second response time
[15:19:15] <icinga-miraheze>	 RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 7573 MB (31% inode=94%);
[15:19:18] <icinga-miraheze>	 RECOVERY - cp3 Current Load on cp3 is OK: OK - load average: 0.04, 0.18, 0.21
[15:19:50] <icinga-miraheze>	 RECOVERY - cp3 Stunnel Http for mw10 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15209 bytes in 1.013 second response time
[15:20:28] <icinga-miraheze>	 RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online
[15:20:35] <icinga-miraheze>	 RECOVERY - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is OK: OK - NGINX Error Rate is 6%
[15:21:03] <icinga-miraheze>	 RECOVERY - cp3 APT on cp3 is OK: APT OK: 22 packages available for upgrade (0 critical updates).
[15:21:20] <icinga-miraheze>	 RECOVERY - ping4 on cp3 is OK: PING OK - Packet loss = 0%, RTA = 250.69 ms
[15:21:35] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[15:21:54] <icinga-miraheze>	 RECOVERY - cp3 Varnish Backends on cp3 is OK: All 7 backends are healthy
[15:26:46] <RhinosF1>	 paladox: what happened
[15:26:51] <paladox>	 hmm?
[15:26:54] <RhinosF1>	 @Lake: things went bump
[15:26:59] <RhinosF1>	 paladox: the icinga alerts
[15:27:16] <RhinosF1>	 Looks like cp3 just went down
[15:27:26] <RhinosF1>	 It's back though
[15:27:27] <paladox>	 oh and its back
[15:28:13] <RhinosF1>	 paladox: we still need to know why
[15:28:21] <RhinosF1>	 Memory usage looks high
[15:29:01] <paladox>	 looks like network?
[15:29:21] <RhinosF1>	 :(
[15:29:59] <paladox>	 i see nothing in the logs but https://grafana.miraheze.org/d/W9MIkA7iz/miraheze-cluster?orgId=1&var-job=node&var-node=cp3.miraheze.org&var-port=9100 looks like network.
[15:30:00] <MirahezeBot>	 [ Grafana ] - grafana.miraheze.org
[15:31:36] <icinga-miraheze>	 PROBLEM - ping4 on cp3 is WARNING: PING WARNING - Packet loss = 0%, RTA = 309.74 ms
[15:32:19] <RhinosF1>	 That's network for sure
[15:35:59] <icinga-miraheze>	 PROBLEM - wiki.mlpwiki.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mlpwiki.net reverse DNS resolves to 192-185-16-85.unifiedlayer.com
[15:36:54] <icinga-miraheze>	 PROBLEM - mw8 Puppet on mw8 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[php7.3-apcu]
[16:00:11] <icinga-miraheze>	 RECOVERY - ping4 on cp3 is OK: PING OK - Packet loss = 0%, RTA = 256.55 ms
[16:00:29] <Reception123>	 !log MariaDB [crappygameswiki]> DELETE FROM oldimage WHERE oi_archive_name = '';
[16:00:33] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[16:03:38] <Not-818c>	 [02miraheze/DataDump] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JY1X6
[16:03:40] <Not-818c>	 [02miraheze/DataDump] 07paladox 0358ca709 - Fix creating dumps
[16:03:41] <Not-818c>	 [02DataDump] 07paladox created branch 03paladox-patch-1 - 13https://git.io/fhhKV
[16:03:43] <Not-818c>	 [02DataDump] 07paladox opened pull request 03#22: Fix creating dumps - 13https://git.io/JY1XP
[16:03:55] <paladox>	 RhinosF1 
[16:05:11] <Not-818c>	 [02miraheze/DataDump] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JY1Xj
[16:05:13] <Not-818c>	 [02miraheze/DataDump] 07paladox 03bb9b3f0 - Update DataDumpGenerateJob.php
[16:05:14] <Not-818c>	 [02DataDump] 07paladox synchronize pull request 03#22: Fix creating dumps - 13https://git.io/JY1XP
[16:05:16] <Not-818c>	 [02CreateWiki] 07Universal-Omega closed pull request 03#207: Default CreateWikiPurposes to '' - 13https://git.io/JY8gw
[16:05:46] <Not-818c>	 [02DataDump] 07paladox closed pull request 03#22: Fix creating dumps - 13https://git.io/JY1XP
[16:05:47] <Not-818c>	 [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JY11L
[16:05:49] <Not-818c>	 [02miraheze/DataDump] 07paladox 036c78be2 - Fix creating dumps (#22)
[16:05:50] <Not-818c>	 [02DataDump] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/fhhKV
[16:05:52] <miraheze-github>	 miraheze/DataDump - paladox the build passed.
[16:05:52] <Not-818c>	 [02miraheze/DataDump] 07paladox deleted branch 03paladox-patch-1
[16:05:54] <Reception123>	 paladox: thanks for fixing it (hopefully). Does that mean that it won't work for 1.36 anymore? if yes, what can we do?
[16:06:12] <miraheze-github>	 miraheze/DataDump - paladox the build passed.
[16:06:12] <icinga-miraheze>	 RECOVERY - mw8 Puppet on mw8 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[16:06:17] <Not-818c>	 [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JY11Z
[16:06:19] <Not-818c>	 [02miraheze/mediawiki] 07paladox 03dd5aa27 - Update DataDump
[16:06:57] <miraheze-github>	 miraheze/DataDump - paladox the build passed.
[16:07:38] <icinga-miraheze>	 PROBLEM - dbbackup1 Check MariaDB Replication c2 on dbbackup1 is WARNING: MariaDB replication - both - WARNING - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 154s
[16:09:37] <icinga-miraheze>	 RECOVERY - dbbackup1 Check MariaDB Replication c2 on dbbackup1 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 91s
[16:09:43] <RhinosF1>	 paladox: ty
[16:10:07] <RhinosF1>	 If it blocks 1.36 then we have an upgrade blockers column on mw repo
[16:14:36] <icinga-miraheze>	 PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.88, 20.84, 16.71
[16:14:55] <RhinosF1>	 Mw board*
[16:16:34] <icinga-miraheze>	 RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.53, 19.02, 16.54
[16:22:15] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 4.77, 4.71, 3.36
[16:24:12] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 1.14, 3.54, 3.11
[16:26:10] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 1.88, 3.06, 2.98
[16:28:20] <Universal_Omega>	 Thanks for finishing that paladox. RhinosF1: from what I understood, my method works on 1.36, paladox makes it work on 1.35 as well.
[16:33:41] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.57, 7.53, 6.49
[16:35:43] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.35, 7.49, 6.63
[16:38:13] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.49, 6.96, 6.10
[16:38:46] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.91, 7.53, 6.35
[16:39:47] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.45, 6.52, 6.44
[16:40:13] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.93, 6.62, 6.08
[16:42:46] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.80, 6.48, 6.22
[16:45:54] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 2.32, 3.54, 2.87
[16:47:52] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 1.13, 2.79, 2.68
[17:05:29] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.57, 6.88, 6.20
[17:07:24] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.45, 6.43, 6.13
[17:15:06] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.27, 7.27, 6.50
[17:17:00] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.34, 7.05, 6.53
[17:18:55] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.09, 6.22, 6.30
[17:32:24] <Reception123>	 Voidwalker: I know you're busy with something else, but if you have a few spare minutes I feel you could have an idea about this. r.e. https://phabricator.miraheze.org/T7055 how come https://csydes.miraheze.org/wiki/Talk:AjaxPoll:Alto%20or%20Tenor%20Saxophone%20-%20Which%20is%20better?/@comment-24258073-20171213063453 gives bad title? it must've worked on a wiki at some point or else it couldn't have been exported
[17:32:25] <MirahezeBot>	 [ ⚓ T7055 Problem with importing data to the Talk:AjaxPoll namespace ] - phabricator.miraheze.org
[17:32:26] <MirahezeBot>	 [ Bad title | C.Syde's Wiki | Miraheze ] - csydes.miraheze.org
[17:33:52] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.74, 6.97, 5.95
[17:34:06] <Voidwalker>	 Sounds like the import might have stuck it in the wrong namespace. It seems like it should be in AjaxPoll talk: instead of Talk:AjaxPoll:
[17:35:52] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.11, 6.99, 6.09
[17:36:56] <dmehus>	 Voidwalker, yeah
[17:37:52] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.71, 6.50, 6.02
[17:44:51] <Reception123>	 Voidwalker: hmm that's weird but all the pages from the XML are like this
[17:44:52] <Reception123>	 <title>Talk:AjaxPoll:Alto or Tenor Saxophone - Which is better?/@comment-24258073-20171213063453</title>
[17:45:41] <RhinosF1>	 Reception123: do we know what wiki it came from
[17:46:28] <Reception123>	     <base>https://csydes.fandom.com/wiki/C.Syde%27s_Wiki</base>
[17:46:28] <MirahezeBot>	 [ Bad title | C.Syde's Wiki | Fandom ] - csydes.fandom.com
[17:46:36] <Reception123>	 that link won't work because of the base tag
[17:46:42] <Reception123>	 so https://csydes.fandom.com/wiki/C.Syde%27s_Wiki 
[17:46:42] <MirahezeBot>	 [ C.Syde's Wiki | Fandom ] - csydes.fandom.com
[17:49:20] <RhinosF1>	 Reception123: fandom have broken i18n but not sure anything between 1.33 and 1.35 that would cause issues but it's possible
[17:50:09] <Reception123>	 hm
[17:54:11] <Not-818c>	 [02miraheze/CreateWiki] 07paladox deleted branch 03Universal-Omega-patch-1
[17:54:13] <Not-818c>	 [02CreateWiki] 07paladox deleted branch 03Universal-Omega-patch-1 - 13https://git.io/vpJTL
[18:20:02] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.04, 7.40, 6.26
[18:22:00] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.63, 6.66, 6.12
[18:22:36] <paladox>	 RhinosF1 https://gerrit.wikimedia.org/r/c/mediawiki/core/+/676522
[18:22:46] <paladox>	 should allow us to use -o rather than --output
[18:23:02] <paladox>	 similar to https://github.com/wikimedia/mediawiki-extensions-SecurePoll/commit/75ec652c3d81665adc2bbd3e2d80a7587db542f2
[18:23:03] <MirahezeBot>	 [ Use -o rather than --output when invoking gpg · wikimedia/mediawiki-extensions-SecurePoll@75ec652 · GitHub ] - github.com
[18:23:19] <Reception123>	 oh, that's interesting
[18:23:51] <RhinosF1>	 paladox: but it shouldn't be using firejail at all
[18:24:31] <RhinosF1>	 But yes
[18:26:19] <Not-818c>	 [02miraheze/MirahezeMagic] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JYMJ7
[18:26:20] <Not-818c>	 [02miraheze/MirahezeMagic] 07paladox 033f2442a - Remove datadump-desc
[18:26:22] <Not-818c>	 [02MirahezeMagic] 07paladox created branch 03paladox-patch-2 - 13https://git.io/fQRGX
[18:26:23] <Not-818c>	 [02MirahezeMagic] 07paladox opened pull request 03#243: Remove datadump-desc - 13https://git.io/JYMJ5
[18:26:41] <Not-818c>	 [02miraheze/MirahezeMagic] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JYMJb
[18:26:42] <Not-818c>	 [02miraheze/MirahezeMagic] 07paladox 03036a911 - Update en.json
[18:26:44] <Not-818c>	 [02MirahezeMagic] 07paladox synchronize pull request 03#243: Remove datadump-desc - 13https://git.io/JYMJ5
[18:26:51] <Not-818c>	 [02MirahezeMagic] 07paladox closed pull request 03#243: Remove datadump-desc - 13https://git.io/JYMJ5
[18:26:53] <Not-818c>	 [02miraheze/MirahezeMagic] 07paladox pushed 033 commits to 03master [+0/-0/±4] 13https://git.io/JYMJx
[18:26:54] <Not-818c>	 [02miraheze/MirahezeMagic] 07paladox 03c5d66f1 - Merge pull request #243 from miraheze/paladox-patch-2
[18:26:56] <Not-818c>	 [02miraheze/MirahezeMagic] 07paladox deleted branch 03paladox-patch-2
[18:26:57] <Not-818c>	 [02MirahezeMagic] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/fQRGX
[18:27:20] <miraheze-github>	 miraheze/MirahezeMagic - paladox the build passed.
[18:27:42] <Not-818c>	 [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JYMUJ
[18:27:44] <Not-818c>	 [02miraheze/mediawiki] 07paladox 03fbddaf1 - Update MM
[18:27:44] <miraheze-github>	 miraheze/MirahezeMagic - paladox the build passed.
[18:27:54] <miraheze-github>	 miraheze/MirahezeMagic - paladox the build passed.
[18:29:18] <Reception123>	 paladox: thanks for fixing :)
[18:30:39] <dmehus>	 thanks paladox :)
[18:35:40] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.58, 7.10, 6.35
[18:36:54] <icinga-miraheze>	 RECOVERY - wiki.mlpwiki.net - reverse DNS on sslhost is OK: rDNS OK - wiki.mlpwiki.net reverse DNS resolves to cp11.miraheze.org
[18:37:39] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.51, 6.10, 6.07
[18:51:21] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.72, 6.37, 5.91
[18:52:55] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.90, 6.45, 5.86
[18:54:54] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 3.94, 5.56, 5.61
[18:55:15] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.10, 5.75, 5.81
[18:56:21] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.58, 7.08, 6.10
[19:00:20] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.96, 5.88, 5.83
[19:03:04] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.44, 7.22, 6.43
[19:04:15] <icinga-miraheze>	 PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.57, 7.53, 6.53
[19:04:59] <icinga-miraheze>	 PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.36, 6.49, 6.01
[19:06:13] <icinga-miraheze>	 RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.53, 6.76, 6.36
[19:06:56] <icinga-miraheze>	 RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.17, 6.18, 5.94
[19:14:35] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.17, 6.69, 6.68
[19:15:55] <icinga-miraheze>	 PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.94, 6.87, 6.04
[19:17:54] <icinga-miraheze>	 RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.52, 5.83, 5.75
[19:38:13] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.47, 6.37, 5.76
[19:40:13] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.56, 5.83, 5.65
[19:42:30] <Not-818c>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JYM3J
[19:42:31] <Not-818c>	 [02miraheze/puppet] 07paladox 038e19c93 - nginx: Add test4 ips temporarily to set_real_ip_from
[19:52:29] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 4.40, 4.24, 2.81
[19:54:27] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 0.94, 3.04, 2.54
[20:02:16] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 4.05, 5.00, 3.50
[20:02:50] <icinga-miraheze>	 PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 19.73, 21.83, 18.50
[20:04:14] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 1.11, 3.51, 3.13
[20:06:13] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 1.10, 2.75, 2.89
[20:06:47] <icinga-miraheze>	 RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.01, 19.52, 18.33
[21:16:54] <icinga-miraheze>	 PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.58, 6.25, 5.51
[21:18:51] <icinga-miraheze>	 RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.47, 6.01, 5.52
[21:44:29] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 3.00, 6.27, 3.73
[21:48:29] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 2.22, 3.82, 3.30
[21:50:29] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 1.00, 2.84, 3.00
[22:26:52] <MH-Discord>	 <Lake> dear lord, another cargo bug now. But this one I think it's related to the MH setup
[22:33:27] <MH-Discord>	 <Lake> it seems like jobs are going very slow, so it's causing this effect 🤔
[22:46:37] <dmehus>	 Lake, that could be possible that jobs are still catching up from the massive delete page script that was run earlier
[22:46:58] <MH-Discord>	 <Lake> oh
[22:47:38] <MH-Discord>	 <Lake> I noticed now that my table filled up to the 360 rows again, but the new page I made wasn't stored again. I will wait and see what happens
[23:13:28] <icinga-miraheze>	 PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 2.16, 3.77, 2.54
[23:15:28] <icinga-miraheze>	 RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 0.60, 2.63, 2.26
[23:40:44] <icinga-miraheze>	 PROBLEM - cp11 Current Load on cp11 is CRITICAL: CRITICAL - load average: 3.59, 4.40, 2.13
[23:42:44] <icinga-miraheze>	 RECOVERY - cp11 Current Load on cp11 is OK: OK - load average: 0.65, 3.01, 1.89
[23:46:44] <icinga-miraheze>	 PROBLEM - cp11 Current Load on cp11 is CRITICAL: CRITICAL - load average: 1.07, 4.82, 3.27
[23:48:44] <icinga-miraheze>	 RECOVERY - cp11 Current Load on cp11 is OK: OK - load average: 0.22, 3.25, 2.88