[00:02:42] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 32.44, 25.05, 19.56 [00:04:39] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.46, 21.05, 18.77 [00:06:37] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.46, 19.12, 18.31 [00:45:12] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJNPv [00:45:14] [02miraheze/services] 07MirahezeSSLBot 039125a06 - BOT: Updating services config for wikis [01:20:34] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.92, 18.07, 16.34 [01:22:35] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 18.11, 17.98, 16.51 [01:52:34] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.21, 18.62, 16.48 [01:54:34] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.31, 21.23, 17.73 [01:56:41] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.14, 19.36, 17.49 [02:06:14] Alert to Miraheze Staff: It looks like the icinga-miraheze bot has stopped! Ping !sre. [02:06:14] https://meta.miraheze.org is 03UP03 [02:06:15] [ Meta ] - meta.miraheze.org [02:06:16] https://icinga.miraheze.org is 03UP03 [02:06:17] [ Icinga Web 2 Login ] - icinga.miraheze.org [02:12:38] PROBLEM - wiki.warfra.ml - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:40:44] PROBLEM - bacula2 Bacula Static on bacula2 is UNKNOWN: NRPE: Command 'check_bacula_static' not defined [02:41:56] !log set puppet2 cores to 3 [02:42:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:44:40] PROBLEM - puppet2 puppetserver on puppet2 is CRITICAL: connect to address 51.89.160.129 and port 8140: Connection refused [02:44:41] PROBLEM - rdb1 Puppet on rdb1 is CRITICAL: CRITICAL: Puppet has 7 failures. Last run 2 minutes ago with 7 failures. Failed resources (up to 3 shown) [02:44:49] PROBLEM - db7 Puppet on db7 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 2 minutes ago with 10 failures. Failed resources (up to 3 shown) [02:44:51] PROBLEM - services1 Puppet on services1 is CRITICAL: CRITICAL: Puppet has 17 failures. Last run 2 minutes ago with 17 failures. Failed resources (up to 3 shown) [02:44:52] PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Puppet has 7 failures. Last run 2 minutes ago with 7 failures. Failed resources (up to 3 shown) [02:44:53] PROBLEM - ldap1 Puppet on ldap1 is CRITICAL: CRITICAL: Puppet has 7 failures. Last run 2 minutes ago with 7 failures. Failed resources (up to 3 shown) [02:44:58] PROBLEM - services2 Puppet on services2 is CRITICAL: CRITICAL: Puppet has 15 failures. Last run 2 minutes ago with 15 failures. Failed resources (up to 3 shown) [02:44:59] PROBLEM - cloud2 Puppet on cloud2 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 2 minutes ago with 8 failures. Failed resources (up to 3 shown) [02:45:11] PROBLEM - rdb2 Puppet on rdb2 is CRITICAL: CRITICAL: Puppet has 7 failures. Last run 2 minutes ago with 7 failures. Failed resources (up to 3 shown) [02:45:11] PROBLEM - cp6 Puppet on cp6 is CRITICAL: CRITICAL: Puppet has 522 failures. Last run 2 minutes ago with 522 failures. Failed resources (up to 3 shown) [02:45:36] PROBLEM - puppet2 Puppet on puppet2 is CRITICAL: CRITICAL: Puppet has 24 failures. Last run 3 minutes ago with 24 failures. Failed resources (up to 3 shown) [02:45:39] PROBLEM - db11 Puppet on db11 is CRITICAL: CRITICAL: Puppet has 7 failures. Last run 3 minutes ago with 7 failures. Failed resources (up to 3 shown) [02:45:42] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 20 failures. Last run 3 minutes ago with 20 failures. Failed resources (up to 3 shown) [02:45:43] PROBLEM - phab1 Puppet on phab1 is CRITICAL: CRITICAL: Puppet has 16 failures. Last run 3 minutes ago with 16 failures. Failed resources (up to 3 shown) [02:45:50] PROBLEM - mon1 Puppet on mon1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [02:45:54] PROBLEM - mw5 Puppet on mw5 is CRITICAL: CRITICAL: Puppet has 533 failures. Last run 2 minutes ago with 533 failures. Failed resources (up to 3 shown) [02:45:55] PROBLEM - mw4 Puppet on mw4 is CRITICAL: CRITICAL: Puppet has 290 failures. Last run 2 minutes ago with 290 failures. Failed resources (up to 3 shown) [02:46:06] PROBLEM - cloud3 Puppet on cloud3 is CRITICAL: CRITICAL: Puppet has 5 failures. Last run 3 minutes ago with 5 failures. Failed resources (up to 3 shown) [02:46:06] PROBLEM - mail1 Puppet on mail1 is CRITICAL: CRITICAL: Puppet has 31 failures. Last run 3 minutes ago with 31 failures. Failed resources (up to 3 shown) [02:46:06] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 3 minutes ago with 8 failures. Failed resources (up to 3 shown) [02:46:11] PROBLEM - db13 Puppet on db13 is CRITICAL: CRITICAL: Puppet has 7 failures. Last run 3 minutes ago with 7 failures. Failed resources (up to 3 shown) [02:46:17] PROBLEM - db12 Puppet on db12 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 3 minutes ago with 8 failures. Failed resources (up to 3 shown) [02:46:17] PROBLEM - cp7 Puppet on cp7 is CRITICAL: CRITICAL: Puppet has 521 failures. Last run 2 minutes ago with 521 failures. Failed resources (up to 3 shown) [02:46:27] PROBLEM - jobrunner2 Puppet on jobrunner2 is CRITICAL: CRITICAL: Puppet has 535 failures. Last run 2 minutes ago with 535 failures. Failed resources (up to 3 shown) [02:46:32] PROBLEM - bacula2 Puppet on bacula2 is CRITICAL: CRITICAL: Puppet has 4 failures. Last run 3 minutes ago with 4 failures. Failed resources (up to 3 shown) [02:46:38] RECOVERY - puppet2 puppetserver on puppet2 is OK: TCP OK - 0.000 second response time on 51.89.160.129 port 8140 [02:46:38] PROBLEM - gluster2 Puppet on gluster2 is CRITICAL: CRITICAL: Puppet has 9 failures. Last run 3 minutes ago with 9 failures. Failed resources (up to 3 shown) [02:46:40] PROBLEM - mw7 Puppet on mw7 is CRITICAL: CRITICAL: Puppet has 536 failures. Last run 2 minutes ago with 536 failures. Failed resources (up to 3 shown) [02:47:14] PROBLEM - gluster1 Puppet on gluster1 is CRITICAL: CRITICAL: Puppet has 11 failures. Last run 3 minutes ago with 11 failures. Failed resources (up to 3 shown) [02:47:33] PROBLEM - jobrunner1 Puppet on jobrunner1 is CRITICAL: CRITICAL: Puppet has 545 failures. Last run 3 minutes ago with 545 failures. Failed resources (up to 3 shown) [02:47:35] RECOVERY - puppet2 Puppet on puppet2 is OK: OK: Puppet is currently enabled, last run 25 seconds ago with 0 failures [02:47:46] PROBLEM - mw6 Puppet on mw6 is CRITICAL: CRITICAL: Puppet has 534 failures. Last run 3 minutes ago with 534 failures. Failed resources (up to 3 shown) [02:48:17] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJNMK [02:48:18] [02miraheze/ssl] 07paladox 03f0573af - Update wiki.iufs.jp.crt [02:48:51] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 246 failures. Last run 2 minutes ago with 246 failures. Failed resources (up to 3 shown): File[/etc/rsyslog.conf],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades],File[/etc/apt/apt.conf.d/20auto-upgrades] [02:48:56] PROBLEM - cp9 Puppet on cp9 is CRITICAL: CRITICAL: Puppet has 519 failures. Last run 3 minutes ago with 519 failures. Failed resources (up to 3 shown): File[wildcard.miraheze.org],File[wildcard.miraheze.org_private],File[/etc/default/stunnel4],File[/etc/stunnel/mediawiki.conf] [02:49:33] PROBLEM - cloud1 Puppet on cloud1 is CRITICAL: CRITICAL: Puppet has 6 failures. Last run 3 minutes ago with 6 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/proxmox.gpg],File[/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg],File[/etc/network/interfaces.d/50-cloud-init.cfg],File[/etc/logrotate.d/puppet] [02:51:10] RECOVERY - test2 Puppet on test2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:52:07] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JJNMM [02:52:09] [02miraheze/ssl] 07paladox 035509ce0 - Remove wiki.warfra.ml Domain not pointing at us. [02:52:10] [02ssl] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vxP9L [02:52:26] RECOVERY - rdb1 Puppet on rdb1 is OK: OK: Puppet is currently enabled, last run 5 seconds ago with 0 failures [02:52:32] RECOVERY - bacula2 Puppet on bacula2 is OK: OK: Puppet is currently enabled, last run 8 seconds ago with 0 failures [02:52:37] [02ssl] 07paladox opened pull request 03#350: Remove wiki.warfra.ml - 13https://github.com/miraheze/ssl/pull/350 [02:52:38] RECOVERY - services1 Puppet on services1 is OK: OK: Puppet is currently enabled, last run 12 seconds ago with 0 failures [02:52:39] RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 24 seconds ago with 0 failures [02:52:41] RECOVERY - db7 Puppet on db7 is OK: OK: Puppet is currently enabled, last run 21 seconds ago with 0 failures [02:52:54] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-1/±0] 13https://git.io/JJNMS [02:52:56] [02miraheze/ssl] 07paladox 03a347e48 - Delete wiki.warfra.ml.crt [02:52:57] [02ssl] 07paladox synchronize pull request 03#350: Remove wiki.warfra.ml - 13https://git.io/JJNMy [02:52:57] RECOVERY - cloud1 Puppet on cloud1 is OK: OK: Puppet is currently enabled, last run 34 seconds ago with 0 failures [02:52:58] RECOVERY - db11 Puppet on db11 is OK: OK: Puppet is currently enabled, last run 36 seconds ago with 0 failures [02:53:03] [02ssl] 07paladox closed pull request 03#350: Remove wiki.warfra.ml - 13https://git.io/JJNMy [02:53:04] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 41 seconds ago with 0 failures [02:53:05] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±1] 13https://git.io/JJNM9 [02:53:06] [02miraheze/ssl] 07paladox 03370ce3a - Remove wiki.warfra.ml (#350) * Remove wiki.warfra.ml Domain not pointing at us. * Delete wiki.warfra.ml.crt [02:53:08] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-2 [02:53:09] [02ssl] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vxP9L [02:53:09] RECOVERY - jobrunner1 Puppet on jobrunner1 is OK: OK: Puppet is currently enabled, last run 4 seconds ago with 0 failures [02:53:15] RECOVERY - services2 Puppet on services2 is OK: OK: Puppet is currently enabled, last run 51 seconds ago with 0 failures [02:53:19] RECOVERY - gluster1 Puppet on gluster1 is OK: OK: Puppet is currently enabled, last run 39 seconds ago with 0 failures [02:53:21] RECOVERY - cloud2 Puppet on cloud2 is OK: OK: Puppet is currently enabled, last run 56 seconds ago with 0 failures [02:53:25] RECOVERY - ldap1 Puppet on ldap1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:53:33] RECOVERY - phab1 Puppet on phab1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:53:38] RECOVERY - cp6 Puppet on cp6 is OK: OK: Puppet is currently enabled, last run 46 seconds ago with 0 failures [02:53:39] RECOVERY - rdb2 Puppet on rdb2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:53:43] RECOVERY - mon1 Puppet on mon1 is OK: OK: Puppet is currently enabled, last run 59 seconds ago with 0 failures [02:53:46] RECOVERY - mw6 Puppet on mw6 is OK: OK: Puppet is currently enabled, last run 34 seconds ago with 0 failures [02:53:50] RECOVERY - mw4 Puppet on mw4 is OK: OK: Puppet is currently enabled, last run 40 seconds ago with 0 failures [02:53:59] RECOVERY - mw5 Puppet on mw5 is OK: OK: Puppet is currently enabled, last run 52 seconds ago with 0 failures [02:54:01] RECOVERY - mail1 Puppet on mail1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:54:06] RECOVERY - cloud3 Puppet on cloud3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:54:07] RECOVERY - db13 Puppet on db13 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:54:08] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:54:18] RECOVERY - db12 Puppet on db12 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:54:18] RECOVERY - cp7 Puppet on cp7 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:54:27] RECOVERY - jobrunner2 Puppet on jobrunner2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:54:35] RECOVERY - gluster2 Puppet on gluster2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:54:39] PROBLEM - bacula2 Bacula Static 1 on bacula2 is CRITICAL: CRITICAL: no terminated jobs [02:54:40] PROBLEM - bacula2 Bacula Static 2 on bacula2 is CRITICAL: CRITICAL: no terminated jobs [02:54:40] RECOVERY - mw7 Puppet on mw7 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:55:16] RECOVERY - wiki.iufs.jp - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.iufs.jp' will expire on Tue 17 Nov 2020 01:47:47 GMT +0000. [02:55:21] RECOVERY - cp9 Puppet on cp9 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:55:24] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 11 seconds ago with 0 failures [02:56:31] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.24, 21.77, 18.66 [02:57:18] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JJNDm [02:57:19] [02miraheze/ssl] 07paladox 0375152f9 - Remove wiki.veloren.net Domain not pointing at us. [02:57:21] [02ssl] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vxP9L [02:57:22] [02ssl] 07paladox opened pull request 03#351: Remove wiki.veloren.net - 13https://git.io/JJNDY [02:57:38] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-1/±0] 13https://git.io/JJNDO [02:57:40] [02miraheze/ssl] 07paladox 031b33080 - Delete wiki.veloren.net.crt [02:57:41] [02ssl] 07paladox synchronize pull request 03#351: Remove wiki.veloren.net - 13https://git.io/JJNDY [02:57:46] [02ssl] 07paladox closed pull request 03#351: Remove wiki.veloren.net - 13https://git.io/JJNDY [02:57:48] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±1] 13https://git.io/JJND3 [02:57:49] [02miraheze/ssl] 07paladox 03f9acb34 - Remove wiki.veloren.net (#351) * Remove wiki.veloren.net Domain not pointing at us. * Delete wiki.veloren.net.crt [02:57:51] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-2 [02:57:52] [02ssl] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vxP9L [02:58:43] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JJNDn [02:58:44] [02miraheze/ssl] 07paladox 0316d175d - Remove wiki.earthmcbe.space Domain not pointing at us. [02:58:46] [02ssl] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vxP9L [02:58:47] [02ssl] 07paladox opened pull request 03#352: Remove wiki.earthmcbe.space - 13https://git.io/JJNDc [02:59:05] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-1/±0] 13https://git.io/JJNDC [02:59:06] [02miraheze/ssl] 07paladox 03dedd2cc - Delete wiki.earthmcbe.space.crt [02:59:07] [02ssl] 07paladox synchronize pull request 03#352: Remove wiki.earthmcbe.space - 13https://git.io/JJNDc [02:59:15] [02ssl] 07paladox closed pull request 03#352: Remove wiki.earthmcbe.space - 13https://git.io/JJNDc [02:59:17] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±1] 13https://git.io/JJNDW [02:59:18] [02miraheze/ssl] 07paladox 03195a3bd - Remove wiki.earthmcbe.space (#352) * Remove wiki.earthmcbe.space Domain not pointing at us. * Delete wiki.earthmcbe.space.crt [02:59:20] [02ssl] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vxP9L [02:59:21] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-2 [02:59:56] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JJND8 [02:59:58] [02miraheze/ssl] 07paladox 034b2dca6 - Remove wiki.ldmsys.net Domain not pointing at us. [03:00:00] [02ssl] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vxP9L [03:00:01] [02ssl] 07paladox opened pull request 03#353: Remove wiki.ldmsys.net - 13https://git.io/JJND4 [03:00:12] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-1/±0] 13https://git.io/JJNDR [03:00:14] [02miraheze/ssl] 07paladox 03697c43c - Delete wiki.ldmsys.net.crt [03:00:15] [02ssl] 07paladox synchronize pull request 03#353: Remove wiki.ldmsys.net - 13https://git.io/JJND4 [03:00:20] [02ssl] 07paladox closed pull request 03#353: Remove wiki.ldmsys.net - 13https://git.io/JJND4 [03:00:21] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±1] 13https://git.io/JJND0 [03:00:23] [02miraheze/ssl] 07paladox 03c710c94 - Remove wiki.ldmsys.net (#353) * Remove wiki.ldmsys.net Domain not pointing at us. * Delete wiki.ldmsys.net.crt [03:00:24] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-2 [03:00:26] [02ssl] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vxP9L [03:01:49] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JJNDa [03:01:51] [02miraheze/ssl] 07paladox 03f9a9e2d - Remove wiki.titan.land Domain not pointing at us. [03:01:52] [02ssl] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vxP9L [03:01:54] [02ssl] 07paladox opened pull request 03#354: Remove wiki.titan.land - 13https://git.io/JJNDV [03:02:05] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-1/±0] 13https://git.io/JJNDw [03:02:06] [02miraheze/ssl] 07paladox 039879b51 - Delete wiki.titan.land.crt [03:02:08] [02ssl] 07paladox synchronize pull request 03#354: Remove wiki.titan.land - 13https://git.io/JJNDV [03:02:14] [02ssl] 07paladox closed pull request 03#354: Remove wiki.titan.land - 13https://git.io/JJNDV [03:02:15] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±1] 13https://git.io/JJNDr [03:02:17] [02miraheze/ssl] 07paladox 034028b1e - Remove wiki.titan.land (#354) * Remove wiki.titan.land Domain not pointing at us. * Delete wiki.titan.land.crt [03:02:18] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-2 [03:02:20] [02ssl] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vxP9L [03:03:15] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JJND6 [03:03:16] [02miraheze/ssl] 07paladox 035d42095 - Remove museummiddelland.nl Domain not pointing at us. [03:03:18] [02ssl] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vxP9L [03:03:54] [02ssl] 07paladox opened pull request 03#355: Remove museummiddelland.nl - 13https://git.io/JJNDi [03:04:02] [02ssl] 07paladox closed pull request 03#355: Remove museummiddelland.nl - 13https://git.io/JJNDi [03:04:04] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-2 [03:04:05] [02ssl] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vxP9L [03:04:31] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JJNDX [03:04:33] [02miraheze/ssl] 07paladox 033ea5b8a - Remove museummiddelland.nl redirect [03:04:34] [02ssl] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vxP9L [03:04:36] [02ssl] 07paladox opened pull request 03#356: Remove museummiddelland.nl redirect - 13https://git.io/JJND1 [03:04:55] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 18.11, 20.39, 19.40 [03:06:28] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JJNDS [03:06:30] [02miraheze/ssl] 07paladox 03c4a622a - Update www.museummiddelland.nl.crt [03:06:31] [02ssl] 07paladox synchronize pull request 03#356: Remove museummiddelland.nl redirect - 13https://git.io/JJND1 [03:22:54] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.92, 21.14, 19.35 [03:24:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.41, 22.89, 20.23 [03:26:46] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 17.11, 20.78, 19.78 [03:28:42] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.37, 18.60, 19.08 [03:29:26] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-7 [03:29:28] [02puppet] 07paladox deleted branch 03paladox-patch-7 - 13https://git.io/vbiAS [03:29:29] [02puppet] 07paladox deleted branch 03paladox-patch-6 - 13https://git.io/vbiAS [03:29:31] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-6 [03:29:34] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-10 [03:29:35] [02puppet] 07paladox deleted branch 03paladox-patch-10 - 13https://git.io/vbiAS [03:29:37] [02puppet] 07paladox deleted branch 03revert-1466-patch-15 - 13https://git.io/vbiAS [03:29:38] [02miraheze/puppet] 07paladox deleted branch 03revert-1466-patch-15 [03:44:29] !log upgrade icingaweb2 on mon1 [03:44:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:52:54] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.79, 20.92, 19.46 [03:54:50] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.73, 19.05, 18.96 [04:17:09] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.12, 21.22, 20.17 [04:19:04] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.27, 19.31, 19.61 [04:22:58] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.14, 20.72, 20.11 [04:24:54] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 19.88, 20.26, 20.00 [04:32:43] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.25, 20.66, 20.18 [04:33:26] Hello wq2d! If you have any questions, feel free to ask and someone should answer soon. [04:33:44] Hello [04:34:06] Bye. [04:34:39] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 17.93, 19.73, 19.91 [04:42:27] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.02, 20.56, 19.77 [04:44:23] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.55, 18.66, 19.18 [05:01:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.49, 20.51, 19.39 [05:03:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.90, 22.76, 20.39 [05:07:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 17.85, 21.52, 20.55 [05:09:52] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.38, 18.88, 19.69 [05:56:47] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.01, 20.71, 18.89 [05:58:42] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.56, 18.76, 18.42 [06:02:35] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.75, 21.66, 19.81 [06:04:31] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.71, 22.10, 20.10 [06:06:26] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.86, 22.29, 20.38 [06:07:50] PROBLEM - gluster1 Disk Space on gluster1 is WARNING: DISK WARNING - free space: / 75645 MB (10% inode=84%); [06:14:13] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.59, 22.57, 21.09 [06:16:11] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.46, 21.76, 20.96 [06:20:04] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 17.38, 19.17, 20.05 [06:23:56] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.85, 21.65, 20.89 [06:25:48] RECOVERY - gluster1 Disk Space on gluster1 is OK: DISK OK - free space: / 75673 MB (11% inode=84%); [06:25:54] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.16, 22.11, 21.08 [06:27:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.31, 21.98, 21.21 [06:31:51] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.18, 22.21, 21.41 [06:33:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.00, 22.89, 21.82 [06:37:51] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.18, 22.86, 21.99 [06:39:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.33, 21.80, 21.76 [06:41:52] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.16, 22.56, 21.99 [06:43:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 17.63, 20.82, 21.45 [06:47:50] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.43, 17.75, 20.09 [06:51:47] PROBLEM - gluster1 Disk Space on gluster1 is WARNING: DISK WARNING - free space: / 75657 MB (10% inode=84%); [06:53:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.99, 21.63, 20.95 [06:55:50] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 17.03, 19.77, 20.35 [06:59:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.18, 21.71, 20.98 [07:01:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.31, 21.41, 20.96 [07:07:51] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.66, 19.41, 20.33 [07:32:39] PROBLEM - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is CRITICAL: CRITICAL - NGINX Error Rate is 95% [07:32:50] Oh ffs [07:33:01] I wanted a relaxing last day before things get busy [07:33:17] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 3 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb [07:33:28] PROBLEM - cp9 HTTP 4xx/5xx ERROR Rate on cp9 is CRITICAL: CRITICAL - NGINX Error Rate is 78% [07:33:39] PROBLEM - ping6 on cp9 is WARNING: PING WARNING - Packet loss = 66%, RTA = 78.17 ms [07:33:44] PROBLEM - ping4 on cp9 is WARNING: PING WARNING - Packet loss = 58%, RTA = 78.24 ms [07:34:04] PROBLEM - cp3 Current Load on cp3 is CRITICAL: CRITICAL - load average: 5.47, 3.67, 1.78 [07:34:23] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.05, 21.00, 18.98 [07:34:39] PROBLEM - cp7 HTTP 4xx/5xx ERROR Rate on cp7 is CRITICAL: CRITICAL - NGINX Error Rate is 69% [07:34:48] PROBLEM - cp9 Current Load on cp9 is CRITICAL: CRITICAL - load average: 3.47, 2.38, 1.18 [07:34:59] PROBLEM - cp3 HTTPS on cp3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:35:12] PROBLEM - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is CRITICAL: CRITICAL - NGINX Error Rate is 83% [07:35:39] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb [07:35:48] RECOVERY - ping6 on cp9 is OK: PING OK - Packet loss = 0%, RTA = 79.04 ms [07:35:54] PROBLEM - cp9 Puppet on cp9 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [07:35:55] RECOVERY - ping4 on cp9 is OK: PING OK - Packet loss = 0%, RTA = 77.01 ms [07:36:24] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.84, 23.22, 20.08 [07:37:05] RECOVERY - cp3 HTTPS on cp3 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 1948 bytes in 9.966 second response time [07:37:15] RECOVERY - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is OK: OK - NGINX Error Rate is 33% [07:37:35] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [07:37:42] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [07:38:29] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 17.25, 22.16, 20.18 [07:38:37] PROBLEM - cp7 HTTP 4xx/5xx ERROR Rate on cp7 is WARNING: WARNING - NGINX Error Rate is 57% [07:40:30] PROBLEM - ping4 on cp9 is WARNING: PING WARNING - Packet loss = 44%, RTA = 75.98 ms [07:40:34] PROBLEM - cp7 HTTP 4xx/5xx ERROR Rate on cp7 is CRITICAL: CRITICAL - NGINX Error Rate is 71% [07:41:12] PROBLEM - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is CRITICAL: CRITICAL - NGINX Error Rate is 83% [07:41:36] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb [07:41:44] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 2400:6180:0:d0::403:f001/cpweb, 51.222.27.129/cpweb [07:42:26] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 33.47, 25.27, 21.60 [07:42:30] PROBLEM - cp7 HTTP 4xx/5xx ERROR Rate on cp7 is WARNING: WARNING - NGINX Error Rate is 53% [07:42:35] RECOVERY - ping4 on cp9 is OK: PING OK - Packet loss = 0%, RTA = 76.37 ms [07:44:17] RECOVERY - cp9 Puppet on cp9 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [07:44:31] PROBLEM - cp7 HTTP 4xx/5xx ERROR Rate on cp7 is CRITICAL: CRITICAL - NGINX Error Rate is 83% [07:45:18] PROBLEM - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is WARNING: WARNING - NGINX Error Rate is 50% [07:46:22] PROBLEM - ping4 on cp6 is WARNING: PING WARNING - Packet loss = 58%, RTA = 0.63 ms [07:46:46] PROBLEM - cp7 HTTP 4xx/5xx ERROR Rate on cp7 is WARNING: WARNING - NGINX Error Rate is 51% [07:47:16] PROBLEM - cp9 Current Load on cp9 is WARNING: WARNING - load average: 0.91, 1.91, 1.80 [07:47:38] PROBLEM - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is CRITICAL: CRITICAL - NGINX Error Rate is 69% [07:47:58] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [07:48:14] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [07:48:56] PROBLEM - cp7 HTTP 4xx/5xx ERROR Rate on cp7 is CRITICAL: CRITICAL - NGINX Error Rate is 77% [07:49:27] RECOVERY - cp9 Current Load on cp9 is OK: OK - load average: 0.87, 1.57, 1.69 [07:50:36] RECOVERY - ping4 on cp6 is OK: PING OK - Packet loss = 0%, RTA = 3.30 ms [07:51:44] PROBLEM - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is WARNING: WARNING - NGINX Error Rate is 56% [07:52:18] PROBLEM - ping6 on cp3 is WARNING: PING WARNING - Packet loss = 28%, RTA = 174.22 ms [07:53:42] PROBLEM - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is CRITICAL: CRITICAL - NGINX Error Rate is 68% [07:54:45] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.54, 23.88, 23.31 [07:56:08] PROBLEM - ping4 on cp3 is WARNING: PING WARNING - Packet loss = 37%, RTA = 164.99 ms [07:58:08] RECOVERY - ping4 on cp3 is OK: PING OK - Packet loss = 0%, RTA = 155.75 ms [07:58:50] PROBLEM - cp7 HTTP 4xx/5xx ERROR Rate on cp7 is WARNING: WARNING - NGINX Error Rate is 58% [08:00:22] RECOVERY - ping6 on cp3 is OK: PING OK - Packet loss = 0%, RTA = 171.56 ms [08:00:53] RECOVERY - cp7 HTTP 4xx/5xx ERROR Rate on cp7 is OK: OK - NGINX Error Rate is 21% [08:01:02] RECOVERY - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is OK: OK - NGINX Error Rate is 12% [08:02:37] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.70, 22.92, 22.62 [08:03:26] PROBLEM - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is WARNING: WARNING - NGINX Error Rate is 53% [08:04:13] PROBLEM - cp9 HTTP 4xx/5xx ERROR Rate on cp9 is WARNING: WARNING - NGINX Error Rate is 50% [08:04:32] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.39, 23.32, 22.87 [08:04:51] PROBLEM - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is CRITICAL: CRITICAL - NGINX Error Rate is 69% [08:05:25] PROBLEM - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is CRITICAL: CRITICAL - NGINX Error Rate is 75% [08:06:11] PROBLEM - cp9 HTTP 4xx/5xx ERROR Rate on cp9 is CRITICAL: CRITICAL - NGINX Error Rate is 93% [08:06:47] PROBLEM - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is WARNING: WARNING - NGINX Error Rate is 55% [08:07:21] PROBLEM - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is WARNING: WARNING - NGINX Error Rate is 44% [08:07:42] PROBLEM - cp7 HTTP 4xx/5xx ERROR Rate on cp7 is WARNING: WARNING - NGINX Error Rate is 52% [08:08:11] RECOVERY - cp9 HTTP 4xx/5xx ERROR Rate on cp9 is OK: OK - NGINX Error Rate is 9% [08:08:19] PROBLEM - cp3 Current Load on cp3 is WARNING: WARNING - load average: 0.30, 1.22, 1.93 [08:08:41] RECOVERY - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is OK: OK - NGINX Error Rate is 15% [08:09:21] RECOVERY - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is OK: OK - NGINX Error Rate is 14% [08:09:38] RECOVERY - cp7 HTTP 4xx/5xx ERROR Rate on cp7 is OK: OK - NGINX Error Rate is 7% [08:10:23] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.43, 21.39, 21.93 [08:12:22] RECOVERY - cp3 Current Load on cp3 is OK: OK - load average: 0.51, 0.85, 1.61 [08:13:01] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 128.199.139.216/cpweb [08:14:57] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [08:16:13] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.00, 22.48, 22.85 [08:18:11] PROBLEM - cp9 HTTP 4xx/5xx ERROR Rate on cp9 is CRITICAL: CRITICAL - NGINX Error Rate is 93% [08:18:22] PROBLEM - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is CRITICAL: CRITICAL - NGINX Error Rate is 91% [08:18:28] PROBLEM - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is CRITICAL: CRITICAL - NGINX Error Rate is 79% [08:18:34] PROBLEM - cp7 HTTP 4xx/5xx ERROR Rate on cp7 is CRITICAL: CRITICAL - NGINX Error Rate is 85% [08:20:12] RECOVERY - cp9 HTTP 4xx/5xx ERROR Rate on cp9 is OK: OK - NGINX Error Rate is 7% [08:20:17] RECOVERY - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is OK: OK - NGINX Error Rate is 5% [08:20:24] RECOVERY - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is OK: OK - NGINX Error Rate is 12% [08:20:33] RECOVERY - cp7 HTTP 4xx/5xx ERROR Rate on cp7 is OK: OK - NGINX Error Rate is 4% [08:35:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.46, 23.26, 22.50 [08:37:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.79, 21.56, 21.98 [08:45:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.12, 23.71, 22.68 [08:47:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 16.26, 20.54, 21.62 [08:51:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.74, 23.50, 22.42 [08:53:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.86, 20.70, 21.54 [08:59:52] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 30.75, 22.13, 21.54 [09:03:52] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 17.32, 21.56, 21.60 [09:09:51] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.25, 21.82, 21.30 [09:15:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.89, 21.60, 21.59 [09:21:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 31.49, 23.20, 21.90 [09:23:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.92, 21.54, 21.45 [09:25:51] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 28.02, 22.59, 21.77 [09:37:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.68, 22.42, 22.49 [09:49:50] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 12.18, 17.11, 19.69 [09:56:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.65, 21.89, 21.07 [09:58:47] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 33.54, 24.95, 22.21 [10:01:15] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 10.47, 13.09, 7.10 [10:05:14] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 1.77, 6.76, 5.85 [10:08:32] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.59, 22.19, 23.41 [10:16:14] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.22, 23.47, 23.24 [10:18:10] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.59, 22.28, 22.84 [10:22:03] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.51, 22.09, 22.48 [10:23:59] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.77, 22.82, 22.71 [10:30:20] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJAvX [10:30:22] [02miraheze/services] 07MirahezeSSLBot 032d620a9 - BOT: Updating services config for wikis [10:31:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.44, 21.02, 21.40 [10:33:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.01, 20.78, 21.27 [10:39:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.79, 22.59, 21.76 [10:43:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.80, 23.51, 22.48 [10:51:51] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.64, 22.50, 22.02 [10:53:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.51, 22.23, 22.00 [10:55:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 30.56, 24.98, 22.99 [11:09:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.34, 22.73, 23.73 [11:15:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.76, 23.56, 23.65 [11:17:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.13, 21.75, 22.99 [11:25:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 30.11, 23.51, 23.10 [11:29:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.04, 22.71, 22.86 [11:31:51] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.96, 24.67, 23.55 [11:39:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.40, 23.93, 23.85 [11:41:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 28.47, 24.35, 23.94 [11:43:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.30, 23.91, 23.89 [11:55:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 31.30, 24.71, 23.22 [11:57:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.99, 23.03, 22.81 [12:01:53] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 33.29, 25.52, 23.65 [12:09:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.66, 21.83, 23.64 [12:11:53] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 34.22, 26.31, 25.03 [12:17:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.75, 20.93, 23.16 [12:19:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.41, 22.68, 23.48 [12:19:55] Where is located CommonSettings.php [12:20:00] ? [12:23:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.49, 22.83, 23.55 [12:36:30] @MrJaroslavik: doesn't exist [12:36:43] Oh [12:37:49] @MrJaroslavik: What you looking for in it? [12:45:50] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.54, 18.69, 20.22 [12:51:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.02, 20.54, 20.44 [12:53:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.32, 23.65, 21.64 [12:55:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.15, 21.59, 21.14 [12:59:51] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.09, 18.54, 20.07 [13:08:40] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 31.18, 23.48, 21.40 [13:16:23] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.03, 22.80, 22.42 [13:18:18] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 23.26, 24.62, 23.21 [13:23:31] RhinosF1 where should be added $wgLogSpamBlacklistHits = true? [13:23:39] LocalExtensions.php or LS.php? [13:26:26] LS.php [13:26:34] Ok [13:42:16] RhinosF1 can you please enable it? I'm not sure how to insert it :( [13:42:21] Because parentheses [13:42:41] Phab task? [13:43:12] Ok, will create [13:51:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.88, 22.22, 23.93 [14:01:51] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 28.20, 24.61, 24.09 [14:03:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.74, 22.05, 23.21 [14:07:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.34, 22.85, 23.18 [14:09:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.18, 22.58, 23.03 [14:11:52] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 29.05, 24.61, 23.71 [14:29:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.03, 20.88, 23.14 [14:41:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 28.50, 23.79, 23.13 [14:49:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 16.69, 22.13, 22.98 [14:51:51] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.57, 24.52, 23.79 [15:13:15] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 9.51, 8.66, 5.24 [15:15:16] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 3.38, 6.71, 4.95 [15:35:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 14.16, 20.69, 23.43 [15:41:51] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.72, 19.79, 21.81 [15:43:51]