[00:03:34] RECOVERY - mw6 Disk Space on mw6 is OK: DISK OK - free space: / 2602 MB (13% inode=75%); [00:03:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.61, 21.42, 17.02 [00:05:49] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.00, 18.30, 16.39 [01:04:11] PROBLEM - ns1 Current Load on ns1 is CRITICAL: CRITICAL - load average: 3.74, 3.36, 1.57 [01:06:09] RECOVERY - ns1 Current Load on ns1 is OK: OK - load average: 0.09, 1.60, 1.25 [07:14:17] !log rm /srv/mediawiki/w/cache/csydeswiki.json on mw* and jbr* [07:14:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:28:10] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JTNc4 [07:28:12] [02miraheze/mw-config] 07Reception123 038b543a9 - comment out jsonconfig for the time being (error with wiki) [07:34:29] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JTNcb [07:34:30] [02miraheze/mw-config] 07Reception123 03127a3d7 - bring back jsonconfig settings [07:39:16] !log $cWJ = new CreateWikiJson( 'csydeswiki' ); $cWJ->resetWiki(); [07:39:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:43:30] !log sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildmessages.php --wiki loginwiki on mw* and jbr* [07:43:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:47:34] PROBLEM - mw6 Disk Space on mw6 is WARNING: DISK WARNING - free space: / 2086 MB (10% inode=75%); [09:11:08] !log update cw_wikis set wiki_dbcluster = 'c4' where wiki_dbname = 'csydeswiki'; (couldn't use script as a fatal was occuring) [09:11:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:11:56] !log ignore the above, appeared to have worked (was a warning) [09:12:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:15:25] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JTNE0 [09:15:27] [02miraheze/mw-config] 07paladox 03a838ea9 - QUICK HACK: $wgMessageCacheType change to NONE I found this fixes "Fatal exception of type "LogicException" (we can revert after) [09:21:37] !log cd /srv/mediawiki/w/cache && rm databases.json ; sudo -u www-data php /srv/mediawiki/w/maint*/mergeMessageFileList.php --output /srv/mediawiki/config/ExtensionMessageFiles.php --wiki loginwiki on mw* (simply because the file wasn't updating thus csydeswiki using another dbcluster) [09:21:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:23:27] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JTNuI [09:23:28] [02miraheze/mw-config] 07paladox 0373c3363 - Revert "QUICK HACK: $wgMessageCacheType change to NONE" This reverts commit a838ea904452cd2e2b1f1b554c65036f743c9942. [09:32:08] !log cd /srv/mediawiki/w/cache && rm databases.json ; sudo -u www-data php /srv/mediawiki/w/maint*/mergeMessageFileList.php --output /srv/mediawiki/config/ExtensionMessageFiles.php --wiki loginwiki on jobrunner* (simply because the file wasn't updating thus csydeswiki using another dbcluster) [09:32:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:12:20] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.77, 19.67, 17.37 [12:14:16] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 17.77, 18.82, 17.32 [14:33:49] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.04, 19.85, 17.46 [14:35:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.22, 20.27, 17.87 [14:37:50] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 19.93, 19.29, 17.76 [14:51:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.23, 20.12, 18.54 [14:53:49] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.89, 18.61, 18.19 [15:03:49] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.91, 23.09, 20.17 [15:05:51] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.16, 19.66, 19.27 [15:34:36] PROBLEM - mw5 Disk Space on mw5 is WARNING: DISK WARNING - free space: / 2085 MB (10% inode=75%); [16:12:25] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.97, 21.00, 18.61 [16:14:20] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 18.09, 19.54, 18.33 [16:21:08] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JTAY3 [16:21:09] [02miraheze/mw-config] 07paladox 03868498b - Revert "Revert "QUICK HACK: $wgMessageCacheType change to NONE"" This reverts commit 73c3363a3259517b6d92a56e8dda1c2a5d97eaaa. [16:23:05] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JTAYB [16:23:06] [02miraheze/mw-config] 07paladox 03cea3011 - Revert "Revert "Revert "QUICK HACK: $wgMessageCacheType change to NONE""" This reverts commit 868498b2e5545ce39341282563df99be2ca41395. [16:24:12] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.93, 22.02, 19.73 [16:26:08] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 13.96, 19.03, 18.90 [16:27:40] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JTAY5 [16:27:41] [02miraheze/mw-config] 07paladox 03fc286a8 - wgJsonConfigs unset store for 'default' [16:34:39] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JTAOS [16:34:41] [02miraheze/puppet] 07Reception123 0309e9000 - temp absent manageInactiveWikis.php due to issues issues described in https://phabricator.miraheze.org/T6403 , multiple wikis reporting closures without them being inactive [16:34:42] [ ⚓ T6403 Script closing wikis when it shouldn’t ] - phabricator.miraheze.org [16:49:51] [02mw-config] 07Universal-Omega commented on pull request 03#3353: Remove FlaggedRevsRestrictionLevels from ManageWikiSettings - 13https://git.io/JTAs1 [16:49:52] [02mw-config] 07Universal-Omega closed pull request 03#3353: Remove FlaggedRevsRestrictionLevels from ManageWikiSettings - 13https://git.io/JTQsd [16:51:51] [02mw-config] 07Universal-Omega synchronize pull request 03#3363: Migrate $wgRestrictionLevels to ManageWikiSettings - 13https://git.io/JTFmu [16:53:05] [02mw-config] 07Universal-Omega synchronize pull request 03#3363: Migrate $wgRestrictionLevels to ManageWikiSettings - 13https://git.io/JTFmu [16:54:43] [02mw-config] 07Universal-Omega synchronize pull request 03#3363: Migrate $wgRestrictionLevels to ManageWikiSettings - 13https://git.io/JTFmu [16:58:03] [02mw-config] 07Universal-Omega edited pull request 03#3363: Migrate $wgRestrictionLevels to ManageWikiSettings/fix $wgFlaggedRevsRestrictionLevels (T5991 | T6212 ) - 13https://git.io/JTFmu [17:02:19] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [17:02:53] PROBLEM - wiki.usagihime.ml - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.usagihime.ml All nameservers failed to answer the query. [17:03:07] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [17:04:01] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [17:04:31] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [17:04:58] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [17:06:00] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [17:06:20] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [17:06:39] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [17:07:02] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [17:07:27] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [17:07:53] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [17:09:05] [02ssl] 07Universal-Omega edited pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [17:09:39] RECOVERY - wiki.usagihime.ml - reverse DNS on sslhost is OK: rDNS OK - wiki.usagihime.ml reverse DNS resolves to cp7.miraheze.org [17:12:18] ping musikanimal I recommend reviewing https://spcodex.wiki/wiki/Special:ManageWiki see https://phabricator.miraheze.org/T6403#125446 and since you have said that you have hidden the notice with .css [17:12:20] [ Manage this wiki's core settings - SPCodex ] - spcodex.wiki [17:12:21] [ ⚓ T6403 Script closing wikis when it shouldn’t ] - phabricator.miraheze.org [17:13:53] thanks for the ping. Yes mine closed again too [17:15:11] no problem :) [17:18:15] miraheze/mw-config/master/fc286a8 - paladox The build passed. https://travis-ci.org/miraheze/mw-config/builds/741434612 [17:42:34] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.64, 20.99, 19.25 [17:44:29] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.14, 18.73, 18.67 [17:52:16] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.45, 21.01, 20.01 [17:56:06] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.86, 18.43, 19.26 [18:13:49] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 33.75, 24.58, 20.73 [18:15:20] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 4 datacenters are down: 128.199.139.216/cpweb, 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb, 51.222.27.129/cpweb [18:16:14] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 16.55, 9.59, 4.54 [18:17:09] PROBLEM - gluster1 Current Load on gluster1 is WARNING: WARNING - load average: 7.13, 6.44, 4.18 [18:17:30] PROBLEM - mai.gyaanipedia.co.in - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for mai.gyaanipedia.co.in could not be found [18:19:12] PROBLEM - gluster1 Current Load on gluster1 is CRITICAL: CRITICAL - load average: 9.24, 7.24, 4.73 [18:21:10] PROBLEM - gluster1 Current Load on gluster1 is WARNING: WARNING - load average: 6.93, 7.11, 4.99 [18:21:32] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 51.222.27.129/cpweb [18:22:10] PROBLEM - wiki.jacksonheights.nyc - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.jacksonheights.nyc could not be found [18:22:11] PROBLEM - n64brew.dev - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for n64brew.dev could not be found [18:22:14] PROBLEM - dc.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for dc.miraheze.org could not be found [18:23:19] PROBLEM - tep.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for tep.wiki could not be found [18:23:20] PROBLEM - wiki.ngscott.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.ngscott.net could not be found [18:23:30] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [18:24:22] RECOVERY - mai.gyaanipedia.co.in - reverse DNS on sslhost is OK: rDNS OK - mai.gyaanipedia.co.in reverse DNS resolves to cp7.miraheze.org [18:25:04] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [18:25:06] RECOVERY - gluster1 Current Load on gluster1 is OK: OK - load average: 4.39, 6.11, 5.09 [18:28:52] RECOVERY - n64brew.dev - reverse DNS on sslhost is OK: rDNS OK - n64brew.dev reverse DNS resolves to cp7.miraheze.org [18:29:07] RECOVERY - wiki.jacksonheights.nyc - reverse DNS on sslhost is OK: rDNS OK - wiki.jacksonheights.nyc reverse DNS resolves to cp7.miraheze.org [18:29:14] RECOVERY - dc.miraheze.org - reverse DNS on sslhost is OK: rDNS OK - dc.miraheze.org reverse DNS resolves to cp6.miraheze.org [18:30:06] RECOVERY - tep.wiki - reverse DNS on sslhost is OK: rDNS OK - tep.wiki reverse DNS resolves to cp7.miraheze.org [18:30:08] RECOVERY - wiki.ngscott.net - reverse DNS on sslhost is OK: rDNS OK - wiki.ngscott.net reverse DNS resolves to cp6.miraheze.org [18:32:20] PROBLEM - tl.awiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for tl.awiki.org could not be found [18:32:22] PROBLEM - opendatascot.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for opendatascot.org could not be found [18:32:23] PROBLEM - munwiki.info - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for munwiki.info could not be found [18:32:24] PROBLEM - www.iceria.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.iceria.org could not be found [18:32:24] PROBLEM - uncyclopediakr.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for uncyclopediakr.miraheze.org could not be found [18:32:24] PROBLEM - www.portalsofphereon.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.portalsofphereon.com could not be found [18:32:27] PROBLEM - wiki.redeemer.live - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.redeemer.live could not be found [18:32:27] PROBLEM - wiki.twilightsignal.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.twilightsignal.com could not be found [18:32:30] PROBLEM - mr.gyaanipedia.co.in - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for mr.gyaanipedia.co.in could not be found [18:33:20] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 8 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb, 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [18:34:53] PROBLEM - gluster1 Current Load on gluster1 is CRITICAL: CRITICAL - load average: 8.90, 7.31, 5.96 [18:35:04] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 7 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb, 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb [18:36:51] RECOVERY - gluster1 Current Load on gluster1 is OK: OK - load average: 3.45, 5.86, 5.60 [18:37:02] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [18:37:13] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [18:39:03] RECOVERY - opendatascot.org - reverse DNS on sslhost is OK: rDNS OK - opendatascot.org reverse DNS resolves to cp6.miraheze.org [18:39:06] RECOVERY - www.iceria.org - reverse DNS on sslhost is OK: rDNS OK - www.iceria.org reverse DNS resolves to cp7.miraheze.org [18:39:07] RECOVERY - uncyclopediakr.miraheze.org - reverse DNS on sslhost is OK: rDNS OK - uncyclopediakr.miraheze.org reverse DNS resolves to cp6.miraheze.org [18:39:08] RECOVERY - www.portalsofphereon.com - reverse DNS on sslhost is OK: rDNS OK - www.portalsofphereon.com reverse DNS resolves to cp6.miraheze.org [18:39:14] RECOVERY - wiki.redeemer.live - reverse DNS on sslhost is OK: rDNS OK - wiki.redeemer.live reverse DNS resolves to cp7.miraheze.org [18:39:14] RECOVERY - wiki.twilightsignal.com - reverse DNS on sslhost is OK: rDNS OK - wiki.twilightsignal.com reverse DNS resolves to cp7.miraheze.org [18:39:15] RECOVERY - tl.awiki.org - reverse DNS on sslhost is OK: rDNS OK - tl.awiki.org reverse DNS resolves to cp7.miraheze.org [18:39:17] RECOVERY - mr.gyaanipedia.co.in - reverse DNS on sslhost is OK: rDNS OK - mr.gyaanipedia.co.in reverse DNS resolves to cp6.miraheze.org [18:39:19] RECOVERY - munwiki.info - reverse DNS on sslhost is OK: rDNS OK - munwiki.info reverse DNS resolves to cp7.miraheze.org [18:39:44] PROBLEM - cp7 Current Load on cp7 is WARNING: WARNING - load average: 6.93, 7.32, 7.59 [18:43:44] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 1.47, 4.57, 6.45 [18:53:28] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:00:46] PROBLEM - wiki.worldsofweary.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.worldsofweary.com could not be found [19:00:47] PROBLEM - orain.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for orain.org could not be found [19:00:52] PROBLEM - wiki.rebirthofthenight.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.rebirthofthenight.com could not be found [19:00:52] PROBLEM - wiki.joust.ro - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.joust.ro could not be found [19:00:59] PROBLEM - wikibeats.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wikibeats.net could not be found [19:07:30] RECOVERY - wiki.worldsofweary.com - reverse DNS on sslhost is OK: rDNS OK - wiki.worldsofweary.com reverse DNS resolves to cp6.miraheze.org [19:07:33] RECOVERY - orain.org - reverse DNS on sslhost is OK: rDNS OK - orain.org reverse DNS resolves to cp6.miraheze.org [19:07:42] RECOVERY - wiki.joust.ro - reverse DNS on sslhost is OK: rDNS OK - wiki.joust.ro reverse DNS resolves to cp6.miraheze.org [19:07:42] RECOVERY - wiki.rebirthofthenight.com - reverse DNS on sslhost is OK: rDNS OK - wiki.rebirthofthenight.com reverse DNS resolves to cp6.miraheze.org [19:07:55] RECOVERY - wikibeats.net - reverse DNS on sslhost is OK: rDNS OK - wikibeats.net reverse DNS resolves to cp7.miraheze.org [19:11:49] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.27, 20.55, 23.48 [19:11:51] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:13:16] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:13:51] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:14:59] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:15:28] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:15:49] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 32.46, 25.17, 24.55 [19:15:57] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:16:20] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:17:08] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:17:41] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:17:49] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.07, 22.72, 23.74 [19:18:34] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:19:01] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:19:23] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:19:46] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:20:14] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:20:37] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:21:04] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:21:29] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:22:23] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:22:53] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:24:06] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:24:41] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:25:11] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:25:34] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:26:22] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:26:49] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:27:14] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:27:52] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:28:14] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:28:34] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:29:09] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:29:27] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:29:58] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:30:22] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:30:51] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:34:17] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:34:45] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:35:11] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:35:32] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:35:54] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:36:15] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:36:36] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:36:58] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:37:49] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 12.16, 17.52, 20.10 [19:37:57] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:38:07] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:39:04] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:39:27] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:39:57] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:40:21] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:41:28] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:41:59] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:42:23] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:42:53] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:43:23] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:43:53] [02ssl] 07Universal-Omega synchronize pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:46:23] [02ssl] 07Universal-Omega edited pull request 03#370: remove non existing wikis. - 13https://git.io/JTd9n [19:46:42] [02ssl] 07Universal-Omega edited pull request 03#370: Remove invalid certs and redirects - 13https://git.io/JTd9n [20:01:08] [02ssl] 07Universal-Omega synchronize pull request 03#370: Remove invalid certs and redirects - 13https://git.io/JTd9n [20:03:49] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 31.22, 24.48, 20.55 [20:05:49] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.35, 23.15, 20.56 [20:07:49] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.20, 23.65, 21.05 [20:09:49] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.83, 22.39, 20.96 [20:12:22] PROBLEM - bacula2 Bacula Static 1 on bacula2 is WARNING: WARNING: Full, 11963030 files, 597.6GB, 2020-10-20 20:09:00 (2.1 weeks ago) [20:19:49] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 30.49, 25.45, 23.05 [20:29:49] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.03, 22.60, 23.43 [20:30:20] PROBLEM - cp9 Current Load on cp9 is WARNING: WARNING - load average: 1.83, 1.83, 1.29 [20:32:21] RECOVERY - cp9 Current Load on cp9 is OK: OK - load average: 1.01, 1.52, 1.24 [20:39:49] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.44, 16.91, 20.11 [20:44:03] PROBLEM - mw6 Current Load on mw6 is WARNING: WARNING - load average: 7.19, 5.76, 5.24 [20:45:49] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.70, 20.04, 20.38 [20:46:02] RECOVERY - mw6 Current Load on mw6 is OK: OK - load average: 5.96, 5.87, 5.35 [20:49:49] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 30.17, 23.28, 21.58 [20:51:49] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.85, 22.74, 21.61 [20:53:53] PROBLEM - mw6 Current Load on mw6 is WARNING: WARNING - load average: 7.12, 6.85, 5.95 [20:55:51] RECOVERY - mw6 Current Load on mw6 is OK: OK - load average: 4.00, 5.74, 5.65 [20:57:49] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 30.13, 23.43, 22.05 [20:59:44] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 23.47, 12.34, 5.86 [21:03:40] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 8.90, 6.37, 4.99 [21:03:44] PROBLEM - cp7 Current Load on cp7 is WARNING: WARNING - load average: 3.00, 6.93, 5.08 [21:05:37] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 3.99, 5.42, 4.81 [21:05:44] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 1.49, 5.07, 4.62 [21:06:03] PROBLEM - bacula2 NTP time on bacula2 is UNKNOWN: check_ntp_time: Invalid hostname/address - time.cloudflare.comUsage: check_ntp_time -H [-4 [21:06:53] PROBLEM - ns1 NTP time on ns1 is CRITICAL: NTP CRITICAL: No response from NTP server [21:07:35] PROBLEM - graylog1 NTP time on graylog1 is CRITICAL: NTP CRITICAL: No response from NTP server [21:07:37] PROBLEM - jobrunner2 NTP time on jobrunner2 is UNKNOWN: check_ntp_time: Invalid hostname/address - time.cloudflare.comUsage: check_ntp_time -H [-4 [21:11:49] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.36, 22.14, 23.75 [21:19:49] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.82, 22.92, 23.19 [21:21:49] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.61, 21.70, 22.73 [21:23:49] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 28.47, 23.97, 23.41 [21:24:55] PROBLEM - db7 Check MariaDB Replication on db7 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 199s [21:27:45] PROBLEM - cp7 Current Load on cp7 is WARNING: WARNING - load average: 7.60, 5.69, 4.29 [21:29:44] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 3.38, 4.72, 4.10 [22:30:22] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JTAVX [22:30:23] [02miraheze/services] 07MirahezeSSLBot 03d6b5451 - BOT: Updating services config for wikis [23:02:26] PROBLEM - n64brew.dev - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for n64brew.dev could not be found [23:03:59] PROBLEM - mw6 Current Load on mw6 is CRITICAL: CRITICAL - load average: 16.56, 8.94, 5.97 [23:04:06] PROBLEM - wiki.patriam.cc - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.patriam.cc could not be found [23:04:07] PROBLEM - www.fraterniterebellion.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.fraterniterebellion.com could not be found [23:04:07] PROBLEM - wiki.climatechange.ai - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.climatechange.ai could not be found [23:04:07] PROBLEM - www.petrawiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.petrawiki.org could not be found [23:04:08] PROBLEM - wiki.iufs.jp - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.iufs.jp could not be found [23:04:08] PROBLEM - wiki.tulpa.info - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.tulpa.info could not be found [23:04:12] PROBLEM - eerstelijnszones.be - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for eerstelijnszones.be could not be found [23:04:12] PROBLEM - holonet.pw - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for holonet.pw could not be found [23:04:15] PROBLEM - mai.gyaanipedia.co.in - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for mai.gyaanipedia.co.in could not be found [23:05:01] RECOVERY - db7 Check MariaDB Replication on db7 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 2s [23:07:58] PROBLEM - mw6 Current Load on mw6 is WARNING: WARNING - load average: 4.50, 7.29, 6.05 [23:09:07] RECOVERY - n64brew.dev - reverse DNS on sslhost is OK: rDNS OK - n64brew.dev reverse DNS resolves to cp6.miraheze.org [23:09:56] RECOVERY - mw6 Current Load on mw6 is OK: OK - load average: 5.96, 6.64, 5.95 [23:10:48] RECOVERY - wiki.patriam.cc - reverse DNS on sslhost is OK: rDNS OK - wiki.patriam.cc reverse DNS resolves to cp7.miraheze.org [23:10:49] RECOVERY - www.fraterniterebellion.com - reverse DNS on sslhost is OK: rDNS OK - www.fraterniterebellion.com reverse DNS resolves to cp6.miraheze.org [23:10:49] RECOVERY - wiki.climatechange.ai - reverse DNS on sslhost is OK: rDNS OK - wiki.climatechange.ai reverse DNS resolves to cp6.miraheze.org [23:10:50] RECOVERY - www.petrawiki.org - reverse DNS on sslhost is OK: rDNS OK - www.petrawiki.org reverse DNS resolves to cp7.miraheze.org [23:10:53] RECOVERY - wiki.iufs.jp - reverse DNS on sslhost is OK: rDNS OK - wiki.iufs.jp reverse DNS resolves to cp6.miraheze.org [23:10:54] RECOVERY - wiki.tulpa.info - reverse DNS on sslhost is OK: rDNS OK - wiki.tulpa.info reverse DNS resolves to cp6.miraheze.org [23:10:57] RECOVERY - eerstelijnszones.be - reverse DNS on sslhost is OK: rDNS OK - eerstelijnszones.be reverse DNS resolves to cp6.miraheze.org [23:10:57] RECOVERY - holonet.pw - reverse DNS on sslhost is OK: rDNS OK - holonet.pw reverse DNS resolves to cp6.miraheze.org [23:11:07] RECOVERY - mai.gyaanipedia.co.in - reverse DNS on sslhost is OK: rDNS OK - mai.gyaanipedia.co.in reverse DNS resolves to cp6.miraheze.org [23:21:49] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 12.06, 15.96, 23.59 [23:22:51] [02ManageWiki] 07Universal-Omega opened pull request 03#191: Add support for custom types within $wgManageWikiNamespacesAdditional (Would help with T4943) - 13https://git.io/JTAoq [23:29:49] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 11.60, 13.43, 19.44