[00:00:12] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 3.94, 5.28, 5.52 [00:02:18] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.62, 6.04, 5.77 [00:04:01] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 128.199.139.216/cpweb [00:04:30] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.68, 5.69, 5.69 [00:04:31] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 40.60, 42.41, 30.46 [00:04:35] PROBLEM - wiki.macc.nyc - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.macc.nyc could not be found [00:04:44] PROBLEM - innersphere.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for innersphere.tk could not be found [00:04:49] PROBLEM - www.petrawiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.petrawiki.org could not be found [00:04:52] RECOVERY - mw6 Disk Space on mw6 is OK: DISK OK - free space: / 2333 MB (12% inode=73%); [00:04:57] PROBLEM - erikapedia.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for erikapedia.com could not be found [00:05:03] PROBLEM - www.mcpk.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.mcpk.wiki could not be found [00:05:04] PROBLEM - en.phgalaxy.xyz - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for en.phgalaxy.xyz could not be found [00:05:05] PROBLEM - wiki.vinesh.eu.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.vinesh.eu.org could not be found [00:05:05] PROBLEM - ndg.nenawiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for ndg.nenawiki.org could not be found [00:05:05] PROBLEM - tep.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for tep.wiki could not be found [00:05:08] PROBLEM - aman.awiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for aman.awiki.org could not be found [00:05:08] PROBLEM - meta.orain.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for meta.orain.org could not be found [00:05:09] PROBLEM - uncyclopediakr.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for uncyclopediakr.miraheze.org could not be found [00:05:09] PROBLEM - spcodex.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for spcodex.wiki could not be found [00:05:12] PROBLEM - wiki.shaazzz.ir - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.shaazzz.ir could not be found [00:05:12] PROBLEM - hololive.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for hololive.wiki could not be found [00:05:14] PROBLEM - allthetropes.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for allthetropes.org could not be found [00:05:16] PROBLEM - tradwiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for tradwiki.org could not be found [00:05:17] PROBLEM - es.gyaanipedia.co.in - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for es.gyaanipedia.co.in could not be found [00:05:20] PROBLEM - chouverse.cludisciples.cyou - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for chouverse.cludisciples.cyou could not be found [00:05:58] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [00:07:48] [02miraheze/WikiDiscover] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkwOd [00:07:49] [02miraheze/WikiDiscover] 07Universal-Omega 03a8d7bcc - add aliases [00:07:51] [02WikiDiscover] 07Universal-Omega synchronize pull request 03#23: Add more Magic Words; convert multiple if statements to a switch - 13https://git.io/JkVgv [00:11:30] RECOVERY - innersphere.tk - reverse DNS on sslhost is OK: rDNS OK - innersphere.tk reverse DNS resolves to cp7.miraheze.org [00:11:33] RECOVERY - wiki.macc.nyc - reverse DNS on sslhost is OK: rDNS OK - wiki.macc.nyc reverse DNS resolves to cp6.miraheze.org [00:11:37] RECOVERY - www.petrawiki.org - reverse DNS on sslhost is OK: rDNS OK - www.petrawiki.org reverse DNS resolves to cp6.miraheze.org [00:11:50] RECOVERY - erikapedia.com - reverse DNS on sslhost is OK: rDNS OK - erikapedia.com reverse DNS resolves to cp6.miraheze.org [00:11:54] RECOVERY - meta.orain.org - reverse DNS on sslhost is OK: rDNS OK - meta.orain.org reverse DNS resolves to cp6.miraheze.org [00:11:55] RECOVERY - spcodex.wiki - reverse DNS on sslhost is OK: rDNS OK - spcodex.wiki reverse DNS resolves to cp6.miraheze.org [00:11:55] RECOVERY - www.mcpk.wiki - reverse DNS on sslhost is OK: rDNS OK - www.mcpk.wiki reverse DNS resolves to cp6.miraheze.org [00:11:56] RECOVERY - allthetropes.org - reverse DNS on sslhost is OK: rDNS OK - allthetropes.org reverse DNS resolves to cp7.miraheze.org [00:11:57] RECOVERY - wiki.vinesh.eu.org - reverse DNS on sslhost is OK: rDNS OK - wiki.vinesh.eu.org reverse DNS resolves to cp6.miraheze.org [00:11:58] RECOVERY - tradwiki.org - reverse DNS on sslhost is OK: rDNS OK - tradwiki.org reverse DNS resolves to cp6.miraheze.org [00:11:58] RECOVERY - aman.awiki.org - reverse DNS on sslhost is OK: rDNS OK - aman.awiki.org reverse DNS resolves to cp6.miraheze.org [00:12:00] RECOVERY - ndg.nenawiki.org - reverse DNS on sslhost is OK: rDNS OK - ndg.nenawiki.org reverse DNS resolves to cp6.miraheze.org [00:12:01] RECOVERY - tep.wiki - reverse DNS on sslhost is OK: rDNS OK - tep.wiki reverse DNS resolves to cp6.miraheze.org [00:12:02] RECOVERY - chouverse.cludisciples.cyou - reverse DNS on sslhost is OK: rDNS OK - chouverse.cludisciples.cyou reverse DNS resolves to cp7.miraheze.org [00:12:04] RECOVERY - en.phgalaxy.xyz - reverse DNS on sslhost is OK: rDNS OK - en.phgalaxy.xyz reverse DNS resolves to cp7.miraheze.org [00:12:07] RECOVERY - wiki.shaazzz.ir - reverse DNS on sslhost is OK: rDNS OK - wiki.shaazzz.ir reverse DNS resolves to cp6.miraheze.org [00:12:08] RECOVERY - uncyclopediakr.miraheze.org - reverse DNS on sslhost is OK: rDNS OK - uncyclopediakr.miraheze.org reverse DNS resolves to cp7.miraheze.org [00:12:09] RECOVERY - hololive.wiki - reverse DNS on sslhost is OK: rDNS OK - hololive.wiki reverse DNS resolves to cp6.miraheze.org [00:12:14] RECOVERY - es.gyaanipedia.co.in - reverse DNS on sslhost is OK: rDNS OK - es.gyaanipedia.co.in reverse DNS resolves to cp7.miraheze.org [00:16:28] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 3.99, 4.46, 5.05 [00:27:29] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.51, 5.03, 4.98 [00:30:56] PROBLEM - mon1 APT on mon1 is CRITICAL: APT CRITICAL: 11 packages available for upgrade (5 critical updates). [00:31:28] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.19, 5.98, 5.35 [00:33:28] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.44, 5.83, 5.37 [00:34:04] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 14.52, 20.62, 23.68 [00:43:26] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.54, 5.76, 5.43 [00:45:25] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.75, 5.35, 5.32 [00:51:24] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 3.27, 4.55, 5.02 [00:54:03] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.71, 18.18, 20.03 [00:57:36] PROBLEM - mw7 APT on mw7 is CRITICAL: APT CRITICAL: 17 packages available for upgrade (5 critical updates). [01:00:22] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkwcT [01:00:24] [02miraheze/services] 07MirahezeSSLBot 035e4fd9f - BOT: Updating services config for wikis [01:02:56] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.84, 22.90, 20.94 [01:04:22] PROBLEM - cloud2 APT on cloud2 is CRITICAL: APT CRITICAL: 90 packages available for upgrade (4 critical updates). [01:04:53] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.44, 21.16, 20.54 [01:07:48] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkwcQ [01:07:50] [02miraheze/mw-config] 07Universal-Omega 03d641daa - Un-restrict $wgDefaultRobotPolicy; convert it to array [01:07:51] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/vbvb3 [01:07:56] [02mw-config] 07Universal-Omega opened pull request 03#3464: Un-restrict $wgDefaultRobotPolicy; convert it to array - 13https://git.io/Jkwc7 [01:08:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.28, 6.09, 5.34 [01:08:48] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.97, 19.16, 19.92 [01:10:08] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkwCv [01:10:10] [02miraheze/mw-config] 07Universal-Omega 03dafd425 - add $wmgDefaultRobotPolicy [01:10:11] [02mw-config] 07Universal-Omega synchronize pull request 03#3464: Un-restrict $wgDefaultRobotPolicy; convert it to array - 13https://git.io/Jkwc7 [01:10:25] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.35, 5.69, 5.28 [01:12:25] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.50, 6.25, 5.55 [01:13:44] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkwC0 [01:13:46] [02miraheze/mw-config] 07Universal-Omega 034ff5fde - add value for wgDefaultRobotPolicy to convert to string [01:13:47] [02mw-config] 07Universal-Omega synchronize pull request 03#3464: Un-restrict $wgDefaultRobotPolicy; convert it to array - 13https://git.io/Jkwc7 [01:16:24] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.82, 5.70, 5.50 [01:19:30] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.30, 24.26, 21.90 [01:22:23] PROBLEM - db12 APT on db12 is CRITICAL: APT CRITICAL: 38 packages available for upgrade (5 critical updates). [01:22:45] miraheze/mw-config/Universal-Omega-patch-1/4ff5fde - Universal Omega The build passed. https://travis-ci.org/miraheze/mw-config/builds/745151908 [01:25:01] [02mw-config] 07dmehus commented on commit 03d641daa7f2a5b472a93a07aa333d8f90eacdbaca - 13https://git.io/JkwWR [01:25:24] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.96, 23.80, 22.64 [01:25:54] [02mw-config] 07Universal-Omega commented on commit 03d641daa7f2a5b472a93a07aa333d8f90eacdbaca - 13https://git.io/JkwWg [01:28:05] PROBLEM - test2 APT on test2 is CRITICAL: APT CRITICAL: 19 packages available for upgrade (5 critical updates). [01:28:22] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 3.13, 4.64, 5.06 [01:29:41] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_35 [+0/-0/±2] 13https://git.io/JkwWb [01:29:42] [02miraheze/mediawiki] 07paladox 03cd69cac - Update BlogPage [01:33:11] !log rebuild lc on mw* and jobrunner* [01:33:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:33:30] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkwlW [01:33:32] [02miraheze/mw-config] 07Universal-Omega 03b2a32c3 - fix defaults [01:33:33] [02mw-config] 07Universal-Omega synchronize pull request 03#3464: Un-restrict $wgDefaultRobotPolicy; convert it to array - 13https://git.io/Jkwc7 [01:35:13] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 30.79, 22.66, 21.59 [01:36:39] PROBLEM - mw7 Current Load on mw7 is WARNING: WARNING - load average: 7.60, 6.29, 4.73 [01:37:08] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/Jkwl5 [01:37:10] [02miraheze/mw-config] 07Universal-Omega 038882cbf - re-order [01:37:10] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.92, 22.67, 21.75 [01:37:11] [02mw-config] 07Universal-Omega synchronize pull request 03#3464: Un-restrict $wgDefaultRobotPolicy; convert it to array - 13https://git.io/Jkwc7 [01:37:47] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jkwlh [01:37:49] [02miraheze/mw-config] 07paladox 03c7d03d8 - Add TemplateStyles to wgManageWikiNamespacesExtraContentModels if extension is installed [01:38:20] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.97, 5.17, 4.76 [01:39:07] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 30.10, 25.88, 23.06 [01:40:53] PROBLEM - mw7 Current Load on mw7 is CRITICAL: CRITICAL - load average: 10.07, 7.85, 5.70 [01:41:10] PROBLEM - test2 Puppet on test2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [01:42:07] PROBLEM - marthaspeaks.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for marthaspeaks.miraheze.org could not be found [01:42:12] PROBLEM - wiki.fraterniterebellion.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.fraterniterebellion.com could not be found [01:42:22] PROBLEM - guia.cineastas.pt - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for guia.cineastas.pt could not be found [01:42:22] PROBLEM - housing.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for housing.wiki could not be found [01:42:23] PROBLEM - wiki.mnzp.xyz - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.mnzp.xyz could not be found [01:42:23] PROBLEM - test1.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for test1.miraheze.org could not be found [01:42:25] PROBLEM - en.wikireserve.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for en.wikireserve.com could not be found [01:42:34] PROBLEM - podpedia.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for podpedia.org could not be found [01:42:34] PROBLEM - wiki.abyogsms.org.in - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.abyogsms.org.in could not be found [01:42:37] PROBLEM - archive.stellurgists.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for archive.stellurgists.wiki could not be found [01:42:47] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 13.77, 7.39, 4.77 [01:43:17] PROBLEM - mw6 APT on mw6 is CRITICAL: APT CRITICAL: 17 packages available for upgrade (5 critical updates). [01:43:25] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jkw86 [01:43:27] [02miraheze/mw-config] 07paladox 0310450de - Revert "Add TemplateStyles to wgManageWikiNamespacesExtraContentModels if extension is installed" This reverts commit c7d03d8e83849f4095e0a69003028ba6937d0fd7. [01:44:44] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 6.84, 6.10, 4.60 [01:45:09] RECOVERY - test2 Puppet on test2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:46:39] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 6.06, 7.06, 5.27 [01:46:43] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 5.38, 5.80, 4.67 [01:46:56] PROBLEM - mw7 Current Load on mw7 is WARNING: WARNING - load average: 4.54, 7.53, 6.43 [01:48:38] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 6.00, 6.51, 5.27 [01:48:53] PROBLEM - mw7 Current Load on mw7 is CRITICAL: CRITICAL - load average: 8.12, 7.88, 6.69 [01:48:55] RECOVERY - marthaspeaks.miraheze.org - reverse DNS on sslhost is OK: rDNS OK - marthaspeaks.miraheze.org reverse DNS resolves to cp7.miraheze.org [01:49:01] RECOVERY - wiki.fraterniterebellion.com - reverse DNS on sslhost is OK: rDNS OK - wiki.fraterniterebellion.com reverse DNS resolves to cp6.miraheze.org [01:49:06] RECOVERY - wiki.mnzp.xyz - reverse DNS on sslhost is OK: rDNS OK - wiki.mnzp.xyz reverse DNS resolves to cp7.miraheze.org [01:49:06] RECOVERY - test1.miraheze.org - reverse DNS on sslhost is OK: rDNS OK - test1.miraheze.org reverse DNS resolves to cp6.miraheze.org [01:49:07] RECOVERY - en.wikireserve.com - reverse DNS on sslhost is OK: rDNS OK - en.wikireserve.com reverse DNS resolves to cp7.miraheze.org [01:49:08] RECOVERY - housing.wiki - reverse DNS on sslhost is OK: rDNS OK - housing.wiki reverse DNS resolves to cp6.miraheze.org [01:49:12] RECOVERY - guia.cineastas.pt - reverse DNS on sslhost is OK: rDNS OK - guia.cineastas.pt reverse DNS resolves to cp6.miraheze.org [01:49:13] PROBLEM - mw6 Current Load on mw6 is WARNING: WARNING - load average: 7.39, 7.04, 5.72 [01:49:14] RECOVERY - podpedia.org - reverse DNS on sslhost is OK: rDNS OK - podpedia.org reverse DNS resolves to cp7.miraheze.org [01:49:14] RECOVERY - wiki.abyogsms.org.in - reverse DNS on sslhost is OK: rDNS OK - wiki.abyogsms.org.in reverse DNS resolves to cp7.miraheze.org [01:50:20] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 5 datacenters are down: 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [01:51:27] PROBLEM - mw6 Current Load on mw6 is CRITICAL: CRITICAL - load average: 10.23, 8.31, 6.37 [01:52:14] PROBLEM - private.revi.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for private.revi.wiki could not be found [01:52:17] PROBLEM - es.gyaanipedia.co.in - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for es.gyaanipedia.co.in could not be found [01:52:18] PROBLEM - www.fraterniterebellion.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.fraterniterebellion.com could not be found [01:52:18] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [01:52:18] PROBLEM - bharatwiki.online - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for bharatwiki.online could not be found [01:52:22] PROBLEM - wiki.kourouklides.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.kourouklides.com could not be found [01:52:28] PROBLEM - debilepedie.gq - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for debilepedie.gq could not be found [01:52:29] PROBLEM - petrawiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for petrawiki.org could not be found [01:52:30] PROBLEM - public.revi.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for public.revi.wiki could not be found [01:52:31] PROBLEM - uncyclopediakr.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for uncyclopediakr.miraheze.org could not be found [01:52:31] PROBLEM - wiki.ricochetuniverse.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.ricochetuniverse.com could not be found [01:52:39] PROBLEM - wiki.helioss.co - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.helioss.co could not be found [01:52:43] PROBLEM - nonbinary.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for nonbinary.wiki could not be found [01:52:44] PROBLEM - battlecats.ml - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for battlecats.ml could not be found [01:52:45] PROBLEM - www.dariawiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.dariawiki.org could not be found [01:52:46] PROBLEM - dc-multiverse.dcwikis.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for dc-multiverse.dcwikis.com could not be found [01:52:46] PROBLEM - wiki.autocountsoft.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.autocountsoft.com could not be found [01:52:47] PROBLEM - cyberlaw.ccdcoe.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for cyberlaw.ccdcoe.org could not be found [01:52:49] PROBLEM - cubashpedia.cf - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for cubashpedia.cf could not be found [01:52:51] PROBLEM - wiki.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.miraheze.org could not be found [01:52:51] PROBLEM - wiki.arnab.at - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.arnab.at could not be found [01:52:52] PROBLEM - viileapedia.ga - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for viileapedia.ga could not be found [01:52:54] PROBLEM - wiki.openhatch.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.openhatch.org could not be found [01:52:54] PROBLEM - wiki.valkyrienskies.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.valkyrienskies.org could not be found [01:55:23] PROBLEM - mw6 Current Load on mw6 is WARNING: WARNING - load average: 6.33, 7.69, 6.59 [01:56:53] RECOVERY - archive.stellurgists.wiki - reverse DNS on sslhost is OK: rDNS OK - archive.stellurgists.wiki reverse DNS resolves to cp7.miraheze.org [01:56:58] PROBLEM - mw7 Current Load on mw7 is WARNING: WARNING - load average: 7.46, 7.82, 7.23 [01:58:57] RECOVERY - www.fraterniterebellion.com - reverse DNS on sslhost is OK: rDNS OK - www.fraterniterebellion.com reverse DNS resolves to cp7.miraheze.org [01:59:03] RECOVERY - wiki.kourouklides.com - reverse DNS on sslhost is OK: rDNS OK - wiki.kourouklides.com reverse DNS resolves to cp6.miraheze.org [01:59:08] RECOVERY - private.revi.wiki - reverse DNS on sslhost is OK: rDNS OK - private.revi.wiki reverse DNS resolves to cp6.miraheze.org [01:59:14] RECOVERY - es.gyaanipedia.co.in - reverse DNS on sslhost is OK: rDNS OK - es.gyaanipedia.co.in reverse DNS resolves to cp7.miraheze.org [01:59:17] RECOVERY - bharatwiki.online - reverse DNS on sslhost is OK: rDNS OK - bharatwiki.online reverse DNS resolves to cp6.miraheze.org [01:59:24] RECOVERY - wiki.helioss.co - reverse DNS on sslhost is OK: rDNS OK - wiki.helioss.co reverse DNS resolves to cp6.miraheze.org [01:59:26] RECOVERY - wiki.ricochetuniverse.com - reverse DNS on sslhost is OK: rDNS OK - wiki.ricochetuniverse.com reverse DNS resolves to cp6.miraheze.org [01:59:26] RECOVERY - public.revi.wiki - reverse DNS on sslhost is OK: rDNS OK - public.revi.wiki reverse DNS resolves to cp6.miraheze.org [01:59:30] RECOVERY - debilepedie.gq - reverse DNS on sslhost is OK: rDNS OK - debilepedie.gq reverse DNS resolves to cp7.miraheze.org [01:59:31] RECOVERY - petrawiki.org - reverse DNS on sslhost is OK: rDNS OK - petrawiki.org reverse DNS resolves to cp7.miraheze.org [01:59:33] RECOVERY - uncyclopediakr.miraheze.org - reverse DNS on sslhost is OK: rDNS OK - uncyclopediakr.miraheze.org reverse DNS resolves to cp6.miraheze.org [01:59:34] RECOVERY - nonbinary.wiki - reverse DNS on sslhost is OK: rDNS OK - nonbinary.wiki reverse DNS resolves to cp7.miraheze.org [01:59:34] RECOVERY - viileapedia.ga - reverse DNS on sslhost is OK: rDNS OK - viileapedia.ga reverse DNS resolves to cp6.miraheze.org [01:59:35] RECOVERY - battlecats.ml - reverse DNS on sslhost is OK: rDNS OK - battlecats.ml reverse DNS resolves to cp7.miraheze.org [01:59:37] RECOVERY - wiki.autocountsoft.com - reverse DNS on sslhost is OK: rDNS OK - wiki.autocountsoft.com reverse DNS resolves to cp6.miraheze.org [01:59:37] RECOVERY - cyberlaw.ccdcoe.org - reverse DNS on sslhost is OK: rDNS OK - cyberlaw.ccdcoe.org reverse DNS resolves to cp7.miraheze.org [01:59:38] RECOVERY - dc-multiverse.dcwikis.com - reverse DNS on sslhost is OK: rDNS OK - dc-multiverse.dcwikis.com reverse DNS resolves to cp7.miraheze.org [01:59:38] RECOVERY - wiki.miraheze.org - reverse DNS on sslhost is OK: rDNS OK - wiki.miraheze.org reverse DNS resolves to cp7.miraheze.org [01:59:39] RECOVERY - wiki.valkyrienskies.org - reverse DNS on sslhost is OK: rDNS OK - wiki.valkyrienskies.org reverse DNS resolves to cp6.miraheze.org [01:59:40] RECOVERY - wiki.arnab.at - reverse DNS on sslhost is OK: rDNS OK - wiki.arnab.at reverse DNS resolves to cp7.miraheze.org [01:59:43] RECOVERY - cubashpedia.cf - reverse DNS on sslhost is OK: rDNS OK - cubashpedia.cf reverse DNS resolves to cp7.miraheze.org [01:59:44] RECOVERY - www.dariawiki.org - reverse DNS on sslhost is OK: rDNS OK - www.dariawiki.org reverse DNS resolves to cp7.miraheze.org [01:59:44] RECOVERY - wiki.openhatch.org - reverse DNS on sslhost is OK: rDNS OK - wiki.openhatch.org reverse DNS resolves to cp7.miraheze.org [02:02:52] PROBLEM - mw7 Current Load on mw7 is CRITICAL: CRITICAL - load average: 9.02, 7.90, 7.34 [02:02:57] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 2.21, 4.21, 5.62 [02:04:50] PROBLEM - mw7 Current Load on mw7 is WARNING: WARNING - load average: 5.52, 7.07, 7.11 [02:06:57] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 1.66, 2.81, 4.72 [02:08:45] RECOVERY - mw7 Current Load on mw7 is OK: OK - load average: 4.26, 6.01, 6.69 [02:09:08] RECOVERY - mw6 Current Load on mw6 is OK: OK - load average: 4.44, 6.10, 6.51 [02:10:50] PROBLEM - cloud1 APT on cloud1 is CRITICAL: APT CRITICAL: 89 packages available for upgrade (4 critical updates). [02:19:48] !log graylog1: enable best_compression for index on elasticsearch [02:19:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:20:56] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 10.89, 15.96, 22.69 [02:21:43] [02mw-config] 07Universal-Omega edited pull request 03#3464: Un-restrict $wgDefaultRobotPolicy; convert it to array - 13https://git.io/Jkwc7 [02:24:51] PROBLEM - cp9 APT on cp9 is CRITICAL: APT CRITICAL: 9 packages available for upgrade (4 critical updates). [02:28:14] PROBLEM - db7 APT on db7 is CRITICAL: APT CRITICAL: 38 packages available for upgrade (5 critical updates). [02:28:45] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 13.05, 14.86, 19.86 [02:33:05] !log graylog1: delete all data in elasticsearch [02:33:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:40:44] PROBLEM - ns1 APT on ns1 is CRITICAL: APT CRITICAL: 9 packages available for upgrade (4 critical updates). [02:40:57] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.97, 4.38, 3.23 [02:42:58] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.55, 5.05, 3.61 [02:44:29] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.50, 21.66, 20.02 [02:45:00] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 5.04, 4.64, 3.61 [02:46:26] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.27, 21.98, 20.30 [02:48:23] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.70, 21.00, 20.16 [02:53:00] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.18, 5.15, 4.27 [02:54:18] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 13.86, 19.20, 19.98 [02:55:01] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 4.91, 4.96, 4.30 [02:59:02] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.48, 5.71, 4.81 [03:05:03] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 3.57, 4.99, 4.82 [03:10:57] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.49, 20.60, 20.23 [03:16:52] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 17.59, 19.74, 20.16 [03:18:01] PROBLEM - mw4 APT on mw4 is CRITICAL: APT CRITICAL: 17 packages available for upgrade (5 critical updates). [03:20:10] PROBLEM - db13 APT on db13 is CRITICAL: APT CRITICAL: 38 packages available for upgrade (5 critical updates). [03:27:45] [02WikiDiscover] 07Universal-Omega edited pull request 03#23: Add additional magic words support ({{NUMBEROFPUBLICWIKIS}}, {{NUMBEROFINACTIVEWIKIS}}, {{NUMBEROFLOCKEDWIKIS}}, {{NUMBEROFDELETEDWIKIS}}, {{NUMBEROFINACTIVITYEXEMPTWIKIS}}, {{NUMBEROFCUSTOMDOMAINS}}, and {{NUMBEROFWIKISINLANGUAGE:language_code}}); convert multiple ifs to switch; add magic word aliases - 13https://git.io/JkVgv [03:28:27] [02WikiDiscover] 07Universal-Omega edited pull request 03#23: Add additional magic words ({{NUMBEROFPUBLICWIKIS}}, {{NUMBEROFINACTIVEWIKIS}}, {{NUMBEROFLOCKEDWIKIS}}, {{NUMBEROFDELETEDWIKIS}}, {{NUMBEROFINACTIVITYEXEMPTWIKIS}}, {{NUMBEROFCUSTOMDOMAINS}}, and {{NUMBEROFWIKISINLANGUAGE:language_code}}); convert multiple ifs to switch; add magic word aliases - 13https://git.io/JkVgv [03:31:18] [02WikiDiscover] 07Universal-Omega edited pull request 03#23: Add additional magic words ({{NUMBEROFPUBLICWIKIS}}, {{NUMBEROFINACTIVEWIKIS}}, {{NUMBEROFLOCKEDWIKIS}}, {{NUMBEROFDELETEDWIKIS}}, {{NUMBEROFINACTIVITYEXEMPTWIKIS}}, {{NUMBEROFCUSTOMDOMAINS}}, and {{NUMBEROFWIKISINLANGUAGE:language_code}}); fix prior magic words to not include deleted wikis; add magic word aliases; convert multiple ifs to switch; - [03:31:18] 13https://git.io/JkVgv [03:31:34] [02WikiDiscover] 07Universal-Omega edited pull request 03#23: Add additional magic words ({{NUMBEROFPUBLICWIKIS}}, {{NUMBEROFINACTIVEWIKIS}}, {{NUMBEROFLOCKEDWIKIS}}, {{NUMBEROFDELETEDWIKIS}}, {{NUMBEROFINACTIVITYEXEMPTWIKIS}}, {{NUMBEROFCUSTOMDOMAINS}}, and {{NUMBEROFWIKISINLANGUAGE:language_code}}); fix prior magic words to not include deleted wikis; add magic word aliases; convert multiple ifs to a switch - [03:31:35] 13https://git.io/JkVgv [03:33:03] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.68, 5.46, 5.04 [03:36:21] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.20, 20.64, 19.80 [03:38:20] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 29.67, 24.63, 21.39 [03:40:46] PROBLEM - cloud3 APT on cloud3 is CRITICAL: APT CRITICAL: 61 packages available for upgrade (4 critical updates). [03:44:15] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 17.84, 22.68, 21.70 [03:52:59] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.23, 5.61, 5.42 [03:53:54] PROBLEM - puppet2 APT on puppet2 is CRITICAL: APT CRITICAL: 13 packages available for upgrade (5 critical updates). [03:54:59] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.11, 4.98, 5.20 [03:56:04] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 12.51, 17.24, 19.84 [03:59:00] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 4.86, 4.64, 5.00 [04:14:58] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 9.27, 6.59, 5.53 [04:16:12] !log sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --report=1 --wiki mcuwiki /home/universalomega/mcuwiki/mcuwiki[1-16].xml (16 XML files) on jobrunner1 [04:16:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:16:35] PROBLEM - mail1 APT on mail1 is CRITICAL: APT CRITICAL: 11 packages available for upgrade (5 critical updates). [04:20:17] !log sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --update --wiki mcuwiki on Jobrunner1 [04:20:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:20:59] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 3.63, 5.16, 5.28 [04:23:00] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 8.49, 6.13, 5.60 [04:24:59] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.65, 5.39, 5.39 [04:25:30] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.53, 20.93, 19.79 [04:27:33] PROBLEM - jobrunner2 APT on jobrunner2 is CRITICAL: APT CRITICAL: 17 packages available for upgrade (5 critical updates). [04:27:34] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 30.78, 22.64, 20.43 [04:29:00] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.88, 5.93, 5.56 [04:31:00] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 3.66, 5.10, 5.31 [04:31:29] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.09, 23.02, 21.10 [04:32:59] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.19, 5.51, 5.42 [04:34:59] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 3.84, 4.87, 5.20 [04:36:58] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.09, 5.11, 5.23 [04:38:58] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 3.12, 4.51, 5.00 [04:39:46] PROBLEM - cp7 APT on cp7 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (5 critical updates). [04:45:14] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.60, 18.48, 20.18 [04:51:58] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 12.37, 7.38, 5.90 [04:57:57] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 3.40, 5.12, 5.42 [05:01:57] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 10.50, 6.85, 5.98 [05:04:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.06, 21.35, 19.67 [05:06:09] PROBLEM - mw5 APT on mw5 is CRITICAL: APT CRITICAL: 17 packages available for upgrade (5 critical updates). [05:07:56] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 2.73, 5.19, 5.59 [05:08:39] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.46, 20.50, 19.60 [05:10:36] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 19.40, 19.50, 19.33 [05:11:55] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.14, 5.73, 5.69 [05:13:55] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.92, 5.55, 5.64 [05:16:26] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.77, 22.10, 20.49 [05:19:53] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 3.47, 4.19, 4.96 [05:22:20] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 31.85, 25.49, 22.18 [05:23:54] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.84, 6.16, 5.65 [05:24:17] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 14.71, 21.55, 21.16 [05:26:15] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 32.34, 25.22, 22.50 [05:27:55] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.06, 5.86, 5.65 [05:28:12] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 14.33, 20.77, 21.20 [05:31:58] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.33, 5.85, 5.64 [05:33:58] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.43, 5.44, 5.53 [05:36:05] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.34, 18.04, 19.99 [05:38:00] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 2.97, 4.37, 5.08 [05:46:04] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.12, 5.36, 5.23 [05:50:12] PROBLEM - services2 APT on services2 is CRITICAL: APT CRITICAL: 12 packages available for upgrade (5 critical updates). [05:51:41] PROBLEM - services1 APT on services1 is CRITICAL: APT CRITICAL: 12 packages available for upgrade (5 critical updates). [05:52:06] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.57, 5.62, 5.38 [05:54:06] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.35, 5.96, 5.54 [05:56:05] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.17, 5.54, 5.42 [05:56:50] PROBLEM - ns2 APT on ns2 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (5 critical updates). [06:00:13] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.24, 5.67, 5.48 [06:03:54] RECOVERY - mon1 APT on mon1 is OK: APT OK: 6 packages available for upgrade (0 critical updates). [06:04:23] PROBLEM - mon1 Puppet on mon1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[php7.3-apcu] [06:04:37] RECOVERY - mail1 APT on mail1 is OK: APT OK: 6 packages available for upgrade (0 critical updates). [06:07:08] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.18, 24.44, 20.81 [06:07:25] PROBLEM - ldap1 APT on ldap1 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (5 critical updates). [06:08:25] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 3.69, 5.79, 5.79 [06:09:07] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 14.43, 20.66, 19.85 [06:11:04] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.94, 18.43, 19.10 [06:12:12] RECOVERY - mon1 Puppet on mon1 is OK: OK: Puppet is currently enabled, last run 8 seconds ago with 0 failures [06:12:24] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.25, 5.71, 5.74 [06:14:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.89, 5.72, 5.74 [06:16:09] RECOVERY - db13 APT on db13 is OK: APT OK: 33 packages available for upgrade (0 critical updates). [06:16:14] RECOVERY - db7 APT on db7 is OK: APT OK: 33 packages available for upgrade (0 critical updates). [06:18:23] RECOVERY - services2 APT on services2 is OK: APT OK: 7 packages available for upgrade (0 critical updates). [06:20:38] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.87, 5.71, 5.63 [06:21:29] RECOVERY - jobrunner2 APT on jobrunner2 is OK: APT OK: 12 packages available for upgrade (0 critical updates). [06:22:01] RECOVERY - mw4 APT on mw4 is OK: APT OK: 12 packages available for upgrade (0 critical updates). [06:23:52] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.20, 25.36, 21.84 [06:24:37] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 3.73, 5.47, 5.61 [06:25:49] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.85, 23.83, 21.72 [06:27:29] RECOVERY - ldap1 APT on ldap1 is OK: APT OK: 5 packages available for upgrade (0 critical updates). [06:28:08] RECOVERY - mw5 APT on mw5 is OK: APT OK: 12 packages available for upgrade (0 critical updates). [06:29:47] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 19.09, 19.44, 20.31 [06:30:37] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.58, 5.77, 5.57 [06:34:36] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.79, 5.61, 5.56 [06:34:51] RECOVERY - cp9 APT on cp9 is OK: APT OK: 5 packages available for upgrade (0 critical updates). [06:35:42] RECOVERY - puppet2 APT on puppet2 is OK: APT OK: 8 packages available for upgrade (0 critical updates). [06:36:49] RECOVERY - cloud1 APT on cloud1 is OK: APT OK: 85 packages available for upgrade (0 critical updates). [06:38:36] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.55, 5.84, 5.61 [06:38:48] RECOVERY - cloud2 APT on cloud2 is OK: APT OK: 86 packages available for upgrade (0 critical updates). [06:40:45] RECOVERY - cloud3 APT on cloud3 is OK: APT OK: 57 packages available for upgrade (0 critical updates). [06:42:34] RECOVERY - db12 APT on db12 is OK: APT OK: 33 packages available for upgrade (0 critical updates). [06:42:50] RECOVERY - ns2 APT on ns2 is OK: APT OK: 5 packages available for upgrade (0 critical updates). [06:47:41] RECOVERY - services1 APT on services1 is OK: APT OK: 7 packages available for upgrade (0 critical updates). [06:48:59] RECOVERY - test2 APT on test2 is OK: APT OK: 14 packages available for upgrade (0 critical updates). [06:49:29] RECOVERY - mw6 APT on mw6 is OK: APT OK: 12 packages available for upgrade (0 critical updates). [06:53:11] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 17.64, 20.45, 19.84 [06:53:50] RECOVERY - cp7 APT on cp7 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [06:53:52] RECOVERY - ns1 APT on ns1 is OK: APT OK: 5 packages available for upgrade (0 critical updates). [06:55:08] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 13.15, 18.18, 19.10 [06:59:36] RECOVERY - mw7 APT on mw7 is OK: APT OK: 12 packages available for upgrade (0 critical updates). [07:04:35] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.57, 5.39, 5.88 [07:10:36] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.24, 5.39, 5.66 [07:14:35] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 3.88, 5.19, 5.56 [07:14:58] [02miraheze/DataDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkreK [07:15:00] [02miraheze/DataDump] 07Universal-Omega 03f7c0093 - Add refresh link [07:15:01] [02DataDump] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/fhhKV [07:15:12] [02DataDump] 07Universal-Omega opened pull request 03#8: Add refresh link (T5474) - 13https://git.io/JkreX [07:16:39] [02miraheze/DataDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/Jkrep [07:16:41] [02miraheze/DataDump] 07Universal-Omega 034110a5d - Add datadump-refresh [07:16:42] [02DataDump] 07Universal-Omega synchronize pull request 03#8: Add refresh link (T5474) - 13https://git.io/JkreX [07:16:47] miraheze/DataDump/Universal-Omega-patch-1/f7c0093 - Universal Omega The build passed. https://travis-ci.com/miraheze/DataDump/builds/203373855 [07:17:59] [02miraheze/DataDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/Jkrvt [07:18:01] [02miraheze/DataDump] 07Universal-Omega 03a29c570 - add datadump-refresh [07:18:02] [02DataDump] 07Universal-Omega synchronize pull request 03#8: Add refresh link (T5474) - 13https://git.io/JkreX [07:22:34] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 10.51, 6.08, 5.59 [07:23:34] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.26, 23.13, 20.53 [07:26:33] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.67, 5.77, 5.59 [07:27:29] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.52, 22.83, 21.22 [07:28:33] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 9.41, 6.84, 5.98 [07:31:25] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 28.93, 23.41, 21.64 [07:33:22] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.18, 22.54, 21.58 [07:34:31] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.85, 5.80, 5.77 [07:35:19] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.38, 24.48, 22.38 [07:40:31] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.81, 5.27, 5.49 [07:41:16] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.26, 23.36, 22.73 [07:45:16] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.27, 23.31, 22.72 [07:46:29] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 3.87, 5.35, 5.56 [07:46:45] [02miraheze/DataDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkrJx [07:46:47] [02miraheze/DataDump] 07Universal-Omega 03c470219 - add author [07:46:48] [02DataDump] 07Universal-Omega synchronize pull request 03#8: Add refresh link (T5474) - 13https://git.io/JkreX [07:47:16] [02miraheze/DataDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkrUe [07:47:18] [02miraheze/DataDump] 07Universal-Omega 037678e44 - add author [07:47:19] [02DataDump] 07Universal-Omega synchronize pull request 03#8: Add refresh link (T5474) - 13https://git.io/JkreX [07:49:16] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.33, 20.41, 21.78 [07:50:28] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.40, 5.52, 5.56 [07:51:15] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.71, 22.25, 22.25 [07:52:28] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 2.53, 4.41, 5.15 [07:53:12] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.04, 21.78, 22.10 [07:54:27] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 3.71, 3.97, 4.88 [07:59:04] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.96, 23.65, 22.76 [08:02:28] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.70, 6.09, 5.45 [08:02:58] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.09, 22.19, 22.46 [08:06:27] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.11, 5.23, 5.25 [08:10:26] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.43, 6.16, 5.61 [08:14:25] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.74, 5.91, 5.70 [08:14:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.95, 21.23, 21.19 [08:16:41] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.09, 19.91, 20.71 [08:24:30] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 18.42, 19.46, 20.21 [08:28:24] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 4.72, 4.71, 5.04 [08:30:19] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.93, 21.70, 20.83 [08:31:15] [02miraheze/DataDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkrLu [08:31:16] [02miraheze/DataDump] 07Universal-Omega 03082ef77 - use single quote [08:31:18] [02DataDump] 07Universal-Omega synchronize pull request 03#8: Add refresh link (T5474) - 13https://git.io/JkreX [08:32:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.83, 5.46, 5.31 [08:34:13] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.31, 21.47, 21.01 [08:40:06] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 28.74, 23.28, 21.68 [08:42:21] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 5.06, 4.97, 5.10 [08:43:52] [02miraheze/DataDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkrtA [08:43:54] [02miraheze/DataDump] 07Universal-Omega 03a4434aa - fix [08:43:56] [02DataDump] 07Universal-Omega synchronize pull request 03#8: Add refresh link (T5474) - 13https://git.io/JkreX [08:46:07] [02miraheze/DataDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkrqO [08:46:09] [02miraheze/DataDump] 07Universal-Omega 031cd5a1b - fix [08:46:10] [02DataDump] 07Universal-Omega synchronize pull request 03#8: Add refresh link (T5474) - 13https://git.io/JkreX [08:46:44] [02miraheze/DataDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkrqZ [08:46:46] [02miraheze/DataDump] 07Universal-Omega 03d92677d - change text [08:46:47] [02DataDump] 07Universal-Omega synchronize pull request 03#8: Add refresh link (T5474) - 13https://git.io/JkreX [08:50:04] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.50, 22.14, 22.34 [08:50:56] [02miraheze/DataDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/Jkrq7 [08:50:57] [02miraheze/DataDump] 07Universal-Omega 03b5feea6 - fix [08:50:59] [02DataDump] 07Universal-Omega synchronize pull request 03#8: Add refresh link (T5474) - 13https://git.io/JkreX [08:51:26] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.90, 5.45, 5.20 [08:51:51] [02miraheze/DataDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/Jkrmv [08:51:53] [02miraheze/DataDump] 07Universal-Omega 035b1cba0 - remove [08:51:54] [02DataDump] 07Universal-Omega synchronize pull request 03#8: Add refresh link (T5474) - 13https://git.io/JkreX [08:53:27] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 3.53, 4.87, 5.03 [09:03:26] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.36, 5.80, 5.19 [09:04:03] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.46, 18.39, 19.94 [09:05:25] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.26, 5.69, 5.23 [09:07:25] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.07, 5.91, 5.36 [09:09:24] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.47, 5.47, 5.27 [09:12:03] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.79, 24.02, 21.73 [09:13:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.37, 6.38, 5.70 [09:14:04] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.31, 22.64, 21.52 [09:15:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.50, 5.85, 5.57 [09:15:40] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JkrOu [09:15:42] [02miraheze/mw-config] 07Universal-Omega 03977d55a - Migrate PageImages to ManageWikiExtensions [09:15:44] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-2 - 13https://git.io/vbvb3 [09:15:50] [02mw-config] 07Universal-Omega opened pull request 03#3465: Migrate PageImages to ManageWikiExtensions - 13https://git.io/JkrOz [09:17:15] [02mw-config] 07Universal-Omega edited pull request 03#3465: Migrate PageImages to ManageWikiExtensions - 13https://git.io/JkrOz [09:17:17] miraheze/mw-config/Universal-Omega-patch-2/977d55a - Universal Omega The build was broken. https://travis-ci.org/miraheze/mw-config/builds/745188394 [09:17:22] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.90, 6.13, 5.70 [09:18:12] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JkrOX [09:18:13] [02miraheze/mw-config] 07Universal-Omega 03fcef66b - fix [09:18:15] [02mw-config] 07Universal-Omega synchronize pull request 03#3465: Migrate PageImages to ManageWikiExtensions - 13https://git.io/JkrOz [09:19:50] miraheze/mw-config/Universal-Omega-patch-2/fcef66b - Universal Omega The build was fixed. https://travis-ci.org/miraheze/mw-config/builds/745188524 [09:20:05] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.05, 21.85, 21.47 [09:21:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.40, 5.89, 5.71 [09:22:03] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.71, 22.16, 21.61 [09:23:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.63, 6.68, 6.03 [09:24:04] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.82, 24.78, 22.66 [09:27:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.58, 5.56, 5.72 [09:28:05] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.25, 22.32, 22.05 [09:36:04] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.97, 22.73, 21.99 [09:38:05] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.39, 20.99, 21.44 [09:41:23] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 3.94, 4.60, 5.10 [09:42:04] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 29.77, 22.65, 21.74 [09:45:22] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.32, 5.41, 5.43 [09:46:03] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.86, 22.69, 21.97 [09:48:03] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.20, 24.67, 22.81 [09:49:22] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.89, 6.17, 5.68 [09:50:04] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 17.64, 21.98, 22.06 [09:52:03] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 32.50, 25.03, 23.09 [09:54:04] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.68, 23.02, 22.61 [09:57:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.21, 5.90, 5.87 [10:02:03] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.01, 23.79, 22.58 [10:05:21] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.12, 5.63, 5.68 [10:06:04] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.53, 22.71, 22.59 [10:07:21] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.04, 5.11, 5.49 [10:08:03] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.82, 24.81, 23.37 [10:09:31] PROBLEM - wiki.openhatch.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.openhatch.org could not be found [10:09:34] PROBLEM - wiki.park-city.club - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.park-city.club could not be found [10:10:05] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.59, 22.45, 22.69 [10:11:20] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.95, 5.93, 5.70 [10:11:38] Hello e11i! If you have any questions, feel free to ask and someone should answer soon. [10:12:03] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.00, 24.12, 23.28 [10:14:03] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.72, 23.51, 23.22 [10:15:19] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.31, 5.77, 5.72 [10:16:23] RECOVERY - wiki.openhatch.org - reverse DNS on sslhost is OK: rDNS OK - wiki.openhatch.org reverse DNS resolves to cp6.miraheze.org [10:16:28] RECOVERY - wiki.park-city.club - reverse DNS on sslhost is OK: rDNS OK - wiki.park-city.club reverse DNS resolves to cp7.miraheze.org [10:23:20] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 3.86, 4.37, 5.09 [10:24:04] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.67, 22.31, 22.14 [10:26:04] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.82, 20.82, 21.58 [10:28:07] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 39.92, 26.36, 23.36 [10:28:18] PROBLEM - cp6 APT on cp6 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (5 critical updates). [10:31:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.95, 5.39, 5.10 [10:33:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.40, 5.56, 5.21 [10:35:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.01, 5.56, 5.22 [10:36:03] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.47, 23.74, 23.55 [10:37:23] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 4.95, 5.07, 5.07 [10:48:03] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 18.22, 18.53, 20.35 [10:50:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.31, 5.79, 5.40 [10:52:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.22, 5.79, 5.43 [10:54:05] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.04, 20.99, 20.78 [10:54:22] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.20, 5.10, 5.22 [10:56:05] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 13.89, 18.58, 19.94 [10:56:22] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 4.14, 4.59, 5.01 [11:00:21] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.40, 5.41, 5.25 [11:04:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.05, 5.40, 5.27 [11:06:57] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.12, 22.37, 20.95 [11:08:23] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 1.94, 3.83, 4.69 [11:08:55] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 13.54, 19.17, 19.95 [11:12:46] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.41, 20.01, 20.05 [11:14:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.46, 5.46, 5.06 [11:16:23] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 4.62, 4.86, 4.87 [11:16:40] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.62, 18.30, 19.40 [11:22:33] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 28.40, 21.63, 20.17 [11:24:30] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 18.27, 19.28, 19.44 [11:27:20] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.22, 5.74, 5.31 [11:29:19] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 3.58, 4.63, 4.94 [11:31:41] PROBLEM - gluster1 APT on gluster1 is CRITICAL: APT CRITICAL: 14 packages available for upgrade (5 critical updates). [11:41:16] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.54, 21.26, 20.36 [11:43:40] PROBLEM - ns1 Current Load on ns1 is CRITICAL: CRITICAL - load average: 1.89, 3.22, 1.72 [11:45:11] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.68, 24.53, 21.95 [11:45:40] RECOVERY - ns1 Current Load on ns1 is OK: OK - load average: 0.06, 1.48, 1.35 [11:47:09] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.99, 23.30, 21.82 [11:51:04] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 35.29, 25.03, 22.50 [12:00:21] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.34, 5.52, 5.01 [12:02:22] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.73, 5.53, 5.10 [12:04:23] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 3.16, 5.07, 5.02 [12:13:27] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.14, 4.87, 4.88 [12:15:27] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.74, 5.80, 5.20 [12:17:27] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.52, 5.17, 5.05 [12:21:27] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 9.57, 7.29, 5.87 [12:29:25] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.54, 5.46, 5.55 [12:30:15] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.06, 21.83, 23.53 [12:33:24] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.64, 5.31, 5.40 [12:34:10] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.78, 23.35, 23.68 [12:35:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.63, 4.97, 5.26 [12:38:23] PROBLEM - cp3 APT on cp3 is CRITICAL: APT CRITICAL: 9 packages available for upgrade (4 critical updates). [12:41:22] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 3.55, 4.35, 4.91 [12:42:11] PROBLEM - db11 APT on db11 is CRITICAL: APT CRITICAL: 38 packages available for upgrade (5 critical updates). [12:45:22] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.54, 5.90, 5.45 [12:49:31] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.43, 5.78, 5.52 [12:53:30] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.57, 6.13, 5.70 [12:54:24] PROBLEM - phab1 APT on phab1 is CRITICAL: APT CRITICAL: 11 packages available for upgrade (5 critical updates). [12:55:30] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.72, 5.89, 5.69 [13:11:32] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 3.56, 4.76, 5.10 [13:20:32] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.94, 5.45, 5.23 [13:22:09] PROBLEM - wiki.abyogsms.org.in - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.abyogsms.org.in All nameservers failed to answer the query. [13:24:18] PROBLEM - wiki.abyogsms.org.in - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:24:32] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.80, 5.77, 5.40 [13:26:31] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.41, 5.53, 5.35 [13:28:31] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.54, 6.08, 5.56 [13:30:30] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.28, 5.45, 5.40 [13:30:56] RECOVERY - wiki.abyogsms.org.in - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.abyogsms.org.in' will expire on Sun 17 Jan 2021 03:52:35 GMT +0000. [13:32:30] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.88, 5.87, 5.55 [13:34:08] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.96, 22.85, 23.98 [13:34:29] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.42, 5.41, 5.43 [13:35:27] RECOVERY - wiki.abyogsms.org.in - reverse DNS on sslhost is OK: rDNS OK - wiki.abyogsms.org.in reverse DNS resolves to cp6.miraheze.org [13:40:28] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 3.33, 4.60, 5.10 [13:42:04] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.74, 21.30, 22.39 [13:44:04] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.04, 20.43, 21.96 [13:44:27] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.39, 5.22, 5.25 [13:48:27] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 46.37, 28.10, 24.18 [13:54:50] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 3.23, 4.74, 5.02 [13:56:17] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.70, 23.34, 23.49 [14:02:10] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 29.39, 22.98, 22.80 [14:02:52] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 8.70, 6.52, 5.50 [14:04:07] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.67, 21.79, 22.40 [14:06:52] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.77, 5.48, 5.32 [14:08:53] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.14, 6.05, 5.56 [14:10:10] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.00, 22.96, 22.43 [14:10:54] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.50, 5.52, 5.43 [14:12:53] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.02, 5.81, 5.54 [14:14:39] PROBLEM - mw6 Disk Space on mw6 is WARNING: DISK WARNING - free space: / 2087 MB (10% inode=73%); [14:14:53] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.00, 5.43, 5.43 [14:20:06] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 14.89, 21.26, 22.54 [14:20:54] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.98, 5.69, 5.44 [14:22:06] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 32.97, 24.36, 23.42 [14:24:04] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.77, 23.51, 23.24 [14:26:54] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 3.36, 5.53, 5.57 [14:30:54] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.71, 6.43, 5.89 [14:32:05] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 32.83, 26.59, 24.21 [14:38:07] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.56, 23.59, 23.68 [14:38:54] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.41, 5.50, 5.77 [14:40:06] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 33.65, 27.06, 24.93 [14:42:57] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 8.96, 6.52, 6.06 [14:46:33] * hispano76 greetings [14:46:57] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.65, 5.97, 5.97 [14:48:56] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.03, 6.30, 6.10 [14:56:56] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 3.46, 5.25, 5.85 [15:00:55] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.57, 5.53, 5.80 [15:02:55] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.87, 5.95, 5.93 [15:11:33] PROBLEM - rdb2 APT on rdb2 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (5 critical updates). [15:15:04] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 13.61, 7.78, 5.16 [15:17:02] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 2.14, 3.57, 4.76 [15:17:02] PROBLEM - cp7 Current Load on cp7 is WARNING: WARNING - load average: 7.77, 8.00, 5.58 [15:19:06] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 4.45, 6.66, 5.39 [15:30:22] [02miraheze/CreateWiki] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JkofX [15:30:24] [02miraheze/CreateWiki] 07paladox 031152d4f - Add file_exists check for creating database/deleted.json [15:30:25] [02CreateWiki] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vpJTL [15:30:26] [02CreateWiki] 07paladox opened pull request 03#173: Add file_exists check for creating database/deleted.json - 13https://git.io/JkofM [15:34:29] [02DataDump] 07paladox closed pull request 03#8: Add refresh link (T5474) - 13https://git.io/JkreX [15:34:31] [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/JkoJV [15:34:32] [02miraheze/DataDump] 07Universal-Omega 034f9d663 - Add refresh link (T5474) (#8) * Add refresh link * Add datadump-refresh * add datadump-refresh * add author * add author * use single quote * fix * fix * change text * fix * remove [15:34:34] [02DataDump] 07paladox deleted branch 03Universal-Omega-patch-1 - 13https://git.io/fhhKV [15:34:35] [02miraheze/DataDump] 07paladox deleted branch 03Universal-Omega-patch-1 [15:42:30] PROBLEM - jobrunner1 APT on jobrunner1 is CRITICAL: APT CRITICAL: 17 packages available for upgrade (5 critical updates). [15:50:05] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 8.74, 6.70, 5.41 [15:52:57] PROBLEM - wiki.pablojprieto.com.es - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.pablojprieto.com.es could not be found [15:52:58] PROBLEM - wikilukas.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wikilukas.tk could not be found [15:56:04] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.59, 5.95, 5.48 [15:58:34] PROBLEM - test2 Puppet on test2 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 6 minutes ago with 0 failures [15:59:52] RECOVERY - wiki.pablojprieto.com.es - reverse DNS on sslhost is OK: rDNS OK - wiki.pablojprieto.com.es reverse DNS resolves to cp7.miraheze.org [15:59:54] RECOVERY - wikilukas.tk - reverse DNS on sslhost is OK: rDNS OK - wikilukas.tk reverse DNS resolves to cp6.miraheze.org [16:00:05] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.18, 22.69, 23.82 [16:00:05] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.85, 6.02, 5.52 [16:02:04] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 30.57, 25.62, 24.76 [16:04:13] PROBLEM - gluster2 APT on gluster2 is CRITICAL: APT CRITICAL: 18 packages available for upgrade (5 critical updates). [16:12:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.85, 5.66, 5.75 [16:15:11] [02miraheze/DataDump] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkotR [16:15:13] [02miraheze/DataDump] 07Universal-Omega 039308deb - Convert global to ConfigRegistry [16:15:14] [02DataDump] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/fhhKV [16:15:16] [02DataDump] 07Universal-Omega opened pull request 03#9: Convert global to ConfigRegistry - 13https://git.io/Jkot0 [16:16:24] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.26, 5.69, 5.70 [16:17:47] [02DataDump] 07paladox closed pull request 03#9: Convert global to ConfigRegistry - 13https://git.io/Jkot0 [16:17:49] [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkotS [16:17:50] [02miraheze/DataDump] 07Universal-Omega 03328250f - Convert global to ConfigRegistry (#9) [16:17:52] [02miraheze/DataDump] 07paladox deleted branch 03Universal-Omega-patch-1 [16:17:53] [02DataDump] 07paladox deleted branch 03Universal-Omega-patch-1 - 13https://git.io/fhhKV [16:18:24] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.01, 5.08, 5.48 [16:19:27] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jkoqf [16:19:28] [02miraheze/mediawiki] 07paladox 030949aa1 - Update CookieWarning to 87dfc43f8d79203eb5798532b6579dbea4ef5662 [16:26:23] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.75, 5.77, 5.50 [16:28:22] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.38, 5.77, 5.55 [16:32:22] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 9.77, 6.32, 5.72 [16:34:24] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.58, 5.89, 5.64 [16:36:12] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/Jkomx [16:36:14] [02miraheze/mw-config] 07Universal-Omega 037c3724c - fix [16:36:15] [02mw-config] 07Universal-Omega synchronize pull request 03#3465: Migrate PageImages to ManageWikiExtensions - 13https://git.io/JkrOz [16:38:12] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JkoYO [16:38:13] [02miraheze/mw-config] 07Universal-Omega 0327bf2a1 - add pageimages [16:38:15] [02mw-config] 07Universal-Omega synchronize pull request 03#3465: Migrate PageImages to ManageWikiExtensions - 13https://git.io/JkrOz [16:38:33] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JkoYZ [16:38:35] [02miraheze/mw-config] 07Universal-Omega 03e7c57cb - fix [16:38:36] [02mw-config] 07Universal-Omega synchronize pull request 03#3465: Migrate PageImages to ManageWikiExtensions - 13https://git.io/JkrOz [16:38:37] PROBLEM - rdb1 APT on rdb1 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (5 critical updates). [16:39:58] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JkoYu [16:39:59] [02miraheze/mw-config] 07Universal-Omega 035a6c8dd - add pageimages [16:40:01] [02mw-config] 07Universal-Omega synchronize pull request 03#3465: Migrate PageImages to ManageWikiExtensions - 13https://git.io/JkrOz [16:42:37] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 8.43, 6.53, 5.85 [16:43:29] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 51.77.107.210/cpweb [16:43:36] PROBLEM - mon1 Current Load on mon1 is CRITICAL: CRITICAL - load average: 4.18, 3.62, 2.83 [16:43:57] PROBLEM - mw6 Current Load on mw6 is CRITICAL: CRITICAL - load average: 9.78, 7.48, 5.41 [16:44:37] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.15, 5.59, 5.59 [16:45:28] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [16:45:33] RECOVERY - mon1 Current Load on mon1 is OK: OK - load average: 2.17, 3.13, 2.75 [16:45:47] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JkoOZ [16:45:49] [02miraheze/mw-config] 07Universal-Omega 030babf19 - fix [16:45:50] [02mw-config] 07Universal-Omega synchronize pull request 03#3465: Migrate PageImages to ManageWikiExtensions - 13https://git.io/JkrOz [16:45:54] RECOVERY - mw6 Current Load on mw6 is OK: OK - load average: 3.75, 6.08, 5.16 [16:48:32] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JkoOw [16:48:34] [02miraheze/mw-config] 07Universal-Omega 0396448c4 - fix [16:48:35] [02mw-config] 07Universal-Omega synchronize pull request 03#3465: Migrate PageImages to ManageWikiExtensions - 13https://git.io/JkrOz [16:50:35] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 7.61, 6.17, 5.79 [16:51:08] [02mw-config] 07Universal-Omega closed pull request 03#3465: Migrate PageImages to ManageWikiExtensions - 13https://git.io/JkrOz [16:51:09] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/JkoOx [16:51:11] [02miraheze/mw-config] 07Universal-Omega 03c414585 - Migrate PageImages to ManageWikiExtensions (#3465) * Migrate PageImages to ManageWikiExtensions * fix * fix * add pageimages * fix * add pageimages * fix * fix [16:52:35] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 5.39, 5.76, 5.68 [16:54:07] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 - 13https://git.io/vbvb3 [16:54:09] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 [16:54:34] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 5.26, 6.01, 5.81 [16:55:15] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/Jko34 [16:55:16] [02miraheze/mw-config] 07Universal-Omega 032b03eea - popups requires pageimages [16:55:18] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-2 - 13https://git.io/vbvb3 [16:55:21] [02mw-config] 07Universal-Omega opened pull request 03#3466: popups requires pageimages - 13https://git.io/Jko3B [16:55:50] [02mw-config] 07Universal-Omega closed pull request 03#3466: popups requires pageimages - 13https://git.io/Jko3B [16:55:51] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jko30 [16:55:53] [02miraheze/mw-config] 07Universal-Omega 035b14923 - popups requires pageimages (#3466) [16:56:34] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.44, 5.30, 5.57 [16:58:49] !log universalomega@jobrunner1:~$ sudo -u www-data /usr/local/bin/foreachwikiindblist /home/universalomega/popups.json /srv/mediawiki/w/extensions/ManageWiki/maintenance/toggleExtension.php pageimages [16:58:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:59:19] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 - 13https://git.io/vbvb3 [16:59:21] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 [17:02:40] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.72, 5.72, 5.61 [17:06:39] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 4.51, 5.09, 5.37 [17:08:39] PROBLEM - jobrunner2 Current Load on jobrunner2 is CRITICAL: CRITICAL - load average: 6.57, 5.49, 5.47 [17:10:38] PROBLEM - jobrunner2 Current Load on jobrunner2 is WARNING: WARNING - load average: 2.66, 4.48, 5.11 [17:11:01] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JkoGL [17:11:02] [02miraheze/mw-config] 07Universal-Omega 0372a2547 - Popups require TextExtracts rather then enable with it [17:11:04] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-2 - 13https://git.io/vbvb3 [17:11:05] [02mw-config] 07Universal-Omega opened pull request 03#3467: Popups require TextExtracts rather then enable with it - 13https://git.io/JkoGt [17:12:33] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JkoGZ [17:12:34] [02miraheze/mw-config] 07Universal-Omega 0320b2d1d - Remove TextExtracts from including with popups [17:12:36] [02mw-config] 07Universal-Omega synchronize pull request 03#3467: Popups require TextExtracts rather then enable with it - 13https://git.io/JkoGt [17:12:38] RECOVERY - jobrunner2 Current Load on jobrunner2 is OK: OK - load average: 1.95, 3.62, 4.72 [17:13:50] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JkoGW [17:13:52] [02miraheze/mw-config] 07Universal-Omega 03ea92993 - fix [17:13:53] [02mw-config] 07Universal-Omega synchronize pull request 03#3467: Popups require TextExtracts rather then enable with it - 13https://git.io/JkoGt [17:19:31] [02mw-config] 07Universal-Omega closed pull request 03#3467: Popups require TextExtracts rather then enable with it - 13https://git.io/JkoGt [17:19:32] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JkoZk [17:19:34] [02miraheze/mw-config] 07Universal-Omega 03e8afe98 - Popups require TextExtracts rather then enable with it (#3467) * Popups require TextExtracts rather then enable with it * Remove TextExtracts from including with popups * fix [17:22:27] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 - 13https://git.io/vbvb3 [17:22:28] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 [17:23:52] !log universalomega@jobrunner1:~$ sudo -u www-data /usr/local/bin/foreachwikiindblist /home/universalomega/popups.json /srv/mediawiki/w/extensions/ManageWiki/maintenance/toggleExtension.php textextracts [17:23:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:32:04] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.67, 18.76, 23.26 [17:43:10] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JkocS [17:43:12] [02miraheze/mw-config] 07Universal-Omega 037d64bfa - Cleanup multiple wfLoadExtension(s) [17:43:13] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-2 - 13https://git.io/vbvb3 [17:43:16] [02mw-config] 07Universal-Omega opened pull request 03#3468: Cleanup multiple wfLoadExtension(s) - 13https://git.io/Jkoc9 [17:43:34] miraheze/mw-config/Universal-Omega-patch-2/ea92993 - Universal Omega The build has errored. https://travis-ci.org/miraheze/mw-config/builds/745247446 [17:44:03] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.41, 19.78, 20.88 [17:46:04] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.55, 19.67, 20.73 [17:47:51] reception123: so do you have any warnings in particular youre looking for? [17:48:28] PROBLEM - mail1 Puppet on mail1 is WARNING: WARNING: Puppet is currently disabled, message: John, last run 6 minutes ago with 0 failures [17:52:03] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.57, 18.12, 19.86 [17:53:54] Naleksuh: Template:Uw-username would be useful [17:54:18] Yeah I tried to find a list of warnings but there wasnt much other than search results which werent really helpful [17:57:14] Naleksuh: Why weren't they helpful? [17:57:26] we haven't got many warnings, the search results I gave you are basically it [18:00:47] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-3 [+0/-0/±1] 13https://git.io/JkoWr [18:00:48] [02miraheze/mw-config] 07Universal-Omega 0390d3f75 - VEForAll requires commentstreams OR pageforms [18:00:50] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-3 - 13https://git.io/vbvb3 [18:00:52] [02mw-config] 07Universal-Omega opened pull request 03#3469: VEForAll requires commentstreams OR pageforms - 13https://git.io/JkoWo [18:02:04] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.27, 20.95, 19.79 [18:02:37] [02mw-config] 07Universal-Omega closed pull request 03#3469: VEForAll requires commentstreams OR pageforms - 13https://git.io/JkoWo [18:02:39] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkoWM [18:02:40] [02miraheze/mw-config] 07Universal-Omega 03e4bfa81 - VEForAll requires commentstreams OR pageforms (#3469) [18:02:42] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-3 - 13https://git.io/vbvb3 [18:02:43] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-3 [18:04:05] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 17.82, 19.72, 19.50 [18:13:08] [02mw-config] 07Universal-Omega closed pull request 03#3468: Cleanup multiple wfLoadExtension(s) - 13https://git.io/Jkoc9 [18:13:09] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkolH [18:13:11] [02miraheze/mw-config] 07Universal-Omega 03ad37cac - Cleanup multiple wfLoadExtension(s) (#3468) [18:13:12] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 - 13https://git.io/vbvb3 [18:13:14] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 [18:18:36] miraheze/mw-config/Universal-Omega-patch-2/7d64bfa - Universal Omega The build has errored. https://travis-ci.org/miraheze/mw-config/builds/745251293 [18:23:07] PROBLEM - mon1 Current Load on mon1 is WARNING: WARNING - load average: 3.95, 3.06, 2.41 [18:24:03] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.61, 20.86, 19.67 [18:25:05] RECOVERY - mon1 Current Load on mon1 is OK: OK - load average: 2.81, 2.91, 2.44 [18:26:03] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.16, 18.56, 18.99 [18:27:09] evening, folks [18:29:18] hi @SPF|Cloud. Nice to see you. :) [18:29:56] hey :) [18:38:23] miraheze/mw-config/Universal-Omega-patch-3/90d3f75 - Universal Omega The build has errored. https://travis-ci.org/miraheze/mw-config/builds/745253631 [18:55:32] miraheze/mw-config/master/ad37cac - Universal Omega The build passed. https://travis-ci.org/miraheze/mw-config/builds/745255223 [19:02:46] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 31.27, 24.43, 20.62 [19:04:43] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.06, 22.59, 20.39 [19:08:38] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 28.90, 24.05, 21.40 [19:10:35] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.56, 23.38, 21.51 [19:11:33] Hi @southparkfan [19:12:32] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 38.73, 29.11, 23.80 [19:16:27] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.49, 23.87, 22.84 [19:18:24] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.07, 25.47, 23.57 [19:22:51] PROBLEM - ns1 APT on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:24:21] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.54, 21.72, 22.81 [19:24:49] RECOVERY - ns1 APT on ns1 is OK: APT OK: 5 packages available for upgrade (0 critical updates). [19:32:11] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.13, 19.97, 21.03 [19:34:10] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 17.49, 18.81, 20.49 [19:36:07] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.18, 17.79, 19.90 [19:51:56] PROBLEM - graylog1 Puppet on graylog1 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 10 minutes ago with 0 failures [19:52:05] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.53, 21.80, 20.08 [19:54:05] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 17.27, 20.22, 19.72 [20:02:39] PROBLEM - graylog1 HTTPS on graylog1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 311 bytes in 0.006 second response time [20:10:47] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://git.io/Jkoay [20:10:48] [02miraheze/puppet] 07paladox 03695a17e - graylog: Allow only ssh tunnel to access web ui [20:10:50] [02puppet] 07paladox created branch 03paladox-patch-3 - 13https://git.io/vbiAS [20:10:51] [02puppet] 07paladox opened pull request 03#1521: graylog: Allow only ssh tunnel to access web ui - 13https://git.io/JkoaS [20:11:25] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-1/±0] 13https://git.io/JkoaQ [20:11:26] [02miraheze/puppet] 07paladox 039081592 - Delete graylog.miraheze.org.conf [20:11:28] [02puppet] 07paladox synchronize pull request 03#1521: graylog: Allow only ssh tunnel to access web ui - 13https://git.io/JkoaS [20:13:49] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 31.97, 22.93, 20.60 [20:16:34] PROBLEM - mw7 Disk Space on mw7 is WARNING: DISK WARNING - free space: / 1967 MB (10% inode=73%); [20:17:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.87, 21.95, 20.97 [20:21:42] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 17.87, 19.66, 20.18 [20:26:34] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 16.92, 8.35, 4.46 [20:27:31] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.77, 20.91, 20.51 [20:28:34] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 3.10, 5.97, 4.07 [20:28:41] RECOVERY - graylog1 HTTPS on graylog1 is OK: HTTP OK: HTTP/1.1 200 OK - 1670 bytes in 0.011 second response time [20:29:28] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 17.87, 19.42, 19.99 [20:39:16] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.70, 20.63, 20.02 [20:45:13] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 13.51, 19.07, 19.86 [20:49:04] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.35, 21.00, 20.51 [20:54:56] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.59, 19.09, 20.13 [21:13:32] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.33, 23.02, 20.73 [21:14:38] PROBLEM - mw6 Disk Space on mw6 is CRITICAL: DISK CRITICAL - free space: / 1111 MB (5% inode=73%); [21:15:29] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.41, 21.35, 20.37 [21:16:51] PROBLEM - mw5 Disk Space on mw5 is WARNING: DISK WARNING - free space: / 2087 MB (10% inode=73%); [21:17:26] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.08, 19.56, 19.85 [21:23:18] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.05, 21.63, 20.55 [21:25:17] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 19.30, 20.04, 20.06 [21:29:15] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.18, 21.41, 20.76 [21:29:15] !log remove allow 443 on graylog1 [21:29:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:33:10] PROBLEM - graylog1 HTTPS on graylog1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:39:00] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 13.37, 18.52, 20.18 [21:49:43] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.64, 20.61, 20.61 [21:53:37] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.47, 19.64, 20.27 [22:08:13] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://git.io/JkoDi [22:08:14] [02miraheze/puppet] 07paladox 03eeacef2 - graylog: Allow mediawiki servers to access port 443 [22:08:16] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://git.io/vbiAS [22:08:17] [02puppet] 07paladox opened pull request 03#1522: graylog: Allow mediawiki servers to access port 443 - 13https://git.io/JkoDP [22:08:42] [02puppet] 07paladox closed pull request 03#1522: graylog: Allow mediawiki servers to access port 443 - 13https://git.io/JkoDP [22:08:43] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkoDy [22:08:45] [02miraheze/puppet] 07paladox 03abbeeaa - graylog: Allow mediawiki servers to access port 443 (#1522) [22:08:46] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://git.io/vbiAS [22:08:48] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [22:09:51] RECOVERY - graylog1 Puppet on graylog1 is OK: OK: Puppet is currently enabled, last run 17 seconds ago with 0 failures [22:12:10] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 30.35, 23.54, 21.26 [22:16:04] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.44, 21.88, 21.18 [22:16:43] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/Jkoyo [22:16:45] [02miraheze/mw-config] 07paladox 0354df160 - Introduce new $wmgCentralLoggingSystem variable Enable on test2 and jobrunner[12] [22:16:46] [02mw-config] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbvb3 [22:16:48] [02mw-config] 07paladox opened pull request 03#3470: Introduce new $wmgCentralLoggingSystem variable - 13https://git.io/JkoyK [22:17:20] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JkoyP [22:17:22] [02miraheze/mw-config] 07paladox 0311da9e2 - Update GlobalLogging.php [22:17:23] [02mw-config] 07paladox synchronize pull request 03#3470: Introduce new $wmgCentralLoggingSystem variable - 13https://git.io/JkoyK [22:19:19] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JkoyF [22:19:21] [02miraheze/mw-config] 07paladox 039c384a0 - Enable new logging system on jobrunner1 [22:19:22] [02mw-config] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vbvb3 [22:19:24] [02mw-config] 07paladox opened pull request 03#3471: Enable new logging system on jobrunner1 - 13https://git.io/Jkoyb [22:20:04] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.79, 18.67, 19.96 [22:21:09] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://git.io/JkoSe [22:21:10] [02miraheze/puppet] 07paladox 038018448 - Enable new central logging system on jobrunner1 [22:21:12] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://git.io/vbiAS [22:21:13] [02puppet] 07paladox opened pull request 03#1523: Enable new central logging system on jobrunner1 - 13https://git.io/JkoSv [22:22:13] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkoS3 [22:22:15] [02miraheze/puppet] 07Southparkfan 034851c22 - Puppetise graylog1 ufw rules for real [22:24:46] [02mw-config] 07paladox closed pull request 03#3471: Enable new logging system on jobrunner1 - 13https://git.io/Jkoyb [22:24:47] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkoSC [22:24:49] [02miraheze/puppet] 07Southparkfan 03eebb41e - Use quotes correctly [22:24:50] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JkoSW [22:24:52] [02miraheze/mw-config] 07paladox 03be70440 - Enable new logging system on jobrunner1 (#3471) [22:24:53] [02mw-config] 07paladox synchronize pull request 03#3470: Introduce new $wmgCentralLoggingSystem variable - 13https://git.io/JkoyK [22:24:55] [02mw-config] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vbvb3 [22:24:56] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-2 [22:25:31] [02puppet] 07paladox closed pull request 03#1523: Enable new central logging system on jobrunner1 - 13https://git.io/JkoSv [22:25:33] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkoSz [22:25:34] [02miraheze/puppet] 07paladox 0366656b3 - Enable new central logging system on jobrunner1 (#1523) [22:25:47] RECOVERY - graylog1 HTTPS on graylog1 is OK: HTTP OK: HTTP/1.1 200 OK - 1670 bytes in 0.012 second response time [22:26:52] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://git.io/vbiAS [22:26:53] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [22:28:49] PROBLEM - jobrunner1 Puppet on jobrunner1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [22:31:49] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.52, 23.37, 21.36 [22:35:46] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.09, 23.80, 22.19 [22:36:55] df -h [22:36:58] whoops :P [22:38:24] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jko9Q [22:38:26] [02miraheze/puppet] 07paladox 037d9c7dd - Fix "Could not find resource 'Service[rsyslog]' in parameter 'notify'" [22:38:58] miraheze/mw-config/paladox-patch-2/9c384a0 - paladox The build has errored. https://travis-ci.org/miraheze/mw-config/builds/745285951 [22:40:49] RECOVERY - jobrunner1 Puppet on jobrunner1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:43:35] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.36, 24.75, 22.96 [22:45:32] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.23, 22.97, 22.56 [22:52:25] miraheze/mw-config/paladox-patch-1/be70440 - paladox The build passed. https://travis-ci.org/miraheze/mw-config/builds/745286844 [22:52:36] RECOVERY - test2 Puppet on test2 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:53:44] !log restart jobrunner/jobchron for syslog test [22:53:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:54:49] PROBLEM - jobrunner1 Puppet on jobrunner1 is WARNING: WARNING: Puppet is currently disabled, message: SPF, last run 2 minutes ago with 0 failures [22:55:18] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 10.88, 15.92, 19.40 [22:56:35] PROBLEM - test2 Puppet on test2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki config] [22:58:04] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JkoQw [22:58:05] [02miraheze/puppet] 07Southparkfan 033480179 - jobrunner/jobchron: redirect stdout/stderr to syslog [22:58:49] RECOVERY - jobrunner1 Puppet on jobrunner1 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [23:02:27] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 3 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.222.27.129/cpweb [23:03:16] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.54, 21.23, 20.09 [23:04:24] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [23:05:13] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.68, 19.69, 19.68 [23:10:00] !log remove all php* files in /tmp on mediawiki servers [23:10:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:10:32] RECOVERY - mw7 Disk Space on mw7 is OK: DISK OK - free space: / 5912 MB (31% inode=73%); [23:10:38] RECOVERY - mw6 Disk Space on mw6 is OK: DISK OK - free space: / 5942 MB (31% inode=73%); [23:10:51] RECOVERY - mw5 Disk Space on mw5 is OK: DISK OK - free space: / 5942 MB (31% inode=73%); [23:17:34] !log install security updates (mainly krb5) [23:17:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:18:13] RECOVERY - gluster2 APT on gluster2 is OK: APT OK: 13 packages available for upgrade (0 critical updates). [23:18:39] RECOVERY - rdb1 APT on rdb1 is OK: APT OK: 5 packages available for upgrade (0 critical updates). [23:19:26] RECOVERY - jobrunner1 APT on jobrunner1 is OK: APT OK: 12 packages available for upgrade (0 critical updates). [23:19:35] RECOVERY - rdb2 APT on rdb2 is OK: APT OK: 5 packages available for upgrade (0 critical updates). [23:20:09] RECOVERY - cp3 APT on cp3 is OK: APT OK: 5 packages available for upgrade (0 critical updates). [23:20:11] RECOVERY - db11 APT on db11 is OK: APT OK: 33 packages available for upgrade (0 critical updates). [23:20:21] RECOVERY - phab1 APT on phab1 is OK: APT OK: 6 packages available for upgrade (0 critical updates). [23:21:38] RECOVERY - gluster1 APT on gluster1 is OK: APT OK: 9 packages available for upgrade (0 critical updates). [23:22:18] RECOVERY - cp6 APT on cp6 is OK: APT OK: 5 packages available for upgrade (0 critical updates). [23:28:13] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkodY [23:28:15] [02miraheze/puppet] 07Southparkfan 035b9793b - ssh: remove deprecated UsePrivilegeSeparation Per https://www.openssh.com/txt/release-7.5, "This release deprecates the sshd_config UsePrivilegeSeparation option, thereby making privilege separation mandatory. Privilege separation has been on by default for almost 15 years and sandboxing has been on by default for almost the last five." [23:33:37] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.19, 20.94, 19.35 [23:35:35] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.13, 18.31, 18.57 [23:41:24] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.44, 19.69, 18.94 [23:41:48] @dmehus , @Universal_Omega , RhinosF1 If one of you have a chance, I'd like to go over something with you. I can do all of you, or none, depending on availability [23:42:04] Just a local CoC for my wikis [23:43:22] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 19.38, 20.14, 19.23 [23:44:46] PROBLEM - mon1 Current Load on mon1 is CRITICAL: CRITICAL - load average: 4.48, 3.77, 3.02 [23:46:44] RECOVERY - mon1 Current Load on mon1 is OK: OK - load average: 2.03, 3.06, 2.85 [23:53:08] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.67, 24.16, 21.02 [23:55:05] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 13.68, 20.21, 19.96