[00:02:40] PROBLEM - cloud1 Current Load on cloud1 is CRITICAL: CRITICAL - load average: 34.68, 21.51, 14.61 [00:04:37] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 16.15, 19.33, 14.66 [00:05:03] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[openssh-client] [00:13:02] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:24:36] @Naleksuh We fixed that with this edit https://meta.miraheze.org/w/index.php?title=Meta:Administrators&diff=146802&oldid=146626&diffmode=source [00:24:37] [ Difference between revisions of "Meta:Administrators" - Miraheze Meta ] - meta.miraheze.org [02:08:47] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkS8s [02:08:48] [02miraheze/mediawiki] 07Universal-Omega 0391304d5 - Updating TreeAndMenu [02:08:50] [02mediawiki] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/vbL5b [02:09:32] [02mediawiki] 07Universal-Omega opened pull request 03#578: Updating TreeAndMenu - 13https://git.io/JkS8C [02:10:44] [02mediawiki] 07Universal-Omega closed pull request 03#578: Updating TreeAndMenu - 13https://git.io/JkS8C [02:10:46] [02miraheze/mediawiki] 07Universal-Omega pushed 032 commits to 03REL1_35 [+0/-0/±2] 13https://git.io/JkS8g [02:10:47] [02miraheze/mediawiki] 07Universal-Omega 03bc8e644 - Merge pull request #578 from miraheze/Universal-Omega-patch-1 Updating TreeAndMenu [02:10:49] [02mediawiki] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://git.io/vbL5b [02:10:50] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [02:11:24] [02mediawiki] 07Universal-Omega closed pull request 03#536: Bump extensions/LinkTitles from `f8f0034` to `c1bc8f8` - 13https://git.io/Jkuot [02:11:25] [02miraheze/mediawiki] 07Universal-Omega pushed 032 commits to 03REL1_35 [+0/-0/±2] 13https://git.io/JkS8w [02:11:27] [02miraheze/mediawiki] 07Universal-Omega 0325b4658 - Merge pull request #536 from miraheze/dependabot/submodules/REL1_35/extensions/LinkTitles-c1bc8f8 Bump extensions/LinkTitles from `f8f0034` to `c1bc8f8` [02:11:28] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/LinkTitles-c1bc8f8 - 13https://git.io/vbL5b [02:11:30] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/LinkTitles-c1bc8f8 [02:14:49] PROBLEM - cp9 Stunnel Http for mw7 on cp9 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 344 bytes in 0.241 second response time [02:14:58] PROBLEM - cp7 Stunnel Http for mw7 on cp7 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 344 bytes in 0.002 second response time [02:15:31] PROBLEM - cp6 Varnish Backends on cp6 is CRITICAL: 1 backends are down. mw7 [02:15:32] PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 1 backends are down. mw7 [02:15:34] PROBLEM - cp6 Stunnel Http for mw7 on cp6 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 344 bytes in 0.003 second response time [02:15:35] PROBLEM - cp9 Varnish Backends on cp9 is CRITICAL: 1 backends are down. mw7 [02:15:48] PROBLEM - mw7 MediaWiki Rendering on mw7 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 344 bytes in 0.005 second response time [02:16:31] PROBLEM - cp3 Stunnel Http for mw7 on cp3 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 344 bytes in 0.693 second response time [02:16:44] PROBLEM - cp7 Varnish Backends on cp7 is CRITICAL: 1 backends are down. mw7 [02:20:11] PROBLEM - mw6 Puppet on mw6 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [02:20:17] PROBLEM - mw4 Puppet on mw4 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [02:21:46] PROBLEM - mw5 Puppet on mw5 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [02:23:45] RECOVERY - mw5 Puppet on mw5 is OK: OK: Puppet is currently enabled, last run 47 seconds ago with 0 failures [02:24:11] RECOVERY - mw6 Puppet on mw6 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:24:14] RECOVERY - mw4 Puppet on mw4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:16:34] PROBLEM - en.phgalaxy.xyz - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'en.phgalaxy.xyz' expires in 15 day(s) (Sat 12 Dec 2020 04:12:20 GMT +0000). [04:24:02] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkSwV [04:24:03] [02miraheze/ssl] 07MirahezeSSLBot 0364ce728 - Bot: Update SSL cert for en.phgalaxy.xyz [04:36:54] RECOVERY - en.phgalaxy.xyz - LetsEncrypt on sslhost is OK: OK - Certificate 'en.phgalaxy.xyz' will expire on Wed 24 Feb 2021 03:23:53 GMT +0000. [04:45:02] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[gdnsd-syntax] [04:53:03] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:37:51] [02miraheze/MirahezeMagic] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkSM3 [05:37:52] [02miraheze/MirahezeMagic] 07Universal-Omega 035754179 - remove redundant function [05:37:54] [02MirahezeMagic] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/fQRGX [05:38:15] [02MirahezeMagic] 07Universal-Omega opened pull request 03#152: remove redundant function from assignImportedEdits.php - 13https://git.io/JkSMZ [06:42:33] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.47, 17.67, 13.45 [06:44:35] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.74, 16.06, 13.36 [06:49:53] !log reception@jobrunner1:~$ sudo -u www-data php /srv/mediawiki/w/maintenance/importImages.php --wiki=minelandwiki --search-recursively --skip-dupes /home/reception/minelandimg [06:49:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:57:12] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/JkSb2 [06:57:13] [02miraheze/ssl] 07Reception123 0331c242e - add evenair.tk cert [07:05:15] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkSNP [07:05:16] [02miraheze/services] 07MirahezeSSLBot 03e41baf3 - BOT: Updating services config for wikis [08:06:04] !log reception@jobrunner1:/srv/mediawiki/w/maintenance$ sudo -u www-data php recountCategories.php --mode=pages --wiki mylittleponywiki [08:06:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:20:09] !log running wikibackups on jobrunner1 (bash ./wikibackups.sh /home/reception/backupwikis.dblist /srv/mediawiki/w/maintenance/dumpBackup.php) [08:20:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:42:50] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.46, 18.19, 14.16 [08:44:48] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 10.95, 15.41, 13.65 [08:52:32] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_35 [+0/-0/±2] 13https://git.io/Jk9mH [08:52:34] [02miraheze/mediawiki] 07paladox 0313f99ef - Update LinkTitles to develop [08:53:42] [02MirahezeMagic] 07paladox closed pull request 03#152: remove redundant function from assignImportedEdits.php - 13https://git.io/JkSMZ [08:53:43] [02miraheze/MirahezeMagic] 07paladox pushed 032 commits to 03master [+0/-0/±2] 13https://git.io/Jk9YU [08:53:45] [02miraheze/MirahezeMagic] 07paladox 03c096256 - Merge pull request #152 from miraheze/Universal-Omega-patch-1 remove redundant function from assignImportedEdits.php [08:53:46] [02MirahezeMagic] 07paladox deleted branch 03Universal-Omega-patch-1 - 13https://git.io/fQRGX [08:53:48] [02miraheze/MirahezeMagic] 07paladox deleted branch 03Universal-Omega-patch-1 [08:54:35] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 2 minutes ago with 3 failures. Failed resources (up to 3 shown): Service[gdnsd],Exec[ufw-allow-udp-from-any-to-any-port-53],Exec[ufw-allow-tcp-from-any-to-any-port-53] [09:00:18] PROBLEM - mw5 Puppet on mw5 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [09:00:32] PROBLEM - mw6 Puppet on mw6 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [09:01:12] PROBLEM - jobrunner1 Puppet on jobrunner1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [09:01:26] PROBLEM - mw7 Puppet on mw7 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [09:01:30] PROBLEM - mw4 Puppet on mw4 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [09:02:46] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 41 seconds ago with 0 failures [09:03:09] RECOVERY - jobrunner1 Puppet on jobrunner1 is OK: OK: Puppet is currently enabled, last run 6 seconds ago with 0 failures [09:03:27] RECOVERY - mw7 Puppet on mw7 is OK: OK: Puppet is currently enabled, last run 36 seconds ago with 0 failures [09:03:27] RECOVERY - mw4 Puppet on mw4 is OK: OK: Puppet is currently enabled, last run 24 seconds ago with 0 failures [09:04:16] RECOVERY - mw5 Puppet on mw5 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:04:32] RECOVERY - mw6 Puppet on mw6 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:25:14] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jk9Zc [09:25:16] [02miraheze/services] 07MirahezeSSLBot 03234c566 - BOT: Updating services config for wikis [10:05:14] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jk9lh [10:05:16] [02miraheze/services] 07MirahezeSSLBot 03f694b53 - BOT: Updating services config for wikis [10:16:32] PROBLEM - wiki.abyogsms.org.in - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.abyogsms.org.in All nameservers failed to answer the query. [10:20:28] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jk9BL [10:20:30] [02miraheze/services] 07MirahezeSSLBot 03c253edf - BOT: Updating services config for wikis [10:24:33] PROBLEM - wiki.abyogsms.org.in - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:25:14] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jk9By [10:25:16] [02miraheze/services] 07MirahezeSSLBot 03b5b1de0 - BOT: Updating services config for wikis [10:51:00] RECOVERY - wiki.abyogsms.org.in - reverse DNS on sslhost is OK: rDNS OK - wiki.abyogsms.org.in reverse DNS resolves to cp6.miraheze.org [10:53:02] RECOVERY - wiki.abyogsms.org.in - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.abyogsms.org.in' will expire on Sun 17 Jan 2021 03:52:35 GMT +0000. [11:44:35] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[ufw-allow-udp-from-any-to-any-port-53],Exec[ufw-allow-tcp-from-any-to-any-port-53] [11:52:35] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [12:11:44] PROBLEM - graylog1 Current Load on graylog1 is CRITICAL: CRITICAL - load average: 4.68, 3.73, 2.72 [12:13:44] PROBLEM - graylog1 Current Load on graylog1 is WARNING: WARNING - load average: 3.58, 3.69, 2.84 [12:14:09] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 9.80, 6.85, 5.01 [12:15:44] RECOVERY - graylog1 Current Load on graylog1 is OK: OK - load average: 1.63, 3.06, 2.72 [12:16:07] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 4.34, 5.80, 4.85 [13:33:07] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.31, 19.09, 14.99 [13:35:05] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.29, 17.49, 14.90 [13:42:47] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 9.16, 6.36, 5.20 [13:44:41] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 4.88, 5.79, 5.14 [13:44:45] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/JkHv7 [13:44:46] [02miraheze/ssl] 07Reception123 03615380c - add planetnomads.speedbomb.de cert [14:13:35] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 8.56, 7.99, 6.40 [14:15:14] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkHIt [14:15:16] [02miraheze/services] 07MirahezeSSLBot 037fedc17 - BOT: Updating services config for wikis [14:15:30] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 4.67, 6.58, 6.06 [14:18:28] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 41.02, 25.46, 18.75 [14:20:26] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.27, 22.01, 18.22 [14:24:28] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.48, 19.58, 18.10 [14:33:24] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 6.81, 5.81, 5.32 [14:35:23] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 4.45, 5.14, 5.12 [14:40:27] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 46.10, 29.42, 22.10 [14:41:57] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 16.55, 13.54, 7.41 [14:44:26] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 10.52, 21.33, 20.70 [14:45:56] PROBLEM - cp7 Current Load on cp7 is WARNING: WARNING - load average: 2.34, 7.05, 6.09 [14:46:28] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 9.30, 17.26, 19.28 [14:47:56] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 2.36, 5.53, 5.65 [15:04:19] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.05, 22.79, 20.73 [15:06:18] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 12.57, 19.14, 19.65 [15:12:51] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 6.88, 5.80, 5.20 [15:13:12] !log reception@jobrunner1:~$ sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --update --wiki gyaanipediawiki [15:13:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:14:50] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 4.57, 5.35, 5.12 [15:20:25] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkHCE [15:20:27] [02miraheze/services] 07MirahezeSSLBot 03d7bb096 - BOT: Updating services config for wikis [15:22:25] PROBLEM - cloud1 Current Load on cloud1 is CRITICAL: CRITICAL - load average: 24.39, 19.73, 17.48 [15:24:26] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 12.16, 16.96, 16.76 [15:32:50] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 8.44, 6.62, 5.58 [15:34:20] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 7.03, 7.00, 5.87 [15:34:50] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 5.79, 6.25, 5.58 [15:36:16] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 6.08, 6.44, 5.78 [15:44:05] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 8.24, 7.90, 6.57 [15:46:01] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 4.01, 6.42, 6.19 [16:02:50] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 9.99, 6.84, 5.67 [16:03:34] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 8.39, 6.88, 5.90 [16:04:50] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 4.84, 5.97, 5.50 [16:05:32] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 4.48, 5.93, 5.67 [16:13:29] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 9.30, 7.30, 6.27 [16:14:19] PROBLEM - cloud1 Current Load on cloud1 is WARNING: WARNING - load average: 21.29, 21.41, 18.66 [16:14:49] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 6.98, 7.47, 6.32 [16:15:25] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 6.20, 6.59, 6.12 [16:16:50] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 4.70, 6.60, 6.15 [16:18:18] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 12.45, 17.76, 17.87 [16:22:50] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 12.77, 8.57, 6.91 [16:23:18] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 8.43, 7.83, 6.83 [16:24:50] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 7.24, 7.70, 6.78 [16:25:15] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 5.61, 7.01, 6.66 [16:27:11] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 3.95, 6.09, 6.37 [16:28:50] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 3.95, 5.85, 6.26 [16:42:51] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 9.14, 7.29, 6.58 [16:43:16] PROBLEM - cloud1 Current Load on cloud1 is WARNING: WARNING - load average: 22.04, 18.49, 17.18 [16:44:34] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[openssh-client] [16:44:50] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 5.87, 6.91, 6.55 [16:45:17] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 14.49, 17.76, 17.16 [16:46:51] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 3.11, 5.59, 6.11 [16:52:30] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 9.00, 6.56, 6.19 [16:52:35] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 16 seconds ago with 0 failures [16:54:27] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 6.32, 6.45, 6.20 [16:59:43] [02miraheze/WikiDiscover] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkHoT [16:59:44] [02miraheze/WikiDiscover] 07translatewiki 03a4ce072 - Localisation updates from https://translatewiki.net. [16:59:45] [ Main page - translatewiki.net ] - translatewiki.net [16:59:46] [02miraheze/MirahezeMagic] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkHoI [16:59:47] [02miraheze/MirahezeMagic] 07translatewiki 03f16d7b5 - Localisation updates from https://translatewiki.net. [16:59:48] [ Main page - translatewiki.net ] - translatewiki.net [16:59:49] [02miraheze/DataDump] 07translatewiki pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JkHoL [16:59:50] [02miraheze/DataDump] 07translatewiki 032e3f818 - Localisation updates from https://translatewiki.net. [16:59:51] [ Main page - translatewiki.net ] - translatewiki.net [17:04:19] PROBLEM - graylog1 Current Load on graylog1 is CRITICAL: CRITICAL - load average: 5.21, 4.45, 3.26 [17:10:08] PROBLEM - graylog1 Current Load on graylog1 is WARNING: WARNING - load average: 3.43, 3.80, 3.34 [17:14:02] RECOVERY - graylog1 Current Load on graylog1 is OK: OK - load average: 2.17, 3.09, 3.17 [17:14:19] PROBLEM - cloud1 Current Load on cloud1 is WARNING: WARNING - load average: 19.20, 20.79, 18.89 [17:14:21] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 9.20, 8.49, 6.98 [17:15:50] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 7.85, 7.48, 6.72 [17:16:20] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 19.00, 19.86, 18.76 [17:16:20] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 6.50, 7.62, 6.84 [17:17:45] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 5.27, 6.79, 6.57 [17:18:16] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 4.94, 6.61, 6.55 [17:23:34] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 12.13, 8.95, 7.39 [17:24:19] PROBLEM - cloud1 Current Load on cloud1 is CRITICAL: CRITICAL - load average: 24.53, 22.47, 19.82 [17:25:29] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 6.28, 7.87, 7.18 [17:25:45] [02miraheze/MirahezeMagic] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkHiy [17:25:46] [02miraheze/MirahezeMagic] 07Universal-Omega 0353fe1f5 - fix [17:25:48] [02MirahezeMagic] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/fQRGX [17:25:53] [02MirahezeMagic] 07Universal-Omega opened pull request 03#153: fix assignImportedEdits.php - 13https://git.io/JkHiS [17:26:20] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 13.76, 19.22, 18.96 [17:27:22] Hello chrississack[m]! If you have any questions, feel free to ask and someone should answer soon. [17:27:23] Hello my favorite Miraheze staff members! Today is Thanksgiving day and that means that we get to s!ng along! i really hope you join me cuz it will be fun! [17:27:25] The world was on fire and no one could save me but you [17:27:28] It's strange what desire will make foolish people do [17:27:30] I never dreamed that I'd meet somebody like you [17:27:35] And I never dreamed that I'd lose somebody like you [17:27:46] No, I don't wanna fall in love (this world is only gonna break your heart) [17:27:47] No, I don't wanna fall in love (this world is only gonna break your heart) [17:27:50] With you [17:27:55] What a wicked game you play, to make me feel this way [17:27:58] What a wicked thing to do, to let me dream of you [17:28:05] I dream of you, Miraheze [17:28:06] What a wicked thing to say, you never felt this way [17:28:10] What a wicked thing to do, to make me dream of you [17:28:13] Hello my favorite Miraheze staff members! Today is Thanksgiving day and that means that we get to s!ng along! i really hope you join me cuz it will be fun! [17:28:17] Hello my favorite Miraheze staff members! Today is Thanksgiving day and that means that we get to s!ng along! i really hope you join me cuz it will be fun! [17:28:21] And I don't wanna fall in love (this world is only gonna break your heart) [17:28:23] No, I don't wanna fall in love (this world is only gonna break your heart) [17:28:27] With you [17:28:30] @System Administrators ^ [17:28:30] The world was on fire and no one could save me but you [17:28:33] It's strange what desire will make foolish people do [17:28:36] I never dreamed that I'd love somebody like you [17:28:40] Hi @RhinosF1 s!ng with me! [17:28:47] You know you want to stop being sh y! [17:28:52] And I never dreamed that I'd lose somebody like you [17:29:20] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 5.05, 6.22, 6.64 [17:29:34] I'd rather my sore throat and headache didn't get worse [17:29:46] [02mediawiki] 07dependabot[bot] opened pull request 03#579: Bump extensions/Comments from `0c76716` to `a309d61` - 13https://git.io/JkHPG [17:29:47] [02mediawiki] 07dependabot[bot] opened pull request 03#580: Bump extensions/ProtectSite from `e60eb81` to `a363bfb` - 13https://git.io/JkHPZ [17:29:49] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/ProtectSite-a363bfb [+0/-0/±1] 13https://git.io/JkHPn [17:29:50] [02miraheze/mediawiki] 07dependabot[bot] 0385acbc5 - Bump extensions/ProtectSite from `e60eb81` to `a363bfb` Bumps [extensions/ProtectSite](https://github.com/wikimedia/mediawiki-extensions-ProtectSite) from `e60eb81` to `a363bfb`. - [Release notes](https://github.com/wikimedia/mediawiki-extensions-ProtectSite/releases) - [Commits](https://github.com/wikimedia/mediawiki-extensions- [17:29:50] ProtectSite/compare/e60eb81bc85ec72049288808a2b1a6afa2f3f71f...a363bfbe3b3b19283f38a46f95b910ef67a7ec60) Signed-off-by: dependabot[bot] [17:29:51] [ GitHub - wikimedia/mediawiki-extensions-ProtectSite: Github mirror of MediaWiki extension ProtectSite - our actual code is hosted with Gerrit (please see https://www.mediawiki.org/wiki/Developer_acces ] - github.com [17:29:51] [ Releases · wikimedia/mediawiki-extensions-ProtectSite · GitHub ] - github.com [17:29:52] [ Page not found · GitHub · GitHub ] - github.com [17:29:52] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/PageForms-c30f003 [+0/-0/±1] 13https://git.io/JkHPc [17:29:53] [02miraheze/mediawiki] 07dependabot[bot] 0371ef9b3 - Bump extensions/PageForms from `8c769b8` to `c30f003` Bumps [extensions/PageForms](https://github.com/wikimedia/mediawiki-extensions-PageForms) from `8c769b8` to `c30f003`. - [Release notes](https://github.com/wikimedia/mediawiki-extensions-PageForms/releases) - [Commits](https://github.com/wikimedia/mediawiki-extensions- [17:29:53] PageForms/compare/8c769b85b8c1ac7dd9873f4e9e470931077d0317...c30f003ab3de3670b81c73f5a8a28cc21984bcc0) Signed-off-by: dependabot[bot] [17:29:54] [ GitHub - wikimedia/mediawiki-extensions-PageForms: Github mirror of MediaWiki extension PageForms - our actual code is hosted with Gerrit (please see https://www.mediawiki.org/wiki/Developer_access fo ] - github.com [17:29:55] [02mediawiki] 07dependabot[bot] opened pull request 03#581: Bump extensions/PageForms from `8c769b8` to `c30f003` - 13https://git.io/JkHPC [17:29:55] [ Releases · wikimedia/mediawiki-extensions-PageForms · GitHub ] - github.com [17:29:56] [ Page not found · GitHub · GitHub ] - github.com [17:29:56] [02mediawiki] 07dependabot[bot] labeled pull request 03#579: Bump extensions/Comments from `0c76716` to `a309d61` - 13https://git.io/JkHPG [17:29:58] [02mediawiki] 07dependabot[bot] labeled pull request 03#580: Bump extensions/ProtectSite from `e60eb81` to `a363bfb` - 13https://git.io/JkHPZ [17:29:59] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/Comments-a309d61 [+0/-0/±1] 13https://git.io/JkHPW [17:30:01] [02miraheze/mediawiki] 07dependabot[bot] 034493bf9 - Bump extensions/Comments from `0c76716` to `a309d61` Bumps [extensions/Comments](https://github.com/wikimedia/mediawiki-extensions-Comments) from `0c76716` to `a309d61`. - [Release notes](https://github.com/wikimedia/mediawiki-extensions-Comments/releases) - [Commits](https://github.com/wikimedia/mediawiki-extensions- [17:30:01] Comments/compare/0c767167bbc68a675c4cad8a144335b1635207aa...a309d6172935a455a22e33c9326fbb6359f392db) Signed-off-by: dependabot[bot] [17:30:02] [ GitHub - wikimedia/mediawiki-extensions-Comments: Github mirror of MediaWiki extension Comments - our actual code is hosted with Gerrit (please see https://www.mediawiki.org/wiki/Developer_access for ] - github.com [17:30:02] [ Releases · wikimedia/mediawiki-extensions-Comments · GitHub ] - github.com [17:30:02] [ Page not found · GitHub · GitHub ] - github.com [17:30:02] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/MirahezeMagic-f16d7b5 [+0/-0/±1] 13https://git.io/JkHPl [17:30:04] [02miraheze/mediawiki] 07dependabot[bot] 036d7abcf - Bump extensions/MirahezeMagic from `d2eada6` to `f16d7b5` Bumps [extensions/MirahezeMagic](https://github.com/miraheze/MirahezeMagic) from `d2eada6` to `f16d7b5`. - [Release notes](https://github.com/miraheze/MirahezeMagic/releases) - [17:30:04] [Commits](https://github.com/miraheze/MirahezeMagic/compare/d2eada64f61c2940e3bfd05883c4128cacec1532...f16d7b5064cb9d5f182886cc1032c8079a3e21a6) Signed-off-by: dependabot[bot] [17:30:05] [ GitHub - miraheze/MirahezeMagic: Miraheze specific messages, scripts and hooks for MediaWiki. ] - github.com [17:30:05] [02mediawiki] 07dependabot[bot] labeled pull request 03#581: Bump extensions/PageForms from `8c769b8` to `c30f003` - 13https://git.io/JkHPC [17:30:05] [ Releases · miraheze/MirahezeMagic · GitHub ] - github.com [17:30:07] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/Comments-a309d61 - 13https://git.io/vbL5b [17:30:08] [ Comparing d2eada64f61c2940e3bfd05883c4128cacec1532...f16d7b5064cb9d5f182886cc1032c8079a3e21a6 · miraheze/MirahezeMagic · GitHub ] - github.com [17:30:08] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/PageForms-c30f003 - 13https://git.io/vbL5b [17:30:10] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/ProtectSite-a363bfb - 13https://git.io/vbL5b [17:30:11] [02mediawiki] 07dependabot[bot] opened pull request 03#582: Bump extensions/MirahezeMagic from `d2eada6` to `f16d7b5` - 13https://git.io/JkHP4 [17:30:13] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/MirahezeMagic-f16d7b5 - 13https://git.io/vbL5b [17:30:14] [02mediawiki] 07dependabot[bot] labeled pull request 03#582: Bump extensions/MirahezeMagic from `d2eada6` to `f16d7b5` - 13https://git.io/JkHP4 [17:30:16] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/skins/Medik-68ddf66 [+0/-0/±1] 13https://git.io/JkHPB [17:30:17] [02miraheze/mediawiki] 07dependabot[bot] 03dbdab73 - Bump skins/Medik from `9ca92c1` to `68ddf66` Bumps [skins/Medik](https://bitbucket.org/wikiskripta/medik) from `9ca92c1` to `68ddf66`. - [Commits](https://bitbucket.org/wikiskripta/medik/branches/compare/68ddf6699cd2a97b8bb1aa69a0031bfb4a1bf3b6..9ca92c1d27b658bd08faa3c8b26fca659c1a57f5) Signed-off-by: dependabot[bot] [17:30:18] [ Bitbucket ] - bitbucket.org [17:30:19] [ wikiskripta / Medik / Compare — Bitbucket ] - bitbucket.org [17:30:19] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/skins/Gamepress-f0d32b1 [+0/-0/±1] 13https://git.io/JkHPR [17:30:20] [02miraheze/mediawiki] 07dependabot[bot] 030611707 - Bump skins/Gamepress from `30d980d` to `f0d32b1` Bumps [skins/Gamepress](https://github.com/wikimedia/mediawiki-skins-Gamepress) from `30d980d` to `f0d32b1`. - [Release notes](https://github.com/wikimedia/mediawiki-skins-Gamepress/releases) - [Commits](https://github.com/wikimedia/mediawiki-skins- [17:30:20] Gamepress/compare/30d980d2d7a5150a4662cbbd3dad69c0b1c5bc26...f0d32b154e0d23ca53714cf7902e75071d43f643) Signed-off-by: dependabot[bot] [17:30:21] [ GitHub - wikimedia/mediawiki-skins-Gamepress: Github mirror of "mediawiki/skins/Gamepress" - our actual code is hosted with Gerrit (please see https://www.mediawiki.org/wiki/Developer_access for contr ] - github.com [17:30:21] [ Releases · wikimedia/mediawiki-skins-Gamepress · GitHub ] - github.com [17:30:22] [02mediawiki] 07dependabot[bot] opened pull request 03#583: Bump skins/Medik from `9ca92c1` to `68ddf66` - 13https://git.io/JkHP0 [17:30:22] [ Page not found · GitHub · GitHub ] - github.com [17:30:23] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/skins/Medik-68ddf66 - 13https://git.io/vbL5b [17:30:25] [02mediawiki] 07dependabot[bot] opened pull request 03#584: Bump skins/Gamepress from `30d980d` to `f0d32b1` - 13https://git.io/JkHPE [17:30:26] [02miraheze/MirahezeMagic] 07paladox pushed 032 commits to 03master [+0/-0/±2] 13https://git.io/JkHPu [17:30:28] [02miraheze/MirahezeMagic] 07paladox 0368b3a72 - Merge pull request #153 from miraheze/Universal-Omega-patch-1 fix assignImportedEdits.php [17:30:29] [02mediawiki] 07dependabot[bot] labeled pull request 03#584: Bump skins/Gamepress from `30d980d` to `f0d32b1` - 13https://git.io/JkHPE [17:30:31] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/skins/Gamepress-f0d32b1 - 13https://git.io/vbL5b [17:30:32] [02mediawiki] 07dependabot[bot] labeled pull request 03#583: Bump skins/Medik from `9ca92c1` to `68ddf66` - 13https://git.io/JkHP0 [17:30:34] [02MirahezeMagic] 07paladox closed pull request 03#153: fix assignImportedEdits.php - 13https://git.io/JkHiS [17:30:35] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/MsCatSelect-7330af5 [+0/-0/±1] 13https://git.io/JkHPz [17:30:37] [02miraheze/mediawiki] 07dependabot[bot] 03d1ac4c8 - Bump extensions/MsCatSelect from `7c4e055` to `7330af5` Bumps [extensions/MsCatSelect](https://github.com/wikimedia/mediawiki-extensions-MsCatSelect) from `7c4e055` to `7330af5`. - [Release notes](https://github.com/wikimedia/mediawiki-extensions-MsCatSelect/releases) - [Commits](https://github.com/wikimedia/mediawiki-extensions- [17:30:37] MsCatSelect/compare/7c4e0554be4236e13c2961aa11998836a9645c26...7330af57f7319e45e25f17907da82f2f69a9afd9) Signed-off-by: dependabot[bot] [17:30:37] [ GitHub - wikimedia/mediawiki-extensions-MsCatSelect: Github mirror of MediaWiki extension MsCatSelect - our actual code is hosted with Gerrit (please see https://www.mediawiki.org/wiki/Developer_acce ] - github.com [17:30:38] [ Releases · wikimedia/mediawiki-extensions-MsCatSelect · GitHub ] - github.com [17:30:38] [ Page not found · GitHub · GitHub ] - github.com [17:30:38] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/MsCatSelect-7330af5 - 13https://git.io/vbL5b [17:30:40] [02mediawiki] 07dependabot[bot] opened pull request 03#585: Bump extensions/MsCatSelect from `7c4e055` to `7330af5` - 13https://git.io/JkHPg [17:30:41] [02MirahezeMagic] 07paladox deleted branch 03Universal-Omega-patch-1 - 13https://git.io/fQRGX [17:30:43] [02miraheze/MirahezeMagic] 07paladox deleted branch 03Universal-Omega-patch-1 [17:30:44] [02mediawiki] 07dependabot[bot] labeled pull request 03#585: Bump extensions/MsCatSelect from `7c4e055` to `7330af5` - 13https://git.io/JkHPg [17:30:46] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/DataDump-2e3f818 [+0/-0/±1] 13https://git.io/JkHP2 [17:30:47] [02miraheze/mediawiki] 07dependabot[bot] 03c12691c - Bump extensions/DataDump from `328250f` to `2e3f818` Bumps [extensions/DataDump](https://github.com/miraheze/DataDump) from `328250f` to `2e3f818`. - [Release notes](https://github.com/miraheze/DataDump/releases) - [Commits](https://github.com/miraheze/DataDump/compare/328250f2459470270e17f37f88467f55305f0c06...2e3f818bf35cef5e5577c3712e7afde9339bad2c) [17:30:47] Signed-off-by: dependabot[bot] [17:30:48] [ GitHub - miraheze/DataDump: An extension that provides dumps of wikis ] - github.com [17:30:49] [02mediawiki] 07dependabot[bot] opened pull request 03#586: Bump extensions/DataDump from `328250f` to `2e3f818` - 13https://git.io/JkHPa [17:30:49] [ Releases · miraheze/DataDump · GitHub ] - github.com [17:30:50] [ Comparing 328250f2459470270e17f37f88467f55305f0c06...2e3f818bf35cef5e5577c3712e7afde9339bad2c · miraheze/DataDump · GitHub ] - github.com [17:30:50] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/DataDump-2e3f818 - 13https://git.io/vbL5b [17:30:52] [02mediawiki] 07dependabot[bot] labeled pull request 03#586: Bump extensions/DataDump from `328250f` to `2e3f818` - 13https://git.io/JkHPa [17:30:53] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/SocialProfile-383f255 [+0/-0/±1] 13https://git.io/JkHPV [17:32:41] RhinosF1: $~a and +z set [17:35:23] Ty [17:36:27] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.06, 19.02, 16.57 [17:38:26] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 19.55, 19.55, 17.08 [17:40:46] PROBLEM - mw7 Puppet on mw7 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [17:41:15] PROBLEM - mw6 Puppet on mw6 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [17:41:39] PROBLEM - mw5 Puppet on mw5 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [17:41:40] PROBLEM - mw4 Puppet on mw4 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [17:42:18] PROBLEM - jobrunner1 Puppet on jobrunner1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [17:43:38] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 6.98, 7.34, 6.81 [17:44:02] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 7.60, 7.71, 7.12 [17:44:41] PROBLEM - www.wikiescola.com.br - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.wikiescola.com.br could not be found [17:44:41] PROBLEM - www.opendatascot.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.opendatascot.org could not be found [17:46:42] RECOVERY - mw7 Puppet on mw7 is OK: OK: Puppet is currently enabled, last run 24 seconds ago with 0 failures [17:47:12] RECOVERY - mw6 Puppet on mw6 is OK: OK: Puppet is currently enabled, last run 26 seconds ago with 0 failures [17:47:30] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 6.08, 6.69, 6.67 [17:47:39] RECOVERY - mw5 Puppet on mw5 is OK: OK: Puppet is currently enabled, last run 20 seconds ago with 0 failures [17:47:53] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 9.60, 7.88, 7.25 [17:48:16] RECOVERY - jobrunner1 Puppet on jobrunner1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [17:49:44] RECOVERY - mw4 Puppet on mw4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [17:49:48] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 6.70, 7.38, 7.14 [17:51:25] RECOVERY - www.wikiescola.com.br - reverse DNS on sslhost is OK: rDNS OK - www.wikiescola.com.br reverse DNS resolves to cp6.miraheze.org [17:51:26] RECOVERY - www.opendatascot.org - reverse DNS on sslhost is OK: rDNS OK - www.opendatascot.org reverse DNS resolves to cp7.miraheze.org [17:51:44] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 9.42, 8.03, 7.39 [17:53:24] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 9.45, 7.75, 7.02 [17:54:36] PROBLEM - cp6 Current Load on cp6 is CRITICAL: CRITICAL - load average: 18.30, 13.15, 6.17 [17:55:15] PROBLEM - cloud1 Current Load on cloud1 is WARNING: WARNING - load average: 21.29, 23.42, 20.33 [17:55:22] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 7.78, 7.98, 7.21 [17:55:34] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 5.78, 7.42, 7.33 [17:58:36] RECOVERY - cp6 Current Load on cp6 is OK: OK - load average: 1.72, 6.55, 5.03 [17:59:16] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 15.67, 19.86, 19.60 [18:01:14] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 8.88, 7.55, 7.22 [18:01:21] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 9.52, 7.90, 7.53 [18:05:10] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 7.23, 7.85, 7.45 [18:07:07] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 6.23, 8.00, 7.88 [18:12:53] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 11.77, 8.41, 7.88 [18:12:54] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 13.32, 9.16, 7.84 [18:16:50] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 5.78, 7.95, 7.91 [18:16:50] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 4.54, 7.35, 7.51 [18:22:50] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 11.08, 8.20, 7.83 [18:22:52] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 13.10, 9.11, 7.99 [18:28:49] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 6.37, 7.51, 7.77 [18:30:50] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 9.46, 8.16, 7.96 [18:34:11] PROBLEM - cloud1 Current Load on cloud1 is CRITICAL: CRITICAL - load average: 25.28, 22.32, 19.66 [18:34:19] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb [18:34:44] PROBLEM - dcmultiverse.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for dcmultiverse.miraheze.org could not be found [18:35:53] PROBLEM - www.mcpk.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.mcpk.wiki could not be found [18:36:04] PROBLEM - froggy.info - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for froggy.info could not be found [18:36:06] PROBLEM - savage-wiki.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for savage-wiki.com could not be found [18:36:07] PROBLEM - es.nocyclo.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for es.nocyclo.tk could not be found [18:36:10] PROBLEM - cloud1 Current Load on cloud1 is WARNING: WARNING - load average: 19.78, 21.22, 19.57 [18:36:18] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [18:38:09] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 13.58, 18.49, 18.77 [18:40:51] [[w:WP:GRAVEDANCING]] for @Reception123 to link to in a future message [18:41:08] ergh [18:41:14] [[w:WP:GRAVEDANCING]] [18:41:15] https://en.wikipedia.org/wiki/WP:GRAVEDANCING [18:41:15] [WIKIPEDIA] WP:GRAVEDANCING | "Some editors are blocked, sometimes indefinitely, due to their own disruptive behaviour or other reasons. Others may retire from the project, either explicitly by declaring themselves retired, or implicitly by simply discontinuing their editing. These editors may have edited Wikipedia only for a short..." [18:41:24] RECOVERY - dcmultiverse.miraheze.org - reverse DNS on sslhost is OK: rDNS OK - dcmultiverse.miraheze.org reverse DNS resolves to cp7.miraheze.org [18:41:25] @Reception123 ^ [18:42:14] thanks :) [18:42:25] np 🙂 [18:42:43] RECOVERY - www.mcpk.wiki - reverse DNS on sslhost is OK: rDNS OK - www.mcpk.wiki reverse DNS resolves to cp7.miraheze.org [18:42:45] RECOVERY - froggy.info - reverse DNS on sslhost is OK: rDNS OK - froggy.info reverse DNS resolves to cp6.miraheze.org [18:42:47] RECOVERY - savage-wiki.com - reverse DNS on sslhost is OK: rDNS OK - savage-wiki.com reverse DNS resolves to cp6.miraheze.org [18:42:49] RECOVERY - es.nocyclo.tk - reverse DNS on sslhost is OK: rDNS OK - es.nocyclo.tk reverse DNS resolves to cp7.miraheze.org [18:48:50] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 4.85, 6.82, 7.87 [18:52:47] miraheze/MirahezeMagic/Universal-Omega-patch-1/53fe1f5 - Universal Omega The build has errored. https://travis-ci.org/miraheze/MirahezeMagic/builds/746122234 [18:52:51] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 12.19, 8.83, 8.35 [18:54:48] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 6 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb, 51.89.160.142/cpweb, 2001:41d0:800:105a::10/cpweb, 2607:5300:205:200::2ac4/cpweb [18:56:42] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [18:57:12] PROBLEM - bharatwiki.online - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for bharatwiki.online could not be found [18:57:14] PROBLEM - wiki.ucmg.ml - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.ucmg.ml could not be found [19:04:00] RECOVERY - bharatwiki.online - reverse DNS on sslhost is OK: rDNS OK - bharatwiki.online reverse DNS resolves to cp7.miraheze.org [19:04:05] RECOVERY - wiki.ucmg.ml - reverse DNS on sslhost is OK: rDNS OK - wiki.ucmg.ml reverse DNS resolves to cp6.miraheze.org [19:06:50] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 4.45, 6.72, 7.69 [19:08:49] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 5.56, 6.72, 7.69 [19:13:12] PROBLEM - cloud1 Current Load on cloud1 is CRITICAL: CRITICAL - load average: 24.76, 21.76, 19.72 [19:14:51] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 8.25, 8.24, 7.90 [19:15:11] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 13.24, 18.59, 18.82 [19:16:51] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 6.72, 7.62, 7.71 [19:17:58] PROBLEM - cp7 Current Load on cp7 is WARNING: WARNING - load average: 7.95, 4.68, 2.60 [19:19:10] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.29, 18.92, 16.74 [19:19:56] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 25.28, 10.95, 5.04 [19:20:49] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 6.33, 5.84, 6.73 [19:21:06] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 28.77, 23.28, 18.65 [19:22:11] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 3 datacenters are down: 2400:6180:0:d0::403:f001/cpweb, 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb [19:22:51] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 9.04, 7.56, 7.49 [19:23:02] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.66, 22.50, 18.94 [19:23:06] PROBLEM - vortekspedia.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for vortekspedia.tk could not be found [19:23:58] PROBLEM - cp7 Current Load on cp7 is WARNING: WARNING - load average: 4.03, 7.24, 4.90 [19:24:10] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [19:24:50] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 6.90, 6.77, 6.95 [19:24:51] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 6.00, 7.05, 7.32 [19:25:58] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 1.36, 5.16, 4.41 [19:26:54] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.04, 19.49, 18.60 [19:28:49] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 4.15, 5.77, 6.53 [19:28:50] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 4.35, 5.83, 6.78 [19:29:54] RECOVERY - vortekspedia.tk - reverse DNS on sslhost is OK: rDNS OK - vortekspedia.tk reverse DNS resolves to cp6.miraheze.org [19:32:50] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 10.97, 7.62, 7.01 [19:34:49] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 6.31, 6.80, 6.77 [19:43:35] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 9.43, 7.52, 6.86 [19:43:42] !log depool mw[4567] and repool [19:43:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:45:33] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 5.22, 6.73, 6.66 [19:46:14] PROBLEM - jobrunner1 Puppet on jobrunner1 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 13 minutes ago with 0 failures [19:46:15] !log rebuild lc on mw[4567] [19:46:44] PROBLEM - mw7 Puppet on mw7 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 13 minutes ago with 0 failures [19:47:13] PROBLEM - mw6 Puppet on mw6 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 14 minutes ago with 0 failures [19:47:28] PROBLEM - jobrunner2 Puppet on jobrunner2 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 1 minute ago with 0 failures [19:47:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:47:40] PROBLEM - mw5 Puppet on mw5 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 14 minutes ago with 0 failures [19:48:08] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 7 datacenters are down: 128.199.139.216/cpweb, 51.77.107.210/cpweb, 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [19:48:29] PROBLEM - mw4 MediaWiki Rendering on mw4 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 16474 bytes in 0.116 second response time [19:49:16] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 3 datacenters are down: 2001:41d0:800:1056::2/cpweb, 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [19:49:30] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 8.01, 7.37, 6.93 [19:49:43] PROBLEM - mw6 Current Load on mw6 is CRITICAL: CRITICAL - load average: 9.86, 7.47, 5.47 [19:50:19] PROBLEM - wiki.s23.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.s23.org could not be found [19:50:24] PROBLEM - vitriol.spoon.army - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for vitriol.spoon.army could not be found [19:50:24] PROBLEM - wiki.kourouklides.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.kourouklides.com could not be found [19:50:28] RECOVERY - mw4 MediaWiki Rendering on mw4 is OK: HTTP OK: HTTP/1.1 200 OK - 19605 bytes in 0.206 second response time [19:50:33] PROBLEM - wiki.graalmilitary.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.graalmilitary.com could not be found [19:50:33] PROBLEM - wiki.candela.digital - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.candela.digital could not be found [19:52:00] PROBLEM - mw4 Puppet on mw4 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [19:54:00] RECOVERY - mw4 Puppet on mw4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [19:54:06] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [19:54:17] PROBLEM - theatlas.pw - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for theatlas.pw could not be found [19:54:18] PROBLEM - wiki.xysspon.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.xysspon.com could not be found [19:54:22] PROBLEM - meta.orain.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for meta.orain.org could not be found [19:54:27] PROBLEM - wiki.lefrenchmelee.fr - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.lefrenchmelee.fr could not be found [19:55:01] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [19:55:20] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 2.54, 6.63, 6.99 [19:55:40] PROBLEM - mw5 Puppet on mw5 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [19:56:49] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 9.02, 6.45, 5.68 [19:57:11] RECOVERY - wiki.kourouklides.com - reverse DNS on sslhost is OK: rDNS OK - wiki.kourouklides.com reverse DNS resolves to cp6.miraheze.org [19:57:20] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 1.43, 4.85, 6.29 [19:57:20] RECOVERY - wiki.s23.org - reverse DNS on sslhost is OK: rDNS OK - wiki.s23.org reverse DNS resolves to cp7.miraheze.org [19:57:25] RECOVERY - vitriol.spoon.army - reverse DNS on sslhost is OK: rDNS OK - vitriol.spoon.army reverse DNS resolves to cp6.miraheze.org [19:57:30] RECOVERY - wiki.candela.digital - reverse DNS on sslhost is OK: rDNS OK - wiki.candela.digital reverse DNS resolves to cp6.miraheze.org [19:57:30] RECOVERY - wiki.graalmilitary.com - reverse DNS on sslhost is OK: rDNS OK - wiki.graalmilitary.com reverse DNS resolves to cp6.miraheze.org [19:58:25] PROBLEM - test2 MediaWiki Rendering on test2 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 16477 bytes in 0.184 second response time [20:01:06] RECOVERY - wiki.lefrenchmelee.fr - reverse DNS on sslhost is OK: rDNS OK - wiki.lefrenchmelee.fr reverse DNS resolves to cp6.miraheze.org [20:01:09] RECOVERY - theatlas.pw - reverse DNS on sslhost is OK: rDNS OK - theatlas.pw reverse DNS resolves to cp6.miraheze.org [20:01:10] RECOVERY - wiki.xysspon.com - reverse DNS on sslhost is OK: rDNS OK - wiki.xysspon.com reverse DNS resolves to cp7.miraheze.org [20:01:20] RECOVERY - meta.orain.org - reverse DNS on sslhost is OK: rDNS OK - meta.orain.org reverse DNS resolves to cp6.miraheze.org [20:01:48] RECOVERY - mw6 Current Load on mw6 is OK: OK - load average: 2.88, 6.44, 6.54 [20:02:44] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 8 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb, 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [20:03:03] PROBLEM - mw6 MediaWiki Rendering on mw6 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 16474 bytes in 0.096 second response time [20:03:14] PROBLEM - cloud1 Current Load on cloud1 is WARNING: WARNING - load average: 22.51, 19.44, 17.33 [20:03:21] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 12.90, 7.93, 6.84 [20:03:46] RECOVERY - mw5 Puppet on mw5 is OK: OK: Puppet is currently enabled, last run 7 seconds ago with 0 failures [20:04:07] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 6 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 2607:5300:205:200::2ac4/cpweb [20:05:02] RECOVERY - mw6 MediaWiki Rendering on mw6 is OK: HTTP OK: HTTP/1.1 200 OK - 19604 bytes in 0.103 second response time [20:05:07] PROBLEM - wiki.teessidehackspace.org.uk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.teessidehackspace.org.uk could not be found [20:05:11] PROBLEM - chouverse.cludisciples.cyou - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for chouverse.cludisciples.cyou could not be found [20:05:11] PROBLEM - petrawiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for petrawiki.org could not be found [20:05:12] PROBLEM - sdiy.info - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for sdiy.info could not be found [20:05:14] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 14.27, 17.86, 17.04 [20:05:14] PROBLEM - vedopedia.witches-empire.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for vedopedia.witches-empire.com could not be found [20:05:16] PROBLEM - wikibeats.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wikibeats.net could not be found [20:05:17] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for zw.fontainebleau-avon.fr could not be found [20:05:18] PROBLEM - planetnomads.speedbomb.de - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for planetnomads.speedbomb.de could not be found [20:05:22] PROBLEM - debilepedie.gq - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for debilepedie.gq could not be found [20:05:22] PROBLEM - wiki.meeusen.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.meeusen.net could not be found [20:05:25] PROBLEM - wiki.worlduniversityandschool.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.worlduniversityandschool.org could not be found [20:05:25] PROBLEM - cyberlaw.ccdcoe.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for cyberlaw.ccdcoe.org could not be found [20:05:27] PROBLEM - en.phgalaxy.xyz - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for en.phgalaxy.xyz could not be found [20:05:28] PROBLEM - wiki.fourta.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.fourta.org could not be found [20:05:29] PROBLEM - piwik.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for piwik.miraheze.org could not be found [20:05:29] PROBLEM - www.allthetropes.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.allthetropes.org could not be found [20:05:31] PROBLEM - wiki.sdbx.dev - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.sdbx.dev could not be found [20:06:27] PROBLEM - opendatascot.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for opendatascot.org could not be found [20:06:53] PROBLEM - www.mcpk.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.mcpk.wiki could not be found [20:06:58] PROBLEM - wiki.mikrodev.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.mikrodev.com could not be found [20:08:06] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [20:08:19] PROBLEM - www.portalsofphereon.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.portalsofphereon.com could not be found [20:08:29] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [20:08:29] PROBLEM - wiki.grottocenter.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.grottocenter.org could not be found [20:08:37] PROBLEM - warmage.necrocity.co.uk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for warmage.necrocity.co.uk could not be found [20:11:55] RECOVERY - wiki.teessidehackspace.org.uk - reverse DNS on sslhost is OK: rDNS OK - wiki.teessidehackspace.org.uk reverse DNS resolves to cp6.miraheze.org [20:11:58] RECOVERY - planetnomads.speedbomb.de - reverse DNS on sslhost is OK: rDNS OK - planetnomads.speedbomb.de reverse DNS resolves to cp6.miraheze.org [20:12:03] RECOVERY - petrawiki.org - reverse DNS on sslhost is OK: rDNS OK - petrawiki.org reverse DNS resolves to cp7.miraheze.org [20:12:04] RECOVERY - wiki.meeusen.net - reverse DNS on sslhost is OK: rDNS OK - wiki.meeusen.net reverse DNS resolves to cp7.miraheze.org [20:12:05] RECOVERY - debilepedie.gq - reverse DNS on sslhost is OK: rDNS OK - debilepedie.gq reverse DNS resolves to cp6.miraheze.org [20:12:07] RECOVERY - chouverse.cludisciples.cyou - reverse DNS on sslhost is OK: rDNS OK - chouverse.cludisciples.cyou reverse DNS resolves to cp6.miraheze.org [20:12:11] RECOVERY - vedopedia.witches-empire.com - reverse DNS on sslhost is OK: rDNS OK - vedopedia.witches-empire.com reverse DNS resolves to cp7.miraheze.org [20:12:11] RECOVERY - wiki.worlduniversityandschool.org - reverse DNS on sslhost is OK: rDNS OK - wiki.worlduniversityandschool.org reverse DNS resolves to cp6.miraheze.org [20:12:11] RECOVERY - sdiy.info - reverse DNS on sslhost is OK: rDNS OK - sdiy.info reverse DNS resolves to cp7.miraheze.org [20:12:13] RECOVERY - zw.fontainebleau-avon.fr - reverse DNS on sslhost is OK: rDNS OK - zw.fontainebleau-avon.fr reverse DNS resolves to cp7.miraheze.org [20:12:15] RECOVERY - wikibeats.net - reverse DNS on sslhost is OK: rDNS OK - wikibeats.net reverse DNS resolves to cp6.miraheze.org [20:12:15] RECOVERY - en.phgalaxy.xyz - reverse DNS on sslhost is OK: rDNS OK - en.phgalaxy.xyz reverse DNS resolves to cp7.miraheze.org [20:12:19] RECOVERY - wiki.fourta.org - reverse DNS on sslhost is OK: rDNS OK - wiki.fourta.org reverse DNS resolves to cp7.miraheze.org [20:12:19] RECOVERY - piwik.miraheze.org - reverse DNS on sslhost is OK: rDNS OK - piwik.miraheze.org reverse DNS resolves to cp6.miraheze.org [20:12:21] RECOVERY - www.allthetropes.org - reverse DNS on sslhost is OK: rDNS OK - www.allthetropes.org reverse DNS resolves to cp6.miraheze.org [20:12:23] RECOVERY - cyberlaw.ccdcoe.org - reverse DNS on sslhost is OK: rDNS OK - cyberlaw.ccdcoe.org reverse DNS resolves to cp6.miraheze.org [20:12:25] RECOVERY - wiki.sdbx.dev - reverse DNS on sslhost is OK: rDNS OK - wiki.sdbx.dev reverse DNS resolves to cp7.miraheze.org [20:13:12] PROBLEM - cloud1 Current Load on cloud1 is CRITICAL: CRITICAL - load average: 26.15, 20.97, 18.28 [20:13:25] PROBLEM - jobrunner2 MediaWiki Rendering on jobrunner2 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 16482 bytes in 0.154 second response time [20:13:27] RECOVERY - opendatascot.org - reverse DNS on sslhost is OK: rDNS OK - opendatascot.org reverse DNS resolves to cp7.miraheze.org [20:13:43] RECOVERY - www.mcpk.wiki - reverse DNS on sslhost is OK: rDNS OK - www.mcpk.wiki reverse DNS resolves to cp7.miraheze.org [20:13:51] RECOVERY - wiki.mikrodev.com - reverse DNS on sslhost is OK: rDNS OK - wiki.mikrodev.com reverse DNS resolves to cp7.miraheze.org [20:14:14] !log rebuild lc on jobrunner* [20:14:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:14:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.59, 21.52, 17.74 [20:15:11] RECOVERY - wiki.grottocenter.org - reverse DNS on sslhost is OK: rDNS OK - wiki.grottocenter.org reverse DNS resolves to cp6.miraheze.org [20:15:11] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 18.01, 19.47, 18.03 [20:15:18] RECOVERY - www.portalsofphereon.com - reverse DNS on sslhost is OK: rDNS OK - www.portalsofphereon.com reverse DNS resolves to cp6.miraheze.org [20:15:25] RECOVERY - jobrunner2 MediaWiki Rendering on jobrunner2 is OK: HTTP OK: HTTP/1.1 200 OK - 19620 bytes in 1.081 second response time [20:15:28] RECOVERY - warmage.necrocity.co.uk - reverse DNS on sslhost is OK: rDNS OK - warmage.necrocity.co.uk reverse DNS resolves to cp6.miraheze.org [20:16:18] PROBLEM - jobrunner1 MediaWiki Rendering on jobrunner1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 16481 bytes in 0.116 second response time [20:18:17] RECOVERY - jobrunner1 MediaWiki Rendering on jobrunner1 is OK: HTTP OK: HTTP/1.1 200 OK - 19619 bytes in 0.219 second response time [20:18:37] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.08, 23.74, 19.66 [20:18:49] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 5.76, 7.20, 7.89 [20:19:12] PROBLEM - cloud1 Current Load on cloud1 is WARNING: WARNING - load average: 21.59, 20.45, 18.78 [20:20:49] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 8.67, 7.88, 8.06 [20:20:50] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 7.16, 7.35, 7.88 [20:21:11] PROBLEM - cloud1 Current Load on cloud1 is CRITICAL: CRITICAL - load average: 28.27, 23.32, 20.00 [20:22:33] PROBLEM - cp6 Stunnel Http for mw4 on cp6 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [20:22:36] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.21, 25.04, 21.01 [20:22:51] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 9.20, 8.19, 8.12 [20:23:23] PROBLEM - wiki.sketch.town - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.sketch.town could not be found [20:24:08] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 3 datacenters are down: 128.199.139.216/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb [20:24:24] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 51.77.107.210/cpweb, 2607:5300:205:200::2ac4/cpweb [20:24:31] RECOVERY - cp6 Stunnel Http for mw4 on cp6 is OK: HTTP OK: HTTP/1.1 200 OK - 15891 bytes in 0.047 second response time [20:24:32] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.43, 22.78, 20.65 [20:24:51] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 7.24, 7.70, 7.95 [20:26:09] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [20:26:50] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 6.44, 7.52, 7.94 [20:28:25] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [20:29:12] PROBLEM - cloud1 Current Load on cloud1 is WARNING: WARNING - load average: 14.92, 22.64, 22.07 [20:30:15] RECOVERY - wiki.sketch.town - reverse DNS on sslhost is OK: rDNS OK - wiki.sketch.town reverse DNS resolves to cp7.miraheze.org [20:30:28] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.76, 19.26, 19.78 [20:30:50] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 10.63, 8.23, 8.08 [20:32:50] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 6.59, 7.67, 7.90 [20:35:11] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 13.72, 17.89, 20.25 [20:36:50] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 3.50, 5.22, 6.61 [20:40:49] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 8.17, 6.55, 7.03 [20:43:13] PROBLEM - cloud1 Current Load on cloud1 is WARNING: WARNING - load average: 21.09, 19.28, 19.52 [20:44:13] !log reception@jobrunner1:~$ sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --update --wiki moviepediawiki [20:44:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:44:50] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 6.32, 6.90, 7.11 [20:45:13] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 14.77, 17.47, 18.82 [20:46:49] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 4.48, 6.06, 6.78 [20:52:50] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 6.51, 6.96, 6.95 [20:53:30] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 8.51, 7.47, 6.82 [20:54:50] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 8.69, 7.58, 7.16 [20:55:26] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 6.09, 6.89, 6.68 [20:56:49] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 7.11, 7.20, 7.06 [20:57:24] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 6.33, 6.55, 6.56 [21:00:49] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 9.93, 7.55, 7.16 [21:03:19] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 9.38, 7.69, 6.94 [21:06:49] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 5.08, 7.16, 7.26 [21:07:15] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 5.60, 7.28, 7.01 [21:09:11] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 5.20, 6.50, 6.75 [21:12:49] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 8.33, 7.86, 7.46 [21:13:08] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 8.07, 7.54, 7.10 [21:14:49] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 5.89, 7.12, 7.24 [21:15:04] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 6.78, 7.15, 7.01 [21:15:40] PROBLEM - db7 Check MariaDB Replication on db7 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 156s [21:16:33] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.84, 22.31, 19.26 [21:17:00] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 5.17, 6.46, 6.78 [21:18:35] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.82, 22.65, 19.70 [21:18:49] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 4.77, 5.54, 6.55 [21:22:59] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 7.31, 6.79, 6.82 [21:23:06] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 8 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb, 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [21:24:24] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 5 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb, 51.89.160.142/cpweb, 51.222.27.129/cpweb [21:24:54] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 4.27, 5.95, 6.52 [21:25:19] PROBLEM - cp7 Stunnel Http for mw4 on cp7 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [21:27:21] RECOVERY - cp7 Stunnel Http for mw4 on cp7 is OK: HTTP OK: HTTP/1.1 200 OK - 15891 bytes in 3.349 second response time [21:29:06] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [21:29:22] PROBLEM - debilepedie.gq - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for debilepedie.gq could not be found [21:29:26] PROBLEM - erikapedia.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for erikapedia.com could not be found [21:29:26] PROBLEM - wiki.candela.digital - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.candela.digital could not be found [21:29:27] PROBLEM - en.phgalaxy.xyz - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for en.phgalaxy.xyz could not be found [21:29:31] PROBLEM - wiki.fourta.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.fourta.org could not be found [21:29:33] PROBLEM - antiguabarbudacalypso.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for antiguabarbudacalypso.com could not be found [21:29:41] PROBLEM - animatedfeet.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for animatedfeet.miraheze.org could not be found [21:29:42] PROBLEM - www.fraterniterebellion.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.fraterniterebellion.com could not be found [21:29:46] PROBLEM - wiki.ripto.gq - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.ripto.gq could not be found [21:29:47] PROBLEM - munwiki.info - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for munwiki.info could not be found [21:32:12] !log increase gluster2 disk by 100gb [21:33:12] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 4 datacenters are down: 51.77.107.210/cpweb, 2001:41d0:800:1056::2/cpweb, 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [21:33:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:34:58] PROBLEM - spiral.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for spiral.wiki could not be found [21:35:00] PROBLEM - franchise.franchising.org.ua - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for franchise.franchising.org.ua could not be found [21:35:09] PROBLEM - www.permanentfuturelab.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.permanentfuturelab.wiki could not be found [21:35:10] RECOVERY - gluster2 Disk Space on gluster2 is OK: DISK OK - free space: / 97458 MB (18% inode=83%); [21:35:11] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [21:36:04] RECOVERY - debilepedie.gq - reverse DNS on sslhost is OK: rDNS OK - debilepedie.gq reverse DNS resolves to cp7.miraheze.org [21:36:14] RECOVERY - en.phgalaxy.xyz - reverse DNS on sslhost is OK: rDNS OK - en.phgalaxy.xyz reverse DNS resolves to cp7.miraheze.org [21:36:20] RECOVERY - wiki.fourta.org - reverse DNS on sslhost is OK: rDNS OK - wiki.fourta.org reverse DNS resolves to cp7.miraheze.org [21:36:21] RECOVERY - erikapedia.com - reverse DNS on sslhost is OK: rDNS OK - erikapedia.com reverse DNS resolves to cp7.miraheze.org [21:36:21] RECOVERY - wiki.candela.digital - reverse DNS on sslhost is OK: rDNS OK - wiki.candela.digital reverse DNS resolves to cp6.miraheze.org [21:36:24] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [21:36:27] RECOVERY - antiguabarbudacalypso.com - reverse DNS on sslhost is OK: rDNS OK - antiguabarbudacalypso.com reverse DNS resolves to cp7.miraheze.org [21:36:31] RECOVERY - wiki.ripto.gq - reverse DNS on sslhost is OK: rDNS OK - wiki.ripto.gq reverse DNS resolves to cp6.miraheze.org [21:36:34] RECOVERY - munwiki.info - reverse DNS on sslhost is OK: rDNS OK - munwiki.info reverse DNS resolves to cp7.miraheze.org [21:36:42] RECOVERY - www.fraterniterebellion.com - reverse DNS on sslhost is OK: rDNS OK - www.fraterniterebellion.com reverse DNS resolves to cp7.miraheze.org [21:36:42] RECOVERY - animatedfeet.miraheze.org - reverse DNS on sslhost is OK: rDNS OK - animatedfeet.miraheze.org reverse DNS resolves to cp7.miraheze.org [21:41:41] RECOVERY - spiral.wiki - reverse DNS on sslhost is OK: rDNS OK - spiral.wiki reverse DNS resolves to cp6.miraheze.org [21:41:44] RECOVERY - franchise.franchising.org.ua - reverse DNS on sslhost is OK: rDNS OK - franchise.franchising.org.ua reverse DNS resolves to cp6.miraheze.org [21:42:08] RECOVERY - www.permanentfuturelab.wiki - reverse DNS on sslhost is OK: rDNS OK - www.permanentfuturelab.wiki reverse DNS resolves to cp6.miraheze.org [21:43:36] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 6.01, 6.85, 6.48 [21:45:17] !log root@gluster2:/home/paladox# gluster volume rebalance mvol fix-layout start [21:45:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:45:31] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 4.94, 5.97, 6.19 [21:53:17] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 7.16, 6.59, 6.18 [21:53:34] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 7.12, 6.73, 6.20 [21:55:17] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 5.50, 6.20, 6.09 [21:55:32] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 4.29, 5.93, 5.98 [21:57:12] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 5 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb, 2001:41d0:800:1056::2/cpweb, 2607:5300:205:200::2ac4/cpweb [21:57:18] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2607:5300:205:200::2ac4/cpweb [21:57:30] PROBLEM - gluster1 Current Load on gluster1 is WARNING: WARNING - load average: 7.89, 7.19, 5.85 [21:59:11] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [21:59:12] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [22:01:31] RECOVERY - gluster1 Current Load on gluster1 is OK: OK - load average: 4.03, 5.79, 5.62 [22:03:24] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 7.39, 6.82, 6.18 [22:05:20] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 4.55, 6.00, 5.96 [22:12:14] PROBLEM - cloud1 Current Load on cloud1 is CRITICAL: CRITICAL - load average: 28.51, 21.31, 18.51 [22:13:13] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 8.25, 6.88, 6.17 [22:13:40] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 6.92, 5.86, 5.60 [22:14:13] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 18.65, 19.73, 18.25 [22:15:11] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 5.05, 6.14, 5.99 [22:15:35] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 5.19, 5.57, 5.52 [22:33:10] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 7.45, 6.03, 5.44 [22:35:06] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 4.46, 5.21, 5.20 [22:47:12] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2400:6180:0:d0::403:f001/cpweb [22:51:14] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [22:53:43] PROBLEM - en.phgalaxy.xyz - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for en.phgalaxy.xyz could not be found [22:59:01] PROBLEM - gluster1 Current Load on gluster1 is CRITICAL: CRITICAL - load average: 8.95, 7.42, 6.08 [22:59:12] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2400:6180:0:d0::403:f001/cpweb, 2001:41d0:800:105a::10/cpweb [23:00:56] RECOVERY - en.phgalaxy.xyz - reverse DNS on sslhost is OK: rDNS OK - en.phgalaxy.xyz reverse DNS resolves to cp6.miraheze.org [23:01:27] PROBLEM - mw7 Puppet on mw7 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:03:22] PROBLEM - gluster1 Current Load on gluster1 is WARNING: WARNING - load average: 6.88, 7.13, 6.29 [23:03:28] PROBLEM - mw7 Puppet on mw7 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 3 hours ago with 1 failures [23:03:31] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [23:05:20] RECOVERY - gluster1 Current Load on gluster1 is OK: OK - load average: 5.54, 6.64, 6.21 [23:05:23] PROBLEM - mw7 Puppet on mw7 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 minute ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [23:12:54] PROBLEM - www.dariawiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.dariawiki.org could not be found [23:12:57] PROBLEM - nonbinary.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for nonbinary.wiki could not be found [23:12:58] PROBLEM - spiral.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for spiral.wiki could not be found [23:12:59] PROBLEM - tep.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for tep.wiki could not be found [23:13:00] PROBLEM - franchise.franchising.org.ua - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for franchise.franchising.org.ua could not be found [23:13:14] PROBLEM - kk.uncyc.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for kk.uncyc.tk could not be found [23:13:24] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 7.85, 6.88, 5.77 [23:14:28] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 3 datacenters are down: 51.77.107.210/cpweb, 51.89.160.142/cpweb, 2607:5300:205:200::2ac4/cpweb [23:15:10] PROBLEM - gluster1 Current Load on gluster1 is CRITICAL: CRITICAL - load average: 9.03, 7.20, 6.50 [23:15:20] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 6.04, 6.79, 5.89 [23:16:07] PROBLEM - es.nocyclo.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for es.nocyclo.tk could not be found [23:16:07] RECOVERY - db7 Check MariaDB Replication on db7 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 0s [23:16:07] PROBLEM - savage-wiki.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for savage-wiki.com could not be found [23:16:10] PROBLEM - wiki.openhatch.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.openhatch.org could not be found [23:16:13] PROBLEM - iesd.mobagenie.my.id - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for iesd.mobagenie.my.id could not be found [23:16:27] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [23:16:28] PROBLEM - en.phgalaxy.xyz - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for en.phgalaxy.xyz could not be found [23:16:33] PROBLEM - en.nocyclo.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for en.nocyclo.tk could not be found [23:16:51] RECOVERY - mw7 Puppet on mw7 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:17:08] RECOVERY - gluster1 Current Load on gluster1 is OK: OK - load average: 4.32, 6.20, 6.23 [23:19:34] RECOVERY - www.dariawiki.org - reverse DNS on sslhost is OK: rDNS OK - www.dariawiki.org reverse DNS resolves to cp6.miraheze.org [23:19:42] RECOVERY - spiral.wiki - reverse DNS on sslhost is OK: rDNS OK - spiral.wiki reverse DNS resolves to cp6.miraheze.org [23:19:47] RECOVERY - franchise.franchising.org.ua - reverse DNS on sslhost is OK: rDNS OK - franchise.franchising.org.ua reverse DNS resolves to cp7.miraheze.org [23:19:51] RECOVERY - nonbinary.wiki - reverse DNS on sslhost is OK: rDNS OK - nonbinary.wiki reverse DNS resolves to cp6.miraheze.org [23:19:52] RECOVERY - tep.wiki - reverse DNS on sslhost is OK: rDNS OK - tep.wiki reverse DNS resolves to cp7.miraheze.org [23:20:13] RECOVERY - kk.uncyc.tk - reverse DNS on sslhost is OK: rDNS OK - kk.uncyc.tk reverse DNS resolves to cp6.miraheze.org [23:22:49] RECOVERY - savage-wiki.com - reverse DNS on sslhost is OK: rDNS OK - savage-wiki.com reverse DNS resolves to cp6.miraheze.org [23:22:49] RECOVERY - es.nocyclo.tk - reverse DNS on sslhost is OK: rDNS OK - es.nocyclo.tk reverse DNS resolves to cp7.miraheze.org [23:22:54] RECOVERY - wiki.openhatch.org - reverse DNS on sslhost is OK: rDNS OK - wiki.openhatch.org reverse DNS resolves to cp6.miraheze.org [23:23:00] RECOVERY - iesd.mobagenie.my.id - reverse DNS on sslhost is OK: rDNS OK - iesd.mobagenie.my.id reverse DNS resolves to cp7.miraheze.org [23:23:16] RECOVERY - en.phgalaxy.xyz - reverse DNS on sslhost is OK: rDNS OK - en.phgalaxy.xyz reverse DNS resolves to cp7.miraheze.org [23:23:26] RECOVERY - en.nocyclo.tk - reverse DNS on sslhost is OK: rDNS OK - en.nocyclo.tk reverse DNS resolves to cp7.miraheze.org [23:30:27] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.48, 17.08, 23.80 [23:38:27] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.61, 15.30, 20.24 [23:52:50] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 7.61, 5.85, 4.93 [23:54:50] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 4.08, 5.14, 4.78