[01:30:55] miraheze/mw-config/Universal-Omega-patch-3/01118a9 - Universal Omega The build has errored. https://travis-ci.org/miraheze/mw-config/builds/749157484 [01:42:22] .tell blackwidowmovie0 While this may be outside the scope of Miraheze system administrators and stewards, I'm asking you to stop spamming your Marvel Cinematic Universe Discord server, each with different invite codes, to Miraheze users, broadly construed. If you do not stop, we will remove the pin for your server in the Miraheze Discord server, and I would just add this is not a good look for you [01:42:22] dmehus: I'll pass that on when blackwidowmovie0 is around. [02:46:50] @dmehus if you're not gonna respond to my message on my talk page on Meta, just tell md [02:46:51] blackwidowmovie0: 2020-12-12 - 01:42:22UTC tell blackwidowmovie0 While this may be outside the scope of Miraheze system administrators and stewards, I'm asking you to stop spamming your Marvel Cinematic Universe Discord server, each with different invite codes, to Miraheze users, broadly construed. If you do not stop, we will remove the pin for your server in the Miraheze Discord server, and I would just add this is not a good look for [03:59:24] @blackwidowmovie0 I'm getting there. I've been busy. [04:31:19] miraheze/MirahezeMagic/Universal-Omega-patch-3/9c1953c - Universal Omega The build has errored. https://travis-ci.org/miraheze/MirahezeMagic/builds/749173438 [05:36:11] [02CreateWiki] 07R4356th synchronize pull request 03#174: Replace "Reason" with "Description" - 13https://git.io/Jk5Qk [05:50:54] [02CreateWiki] 07R4356th commented on pull request 03#174: Replace "Reason" with "Description" - 13https://git.io/JI608 [05:55:12] [02CreateWiki] 07Universal-Omega commented on pull request 03#174: Replace "Reason" with "Description" - 13https://git.io/JI60j [05:58:40] [02CreateWiki] 07R4356th commented on pull request 03#174: Replace "Reason" with "Description" - 13https://git.io/JI6E0 [06:01:52] [02CreateWiki] 07Universal-Omega commented on pull request 03#174: Replace "Reason" with "Description" - 13https://git.io/JI6EF [06:02:51] [02CreateWiki] 07Universal-Omega edited a comment on pull request 03#174: Replace "Reason" with "Description" - 13https://git.io/JI6EF [06:18:26] [02CreateWiki] 07R4356th commented on pull request 03#174: Replace "Reason" with "Description" - 13https://git.io/JI6gq [06:20:25] [02CreateWiki] 07Universal-Omega commented on pull request 03#174: Replace "Reason" with "Description" - 13https://git.io/JI6gE [06:22:14] [02CreateWiki] 07Universal-Omega commented on pull request 03#174: Replace "Reason" with "Description" - 13https://git.io/JI6gi [06:25:14] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JI6gx [06:25:16] [02miraheze/ManageWiki] 07Universal-Omega 0362829c9 - add {{Identical|Description}} [06:25:17] [02ManageWiki] 07Universal-Omega synchronize pull request 03#209: Add description field to Special:ManageWiki (T6575) - 13https://git.io/JIKXP [06:41:29] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/JI6aw [06:41:30] [02miraheze/ssl] 07Reception123 03375ff1a - add wiki.etn.link cert [06:41:58] [02CreateWiki] 07Universal-Omega synchronize pull request 03#175: Allow for wiki descriptions (T6575) - 13https://git.io/JIKXt [06:45:52] RECOVERY - wiki.abyogsms.org.in - reverse DNS on sslhost is OK: rDNS OK - wiki.abyogsms.org.in reverse DNS resolves to cp6.miraheze.org [06:48:52] RECOVERY - wiki.abyogsms.org.in - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.abyogsms.org.in' will expire on Sun 17 Jan 2021 03:52:35 GMT +0000. [09:19:25] !log stripped 2FA from a user on phabricator [09:19:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:33:00] Hi, in tuscriaturas miraheze we want reference previews active on default but we want to disable the popups on links that are not references [13:54:39] PROBLEM - vitriol.spoon.army - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'vitriol.spoon.army' expires in 15 day(s) (Mon 28 Dec 2020 13:52:40 GMT +0000). [13:56:40] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIi8p [13:56:41] [02miraheze/ssl] 07MirahezeSSLBot 0340619dc - Bot: Update SSL cert for vitriol.spoon.army [14:26:33] @avengium you can have both Reference Previews and Navigation Popups (the gadget) on your wiki, but individual users cannot enable both the Reference Previews extension feature and the gadget concurrently; the two are incompatible to be used simultaneously [14:28:23] RECOVERY - vitriol.spoon.army - LetsEncrypt on sslhost is OK: OK - Certificate 'vitriol.spoon.army' will expire on Fri 12 Mar 2021 12:56:33 GMT +0000. [14:55:32] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.01, 18.46, 15.20 [14:57:32] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 13.00, 16.43, 14.85 [16:04:39] @dmehus That current link is defunct [16:04:42] I need a new one [16:05:10] https://discord.gg/DGVbG6b3Cx [16:05:13] that one^ [16:16:34] @Reception123#2349 I have a new Data Dump awaiting importation _(also sending in Discord)_ [16:19:34] for starwarswiki [16:25:16] PROBLEM - icclopedia.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'icclopedia.org' expires in 15 day(s) (Mon 28 Dec 2020 16:21:20 GMT +0000). [16:27:06] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIi77 [16:27:08] [02miraheze/ssl] 07MirahezeSSLBot 03d2b4065 - Bot: Update SSL cert for icclopedia.org [16:34:56] PROBLEM - ipv6bolivia.tk - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'ipv6bolivia.tk' expires in 15 day(s) (Mon 28 Dec 2020 16:28:59 GMT +0000). [16:35:16] @blackwidowmovie0 I've talked to @Reception123 actually, and we're considering removing your Discord server's pin actually, as other user reports keep coming in of you sending unsolicited direct messages promoting your server [16:35:48] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIidm [16:35:49] [02miraheze/ssl] 07MirahezeSSLBot 0301043bb - Bot: Update SSL cert for ipv6bolivia.tk [16:37:03] @dmehus I haven't sent them since at least 12 hours ago [16:40:52] PROBLEM - munwiki.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'munwiki.info' expires in 15 day(s) (Mon 28 Dec 2020 16:38:20 GMT +0000). [16:46:00] @blackwidowmovie0 Haven't seen what? Nonetheless, the pin's been updated to remove your Discord server, as we do not like to have our users be sent mass unsolicited direct messages advertising some facet of a wiki or Discord server [16:46:26] haven't sent? [16:46:43] I haven't sent the invites since 12 hours ago [16:46:54] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIiFK [16:46:55] [02miraheze/ssl] 07MirahezeSSLBot 03bd7b942 - Bot: Update SSL cert for munwiki.info [16:47:02] and I only planned on sending them once [16:47:22] I just wanted to mention that you sent me 3. [16:48:39] 2 were invalid [16:53:32] That does not mean you are allowed to send me two more. I am going to step out of this conversation adhering to the Code of Conduct. Thank you. [17:00:03] RECOVERY - icclopedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'icclopedia.org' will expire on Fri 12 Mar 2021 15:26:58 GMT +0000. [17:01:19] RECOVERY - munwiki.info - LetsEncrypt on sslhost is OK: OK - Certificate 'munwiki.info' will expire on Fri 12 Mar 2021 15:46:47 GMT +0000. [17:02:35] RECOVERY - ipv6bolivia.tk - LetsEncrypt on sslhost is OK: OK - Certificate 'ipv6bolivia.tk' will expire on Sat 27 Feb 2021 09:01:10 GMT +0000. [17:04:20] @Universal_Omega I got the full dump [17:04:32] DM me in Discord [17:05:01] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 6.83, 5.10, 3.89 [17:05:43] Arso, thank you for vistal editor as it really helped [17:06:59] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 4.32, 4.63, 3.86 [17:32:54] where @BurningPrincess1, which wiki? [17:34:13] On Idril [17:34:39] I am happy Mirahese makes adding stuff easyer as its a pain to do tables in wikitext alone [19:00:58] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-3 [+0/-0/±1] 13https://git.io/JIPfH [19:01:00] [02miraheze/mw-config] 07Universal-Omega 03f085cb8 - set $wgMFAutodetectMobileView['default'] to true (hopefully will fix T6362) Per https://github.com/wikimedia/mediawiki-extensions-MobileFrontend/blob/1421405c1fe4f3dbafe0cd18f43aa9054b974c4e/extension.json#L911 [19:01:00] [ mediawiki-extensions-MobileFrontend/extension.json at 1421405c1fe4f3dbafe0cd18f43aa9054b974c4e · wikimedia/mediawiki-extensions-MobileFrontend · GitHub ] - github.com [19:01:01] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-3 - 13https://git.io/vbvb3 [19:01:23] [02mw-config] 07Universal-Omega opened pull request 03#3556: set $wgMFAutodetectMobileView['default'] to true (hopefully will fix T6362) - 13https://git.io/JIPfQ [19:01:49] [02mw-config] 07Universal-Omega closed pull request 03#3556: set $wgMFAutodetectMobileView['default'] to true (hopefully will fix T6362) - 13https://git.io/JIPfQ [19:01:50] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIPfd [19:01:52] [02miraheze/mw-config] 07Universal-Omega 03960ccf3 - set $wgMFAutodetectMobileView['default'] to true (hopefully will fix T6362) (#3556) Per https://github.com/wikimedia/mediawiki-extensions-MobileFrontend/blob/1421405c1fe4f3dbafe0cd18f43aa9054b974c4e/extension.json#L911 [19:01:53] [ mediawiki-extensions-MobileFrontend/extension.json at 1421405c1fe4f3dbafe0cd18f43aa9054b974c4e · wikimedia/mediawiki-extensions-MobileFrontend · GitHub ] - github.com [19:01:53] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-3 [19:01:55] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-3 - 13https://git.io/vbvb3 [19:33:42] miraheze/mw-config/Universal-Omega-patch-3/f085cb8 - Universal Omega The build has errored. https://travis-ci.org/miraheze/mw-config/builds/749311813 [19:51:08] !log reception@jobrunner1:/srv/mediawiki/w/maintenance$ sudo -u www-data php rebuildImages.php --wiki schizoidnightmareswiki [19:51:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:52:25] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03revert-3556-Universal-Omega-patch-3 [+0/-0/±1] 13https://git.io/JIPTy [19:52:26] [02miraheze/mw-config] 07Universal-Omega 036f4ec45 - Revert "set $wgMFAutodetectMobileView['default'] to true (hopefully will fix T6362) (#3556)" This reverts commit 960ccf33bc70066305b5becfb35310c506f5a429. [19:52:28] [02mw-config] 07Universal-Omega created branch 03revert-3556-Universal-Omega-patch-3 - 13https://git.io/vbvb3 [19:52:29] [02mw-config] 07Universal-Omega opened pull request 03#3557: Revert "set $wgMFAutodetectMobileView['default'] to true (hopefully will fix T6362)" - 13https://git.io/JIPTS [19:52:38] [02mw-config] 07Universal-Omega closed pull request 03#3557: Revert "set $wgMFAutodetectMobileView['default'] to true (hopefully will fix T6362)" - 13https://git.io/JIPTS [19:52:39] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIPT9 [19:52:41] [02miraheze/mw-config] 07Universal-Omega 03fb59ba9 - Revert "set $wgMFAutodetectMobileView['default'] to true (hopefully will fix T6362) (#3556)" (#3557) This reverts commit 960ccf33bc70066305b5becfb35310c506f5a429. [19:52:43] [02miraheze/mw-config] 07Universal-Omega deleted branch 03revert-3556-Universal-Omega-patch-3 [19:52:44] [02mw-config] 07Universal-Omega deleted branch 03revert-3556-Universal-Omega-patch-3 - 13https://git.io/vbvb3 [20:08:05] PROBLEM - mw7 Current Load on mw7 is WARNING: WARNING - load average: 6.97, 6.98, 5.57 [20:10:03] RECOVERY - mw7 Current Load on mw7 is OK: OK - load average: 4.69, 6.25, 5.47 [20:24:31] miraheze/mw-config/revert-3556-Universal-Omega-patch-3/6f4ec45 - Universal Omega The build has errored. https://travis-ci.org/miraheze/mw-config/builds/749318366 [20:31:17] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-3 [+0/-0/±1] 13https://git.io/JIPLw [20:31:19] [02miraheze/mw-config] 07Universal-Omega 033fe91fc - fix a few 'from' values [20:31:20] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-3 - 13https://git.io/vbvb3 [20:31:23] [02mw-config] 07Universal-Omega opened pull request 03#3558: fix a few 'from' values - 13https://git.io/JIPLi [20:32:50] [02mw-config] 07Universal-Omega reviewed pull request 03#3554 commit - 13https://git.io/JIPLj [20:36:03] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 8.23, 6.67, 5.17 [20:37:00] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 8.63, 6.86, 5.42 [20:38:02] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 5.28, 6.08, 5.14 [20:38:58] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 5.29, 6.14, 5.32 [20:49:47] Hi there. What do I need to do so that the TOC links are displayed in Russian? Is this feature available on Miraheze? I mean when I'm going thru a TOC link I see an insrpiction like this in my addressbar: #.D0.91.D0.BB.D0.BE.D0.B3.D0.B3.D0.B5.D1.80.D1.8B [20:52:42] @CamaradSecuritat I am really not sure, but this could be an issue related to Miraheze's parsing of unicode characters in wikilinks, so maybe a [[Phabricator]] task would your best bet? @Reception123, any objections to that? [20:53:19] no, that would be fine [21:01:35] miraheze/mw-config/Universal-Omega-patch-3/3fe91fc - Universal Omega The build passed. https://travis-ci.org/miraheze/mw-config/builds/749323695 [21:21:23] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIPmg [21:21:24] [02miraheze/puppet] 07paladox 039381558 - Varnish: Only whitelist matomo.miraheze.org for connect-src [21:26:03] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 8 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb, 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [21:26:10] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIPYR [21:26:12] [02miraheze/mw-config] 07paladox 035eccc78 - Revert "Revert "Revert "Revert "QUICK HACK: $wgMessageCacheType change to NONE"""" This reverts commit cea3011baf5e56aea1ac7b8c4702861505f659b9. [21:26:21] PROBLEM - cp9 Puppet on cp9 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[varnish] [21:26:52] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 7 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [21:27:11] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 6.83, 6.10, 5.31 [21:27:35] PROBLEM - cp9 Stunnel Http for mw7 on cp9 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [21:28:10] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 33.14, 24.36, 19.58 [21:28:21] PROBLEM - cp7 Stunnel Http for mw7 on cp7 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [21:28:57] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIPYD [21:28:59] [02miraheze/mw-config] 07paladox 036fb3b23 - Revert "Revert "Revert "Revert "Revert "QUICK HACK: $wgMessageCacheType change to NONE""""" This reverts commit 5eccc782cf515b1fddac528863b3b5e7bc00f807. [21:29:43] RECOVERY - cp9 Stunnel Http for mw7 on cp9 is OK: HTTP OK: HTTP/1.1 200 OK - 15955 bytes in 7.048 second response time [21:30:02] !log killed a varnish process on cp9 as the varnish service wasn't maintaining it causing the service to fail when restarting. [21:30:22] RECOVERY - cp9 Puppet on cp9 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [21:30:24] RECOVERY - cp7 Stunnel Http for mw7 on cp7 is OK: HTTP OK: HTTP/1.1 200 OK - 15955 bytes in 4.305 second response time [21:31:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:33:35] PROBLEM - rothwell-leeds.co.uk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for rothwell-leeds.co.uk could not be found [21:33:35] PROBLEM - www.pornwiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.pornwiki.org could not be found [21:33:36] PROBLEM - www.christipedia.nl - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.christipedia.nl could not be found [21:33:37] PROBLEM - madgenderscience.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for madgenderscience.wiki could not be found [21:33:38] PROBLEM - www.wikiescola.com.br - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.wikiescola.com.br could not be found [21:33:39] PROBLEM - wiki.ostans.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.ostans.com could not be found [21:33:42] PROBLEM - erikapedia.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for erikapedia.com could not be found [21:34:37] PROBLEM - mw5 MediaWiki Rendering on mw5 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:35:04] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 8.77, 7.27, 5.88 [21:35:09] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 9.78, 7.61, 6.26 [21:35:36] PROBLEM - wiki.ssij.se - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.ssij.se could not be found [21:36:14] PROBLEM - mw7 Current Load on mw7 is WARNING: WARNING - load average: 6.83, 6.96, 6.20 [21:36:43] RECOVERY - mw5 MediaWiki Rendering on mw5 is OK: HTTP OK: HTTP/1.1 200 OK - 19981 bytes in 6.014 second response time [21:37:06] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 4.82, 6.57, 6.05 [21:37:24] PROBLEM - www.programming.red - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.programming.red could not be found [21:37:25] PROBLEM - housing.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for housing.wiki could not be found [21:37:25] PROBLEM - gluster1 Current Load on gluster1 is CRITICAL: CRITICAL - load average: 8.97, 7.20, 5.24 [21:37:29] PROBLEM - mon1 Current Load on mon1 is CRITICAL: CRITICAL - load average: 5.03, 2.83, 1.65 [21:37:30] PROBLEM - wiki.invinciblewellbeing.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.invinciblewellbeing.com could not be found [21:37:32] PROBLEM - nenawiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for nenawiki.org could not be found [21:37:33] PROBLEM - en.wikireserve.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for en.wikireserve.com could not be found [21:37:36] PROBLEM - wiki.helioss.co - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.helioss.co could not be found [21:37:38] PROBLEM - aman.awiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for aman.awiki.org could not be found [21:37:39] PROBLEM - archive.stellurgists.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for archive.stellurgists.wiki could not be found [21:37:40] PROBLEM - wiki.warframestat.us - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.warframestat.us could not be found [21:37:40] PROBLEM - bharatwiki.online - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for bharatwiki.online could not be found [21:37:42] PROBLEM - wiki.tulpa.info - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.tulpa.info could not be found [21:37:43] PROBLEM - wiki.ripto.gq - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.ripto.gq could not be found [21:37:47] PROBLEM - wiki.nowchess.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.nowchess.org could not be found [21:37:59] PROBLEM - wiki.anglish.info - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.anglish.info could not be found [21:38:00] PROBLEM - wiki.gesamtschule-nordkirchen.de - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.gesamtschule-nordkirchen.de could not be found [21:38:00] PROBLEM - meta.orain.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for meta.orain.org could not be found [21:38:13] RECOVERY - mw7 Current Load on mw7 is OK: OK - load average: 4.07, 6.08, 5.98 [21:39:06] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 3.65, 6.03, 5.77 [21:39:26] PROBLEM - gluster1 Current Load on gluster1 is WARNING: WARNING - load average: 7.83, 7.29, 5.51 [21:39:31] RECOVERY - mon1 Current Load on mon1 is OK: OK - load average: 2.08, 2.56, 1.71 [21:40:15] PROBLEM - cp9 Stunnel Http for mw6 on cp9 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [21:40:29] PROBLEM - cp7 Stunnel Http for mw6 on cp7 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [21:40:30] RECOVERY - www.pornwiki.org - reverse DNS on sslhost is OK: rDNS OK - www.pornwiki.org reverse DNS resolves to cp7.miraheze.org [21:40:30] RECOVERY - rothwell-leeds.co.uk - reverse DNS on sslhost is OK: rDNS OK - rothwell-leeds.co.uk reverse DNS resolves to cp6.miraheze.org [21:40:33] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-7 [+0/-0/±1] 13https://git.io/JIPOE [21:40:35] [02miraheze/mw-config] 07Universal-Omega 03489ec97 - set $wgFragmentMode (to fix T6584) [21:40:36] RECOVERY - www.wikiescola.com.br - reverse DNS on sslhost is OK: rDNS OK - www.wikiescola.com.br reverse DNS resolves to cp9.miraheze.org [21:40:36] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-7 - 13https://git.io/vbvb3 [21:40:38] RECOVERY - wiki.ostans.com - reverse DNS on sslhost is OK: rDNS OK - wiki.ostans.com reverse DNS resolves to cp7.miraheze.org [21:40:39] [02mw-config] 07Universal-Omega opened pull request 03#3559: set $wgFragmentMode (to fix T6584) - 13https://git.io/JIPOu [21:40:41] PROBLEM - mw6 MediaWiki Rendering on mw6 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:42:21] RECOVERY - cp9 Stunnel Http for mw6 on cp9 is OK: HTTP OK: HTTP/1.1 200 OK - 15955 bytes in 9.866 second response time [21:42:24] RECOVERY - wiki.ssij.se - reverse DNS on sslhost is OK: rDNS OK - wiki.ssij.se reverse DNS resolves to cp7.miraheze.org [21:42:46] RECOVERY - mw6 MediaWiki Rendering on mw6 is OK: HTTP OK: HTTP/1.1 200 OK - 19981 bytes in 6.522 second response time [21:43:21] RECOVERY - gluster1 Current Load on gluster1 is OK: OK - load average: 5.57, 6.54, 5.61 [21:44:09] RECOVERY - wiki.invinciblewellbeing.com - reverse DNS on sslhost is OK: rDNS OK - wiki.invinciblewellbeing.com reverse DNS resolves to cp6.miraheze.org [21:44:11] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [21:44:11] RECOVERY - nenawiki.org - reverse DNS on sslhost is OK: rDNS OK - nenawiki.org reverse DNS resolves to cp7.miraheze.org [21:44:14] RECOVERY - en.wikireserve.com - reverse DNS on sslhost is OK: rDNS OK - en.wikireserve.com reverse DNS resolves to cp7.miraheze.org [21:44:18] RECOVERY - www.programming.red - reverse DNS on sslhost is OK: rDNS OK - www.programming.red reverse DNS resolves to cp6.miraheze.org [21:44:19] RECOVERY - housing.wiki - reverse DNS on sslhost is OK: rDNS OK - housing.wiki reverse DNS resolves to cp7.miraheze.org [21:44:23] RECOVERY - aman.awiki.org - reverse DNS on sslhost is OK: rDNS OK - aman.awiki.org reverse DNS resolves to cp6.miraheze.org [21:44:30] RECOVERY - wiki.warframestat.us - reverse DNS on sslhost is OK: rDNS OK - wiki.warframestat.us reverse DNS resolves to cp7.miraheze.org [21:44:30] RECOVERY - bharatwiki.online - reverse DNS on sslhost is OK: rDNS OK - bharatwiki.online reverse DNS resolves to cp7.miraheze.org [21:44:33] RECOVERY - wiki.tulpa.info - reverse DNS on sslhost is OK: rDNS OK - wiki.tulpa.info reverse DNS resolves to cp6.miraheze.org [21:44:33] PROBLEM - www.bluevies.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.bluevies.org could not be found [21:44:33] PROBLEM - commons.gyaanipedia.co.in - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for commons.gyaanipedia.co.in could not be found [21:44:35] RECOVERY - archive.stellurgists.wiki - reverse DNS on sslhost is OK: rDNS OK - archive.stellurgists.wiki reverse DNS resolves to cp7.miraheze.org [21:44:35] RECOVERY - wiki.helioss.co - reverse DNS on sslhost is OK: rDNS OK - wiki.helioss.co reverse DNS resolves to cp6.miraheze.org [21:44:35] RECOVERY - cp7 Stunnel Http for mw6 on cp7 is OK: HTTP OK: HTTP/1.1 200 OK - 15955 bytes in 1.918 second response time [21:44:36] PROBLEM - wiki.campaign-labour.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.campaign-labour.org could not be found [21:44:39] RECOVERY - wiki.ripto.gq - reverse DNS on sslhost is OK: rDNS OK - wiki.ripto.gq reverse DNS resolves to cp6.miraheze.org [21:44:40] RECOVERY - wiki.anglish.info - reverse DNS on sslhost is OK: rDNS OK - wiki.anglish.info reverse DNS resolves to cp7.miraheze.org [21:44:40] RECOVERY - wiki.nowchess.org - reverse DNS on sslhost is OK: rDNS OK - wiki.nowchess.org reverse DNS resolves to cp6.miraheze.org [21:44:43] RECOVERY - meta.orain.org - reverse DNS on sslhost is OK: rDNS OK - meta.orain.org reverse DNS resolves to cp7.miraheze.org [21:44:43] RECOVERY - wiki.gesamtschule-nordkirchen.de - reverse DNS on sslhost is OK: rDNS OK - wiki.gesamtschule-nordkirchen.de reverse DNS resolves to cp7.miraheze.org [21:45:51] [02mw-config] 07paladox closed pull request 03#3559: set $wgFragmentMode (to fix T6584) - 13https://git.io/JIPOu [21:45:52] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIPO1 [21:45:54] [02miraheze/mw-config] 07Universal-Omega 0377fd0a4 - set $wgFragmentMode (to fix T6584) (#3559) [21:45:55] [02mw-config] 07paladox deleted branch 03Universal-Omega-patch-7 - 13https://git.io/vbvb3 [21:45:57] [02miraheze/mw-config] 07paladox deleted branch 03Universal-Omega-patch-7 [21:47:04] RECOVERY - erikapedia.com - reverse DNS on sslhost is OK: rDNS OK - erikapedia.com reverse DNS resolves to cp7.miraheze.org [21:47:18] PROBLEM - mw6 MediaWiki Rendering on mw6 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:47:31] PROBLEM - gluster1 Current Load on gluster1 is WARNING: WARNING - load average: 7.41, 7.17, 6.11 [21:47:32] RECOVERY - www.christipedia.nl - reverse DNS on sslhost is OK: rDNS OK - www.christipedia.nl reverse DNS resolves to cp6.miraheze.org [21:47:34] RECOVERY - madgenderscience.wiki - reverse DNS on sslhost is OK: rDNS OK - madgenderscience.wiki reverse DNS resolves to cp6.miraheze.org [21:47:42] PROBLEM - cp3 Stunnel Http for mw6 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [21:47:50] PROBLEM - vitriol.spoon.army - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for vitriol.spoon.army could not be found [21:47:50] PROBLEM - wiki.worldsofweary.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.worldsofweary.com could not be found [21:47:51] PROBLEM - wiki.arnab.at - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.arnab.at could not be found [21:47:54] PROBLEM - miraheze.gq - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for miraheze.gq could not be found [21:47:56] PROBLEM - wiki.usagihime.ml - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.usagihime.ml could not be found [21:48:18] PROBLEM - mw7 MediaWiki Rendering on mw7 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 344 bytes in 3.145 second response time [21:48:18] !Log restart nginx & php7.3-fpm on mw* and jobrunner. Php being slow. [21:48:23] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 5 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb [21:48:27] PROBLEM - cp9 Stunnel Http for mw7 on cp9 is CRITICAL: HTTP CRITICAL - No data received from host [21:48:57] !log restart nginx & php7.3-fpm on mw* and jobrunner. php being slow. [21:49:00] PROBLEM - cp9 Stunnel Http for mw6 on cp9 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [21:49:17] RECOVERY - mw6 MediaWiki Rendering on mw6 is OK: HTTP OK: HTTP/1.1 200 OK - 19980 bytes in 0.106 second response time [21:49:44] RECOVERY - cp3 Stunnel Http for mw6 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15955 bytes in 1.054 second response time [21:50:12] PROBLEM - mw7 Current Load on mw7 is CRITICAL: CRITICAL - load average: 8.69, 7.27, 6.42 [21:50:14] RECOVERY - mw7 MediaWiki Rendering on mw7 is OK: HTTP OK: HTTP/1.1 200 OK - 19981 bytes in 0.402 second response time [21:50:24] RECOVERY - cp9 Stunnel Http for mw7 on cp9 is OK: HTTP OK: HTTP/1.1 200 OK - 15955 bytes in 0.566 second response time [21:50:26] PROBLEM - mw6 Current Load on mw6 is CRITICAL: CRITICAL - load average: 9.08, 8.52, 7.11 [21:50:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:50:56] RECOVERY - cp9 Stunnel Http for mw6 on cp9 is OK: HTTP OK: HTTP/1.1 200 OK - 15955 bytes in 0.326 second response time [21:51:14] RECOVERY - www.bluevies.org - reverse DNS on sslhost is OK: rDNS OK - www.bluevies.org reverse DNS resolves to cp6.miraheze.org [21:51:15] RECOVERY - commons.gyaanipedia.co.in - reverse DNS on sslhost is OK: rDNS OK - commons.gyaanipedia.co.in reverse DNS resolves to cp7.miraheze.org [21:51:24] RECOVERY - wiki.campaign-labour.org - reverse DNS on sslhost is OK: rDNS OK - wiki.campaign-labour.org reverse DNS resolves to cp7.miraheze.org [21:52:21] PROBLEM - mw6 Current Load on mw6 is WARNING: WARNING - load average: 6.92, 7.87, 7.04 [21:54:16] PROBLEM - mw6 Current Load on mw6 is CRITICAL: CRITICAL - load average: 9.62, 8.27, 7.26 [21:54:22] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [21:54:30] RECOVERY - vitriol.spoon.army - reverse DNS on sslhost is OK: rDNS OK - vitriol.spoon.army reverse DNS resolves to cp7.miraheze.org [21:54:33] RECOVERY - wiki.worldsofweary.com - reverse DNS on sslhost is OK: rDNS OK - wiki.worldsofweary.com reverse DNS resolves to cp7.miraheze.org [21:54:38] RECOVERY - wiki.arnab.at - reverse DNS on sslhost is OK: rDNS OK - wiki.arnab.at reverse DNS resolves to cp7.miraheze.org [21:54:44] RECOVERY - miraheze.gq - reverse DNS on sslhost is OK: rDNS OK - miraheze.gq reverse DNS resolves to cp7.miraheze.org [21:54:44] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [21:54:48] RECOVERY - wiki.usagihime.ml - reverse DNS on sslhost is OK: rDNS OK - wiki.usagihime.ml reverse DNS resolves to cp6.miraheze.org [21:56:20] .tell Reception123 Okay, cool, thanks. :) [21:56:21] dmehus: I'll pass that on when Reception123 is around. [21:58:07] PROBLEM - mw6 Current Load on mw6 is WARNING: WARNING - load average: 4.79, 6.97, 6.99 [21:58:12] PROBLEM - mw7 Current Load on mw7 is WARNING: WARNING - load average: 5.62, 6.99, 6.80 [21:58:19] PROBLEM - wiki.make717.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.make717.org could not be found [21:58:20] PROBLEM - savagepedia.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for savagepedia.wiki could not be found [21:58:26] PROBLEM - munwiki.info - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for munwiki.info could not be found [21:58:26] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 8 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb, 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [21:58:27] PROBLEM - mrlove.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for mrlove.wiki could not be found [21:58:42] PROBLEM - wikiescola.com.br - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wikiescola.com.br could not be found [21:58:46] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 7 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [21:59:21] RECOVERY - gluster1 Current Load on gluster1 is OK: OK - load average: 6.25, 6.66, 6.43 [21:59:45] !log upgrade grafana on mon1 [21:59:45] PROBLEM - mw6 MediaWiki Rendering on mw6 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:00:27] [02mw-config] 07Universal-Omega closed pull request 03#3558: fix a few 'from' values - 13https://git.io/JIPLi [22:00:28] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIP3R [22:00:30] [02miraheze/mw-config] 07Universal-Omega 034e49a7b - fix a few 'from' values (#3558) [22:00:31] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-3 [22:00:33] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-3 - 13https://git.io/vbvb3 [22:00:33] PROBLEM - www.permanentfuturelab.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.permanentfuturelab.wiki could not be found [22:00:36] PROBLEM - wiki.helioss.co - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.helioss.co could not be found [22:00:36] PROBLEM - www.opendatascot.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.opendatascot.org could not be found [22:00:36] PROBLEM - wiki.ripto.gq - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.ripto.gq could not be found [22:00:37] PROBLEM - wiki.grottocenter.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.grottocenter.org could not be found [22:00:38] PROBLEM - mr.gyaanipedia.co.in - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for mr.gyaanipedia.co.in could not be found [22:00:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:00:48] PROBLEM - wiki.pablojprieto.com.es - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.pablojprieto.com.es could not be found [22:00:49] PROBLEM - wiki.responsibly.ai - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.responsibly.ai could not be found [22:00:52] PROBLEM - wiki.triplescripts.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.triplescripts.org could not be found [22:01:14] [02mw-config] 07Universal-Omega closed pull request 03#3549: set $wgGenerateThumbnailOnParse to true if PdfHandler is enabled (to fix T6469) - 13https://git.io/JIEzC [22:01:16] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIP3E [22:01:17] [02miraheze/mw-config] 07Universal-Omega 03f5afeec - set $wgGenerateThumbnailOnParse to true if PdfHandler is enabled (#3549) [22:01:19] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-5 [22:01:20] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-5 - 13https://git.io/vbvb3 [22:01:44] RECOVERY - mw6 MediaWiki Rendering on mw6 is OK: HTTP OK: HTTP/1.1 200 OK - 19981 bytes in 0.301 second response time [22:02:24] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [22:02:43] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [22:05:09] RECOVERY - wiki.make717.org - reverse DNS on sslhost is OK: rDNS OK - wiki.make717.org reverse DNS resolves to cp7.miraheze.org [22:05:09] RECOVERY - savagepedia.wiki - reverse DNS on sslhost is OK: rDNS OK - savagepedia.wiki reverse DNS resolves to cp6.miraheze.org [22:05:12] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 7.78, 6.81, 6.15 [22:05:20] RECOVERY - munwiki.info - reverse DNS on sslhost is OK: rDNS OK - munwiki.info reverse DNS resolves to cp7.miraheze.org [22:05:22] RECOVERY - mrlove.wiki - reverse DNS on sslhost is OK: rDNS OK - mrlove.wiki reverse DNS resolves to cp7.miraheze.org [22:05:30] RECOVERY - wikiescola.com.br - reverse DNS on sslhost is OK: rDNS OK - wikiescola.com.br reverse DNS resolves to cp7.miraheze.org [22:05:46] PROBLEM - mw6 Current Load on mw6 is CRITICAL: CRITICAL - load average: 8.41, 7.64, 7.27 [22:06:39] !ops My JavaScript files won't load anywhere on Miraheze, and when trying to use VisualEditor on any wiki, "Could not connect to the server. Make sure you have a working internet connection and try again." is returned. Also getting semi-frequent 502 Bad Gateway errors [22:06:39] * SigmaBotMH yelps for help [22:07:11] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 5.88, 6.28, 6.03 [22:07:20] RECOVERY - wiki.grottocenter.org - reverse DNS on sslhost is OK: rDNS OK - wiki.grottocenter.org reverse DNS resolves to cp6.miraheze.org [22:07:20] RECOVERY - mr.gyaanipedia.co.in - reverse DNS on sslhost is OK: rDNS OK - mr.gyaanipedia.co.in reverse DNS resolves to cp7.miraheze.org [22:07:28] RECOVERY - www.permanentfuturelab.wiki - reverse DNS on sslhost is OK: rDNS OK - www.permanentfuturelab.wiki reverse DNS resolves to cp6.miraheze.org [22:07:33] RECOVERY - www.opendatascot.org - reverse DNS on sslhost is OK: rDNS OK - www.opendatascot.org reverse DNS resolves to cp7.miraheze.org [22:07:33] RECOVERY - wiki.ripto.gq - reverse DNS on sslhost is OK: rDNS OK - wiki.ripto.gq reverse DNS resolves to cp6.miraheze.org [22:07:34] RECOVERY - wiki.responsibly.ai - reverse DNS on sslhost is OK: rDNS OK - wiki.responsibly.ai reverse DNS resolves to cp6.miraheze.org [22:07:35] RECOVERY - wiki.pablojprieto.com.es - reverse DNS on sslhost is OK: rDNS OK - wiki.pablojprieto.com.es reverse DNS resolves to cp7.miraheze.org [22:07:36] RECOVERY - wiki.helioss.co - reverse DNS on sslhost is OK: rDNS OK - wiki.helioss.co reverse DNS resolves to cp6.miraheze.org [22:07:41] PROBLEM - mw6 Current Load on mw6 is WARNING: WARNING - load average: 6.55, 7.34, 7.22 [22:07:42] RECOVERY - wiki.triplescripts.org - reverse DNS on sslhost is OK: rDNS OK - wiki.triplescripts.org reverse DNS resolves to cp7.miraheze.org [22:07:45] miraheze/mw-config/Universal-Omega-patch-7/489ec97 - Universal Omega The build has errored. https://travis-ci.org/miraheze/mw-config/builds/749332573 [22:08:03] PROBLEM - mw7 Current Load on mw7 is CRITICAL: CRITICAL - load average: 8.69, 7.43, 7.11 [22:10:01] PROBLEM - mw7 Current Load on mw7 is WARNING: WARNING - load average: 6.73, 7.28, 7.10 [22:10:43] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://git.io/JIP3y [22:10:44] [02miraheze/puppet] 07paladox 03d09baaa - Matomo: Add a queued tracking cron [22:10:46] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://git.io/vbiAS [22:10:47] [02puppet] 07paladox opened pull request 03#1552: Matomo: Add a queued tracking cron - 13https://git.io/JIP3S [22:11:34] [02puppet] 07paladox closed pull request 03#1552: Matomo: Add a queued tracking cron - 13https://git.io/JIP3S [22:11:36] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIP39 [22:11:37] [02miraheze/puppet] 07paladox 03cbaf8f4 - Matomo: Add a queued tracking cron (#1552) [22:11:39] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://git.io/vbiAS [22:11:40] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [22:13:37] [02puppet] 07Universal-Omega opened pull request 03#1553: revert "Varnish: Only whitelist matomo.miraheze.org for connect-src" - 13https://git.io/JIP35 [22:15:56] RECOVERY - mw7 Current Load on mw7 is OK: OK - load average: 3.76, 5.61, 6.46 [22:15:59] PROBLEM - gluster1 Current Load on gluster1 is CRITICAL: CRITICAL - load average: 9.72, 6.97, 6.24 [22:16:25] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 8 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb, 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [22:17:19] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 6 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 2607:5300:205:200::2ac4/cpweb [22:17:19] RECOVERY - mw6 Current Load on mw6 is OK: OK - load average: 4.54, 5.91, 6.70 [22:17:56] PROBLEM - gluster1 Current Load on gluster1 is WARNING: WARNING - load average: 7.52, 7.22, 6.43 [22:18:11] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIPs3 [22:18:13] [02miraheze/puppet] 07paladox 03a283652 - varnish: Add 'self' back to connect-src [22:18:36] [02puppet] 07Universal-Omega closed pull request 03#1553: revert "Varnish: Only whitelist matomo.miraheze.org for connect-src" - 13https://git.io/JIP35 [22:18:56] PROBLEM - vedopedia.witches-empire.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for vedopedia.witches-empire.com could not be found [22:19:13] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [22:19:32] [02mw-config] 07paladox commented on pull request 03#3549: set $wgGenerateThumbnailOnParse to true if PdfHandler is enabled (to fix T6469) - 13https://git.io/JIPsW [22:19:53] RECOVERY - gluster1 Current Load on gluster1 is OK: OK - load average: 6.56, 6.77, 6.35 [22:20:39] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03revert-3549-Universal-Omega-patch-5 [+0/-0/±1] 13https://git.io/JIPsg [22:20:41] [02miraheze/mw-config] 07Universal-Omega 039c45bc5 - Revert "set $wgGenerateThumbnailOnParse to true if PdfHandler is enabled (#3549)" This reverts commit f5afeec0901ba388ae2cd8ab2f200476b89feca1. [22:20:42] [02mw-config] 07Universal-Omega created branch 03revert-3549-Universal-Omega-patch-5 - 13https://git.io/vbvb3 [22:20:44] [02mw-config] 07Universal-Omega opened pull request 03#3560: Revert " set $wgGenerateThumbnailOnParse to true if PdfHandler is enabled (to fix T6469)" - 13https://git.io/JIPsi [22:20:50] [02mw-config] 07Universal-Omega closed pull request 03#3560: Revert " set $wgGenerateThumbnailOnParse to true if PdfHandler is enabled (to fix T6469)" - 13https://git.io/JIPsi [22:20:51] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIPsD [22:20:53] [02miraheze/mw-config] 07Universal-Omega 03570fef0 - Revert "set $wgGenerateThumbnailOnParse to true if PdfHandler is enabled (#3549)" (#3560) This reverts commit f5afeec0901ba388ae2cd8ab2f200476b89feca1. [22:20:54] [02mw-config] 07Universal-Omega deleted branch 03revert-3549-Universal-Omega-patch-5 - 13https://git.io/vbvb3 [22:20:56] [02miraheze/mw-config] 07Universal-Omega deleted branch 03revert-3549-Universal-Omega-patch-5 [22:22:48] PROBLEM - cp6 Stunnel Http for mw6 on cp6 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:22:51] PROBLEM - cp3 Stunnel Http for mw6 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:23:08] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIPGW [22:23:10] [02miraheze/puppet] 07paladox 035a7642d - Varnish: Remove connect-src [22:23:14] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 6 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb [22:24:29] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [22:24:46] RECOVERY - cp6 Stunnel Http for mw6 on cp6 is OK: HTTP OK: HTTP/1.1 200 OK - 15955 bytes in 0.022 second response time [22:24:50] RECOVERY - cp3 Stunnel Http for mw6 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15955 bytes in 1.026 second response time [22:25:08] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [22:25:43] RECOVERY - vedopedia.witches-empire.com - reverse DNS on sslhost is OK: rDNS OK - vedopedia.witches-empire.com reverse DNS resolves to cp6.miraheze.org [22:26:01] PROBLEM - cp7 Puppet on cp7 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[varnish] [22:26:38] PROBLEM - cp6 Puppet on cp6 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[varnish] [22:27:59] RECOVERY - cp7 Puppet on cp7 is OK: OK: Puppet is currently enabled, last run 9 seconds ago with 0 failures [22:28:48] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[varnish] [22:30:56] PROBLEM - mw6 Current Load on mw6 is CRITICAL: CRITICAL - load average: 9.19, 7.40, 6.84 [22:32:35] RECOVERY - cp6 Puppet on cp6 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:32:55] PROBLEM - mw6 Current Load on mw6 is WARNING: WARNING - load average: 7.56, 7.29, 6.86 [22:34:46] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 5 datacenters are down: 128.199.139.216/cpweb, 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb, 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [22:34:46] PROBLEM - mw7 Current Load on mw7 is CRITICAL: CRITICAL - load average: 8.56, 6.96, 6.49 [22:36:16] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 7.70, 7.58, 6.48 [22:36:38] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 2400:6180:0:d0::403:f001/cpweb, 2001:41d0:800:1056::2/cpweb [22:38:16] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 4.40, 6.69, 6.31 [22:38:45] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [22:38:55] PROBLEM - mw6 Current Load on mw6 is CRITICAL: CRITICAL - load average: 9.32, 8.10, 7.33 [22:40:42] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [22:40:56] PROBLEM - mw6 Current Load on mw6 is WARNING: WARNING - load average: 6.44, 7.55, 7.23 [22:42:24] miraheze/mw-config/revert-3549-Universal-Omega-patch-5/9c45bc5 - Universal Omega The build has errored. https://travis-ci.org/miraheze/mw-config/builds/749336632 [22:44:45] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:44:45] PROBLEM - mw7 Current Load on mw7 is WARNING: WARNING - load average: 5.77, 7.41, 7.28 [22:44:57] RECOVERY - mw6 Current Load on mw6 is OK: OK - load average: 3.76, 5.81, 6.62 [22:48:31] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-6 [+0/-0/±1] 13https://git.io/JIPZw [22:48:33] [02miraheze/mw-config] 07Universal-Omega 03d05cebb - use += [22:48:34] [02mw-config] 07Universal-Omega synchronize pull request 03#3553: Update configs to modern wikibase; re-format Wikibase.php - 13https://git.io/JI2Jl [22:48:49] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 17.54, 20.20, 23.65 [22:50:49] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 34.74, 24.65, 24.79 [22:52:43] PROBLEM - mw7 Current Load on mw7 is CRITICAL: CRITICAL - load average: 9.16, 7.62, 7.34 [22:54:55] PROBLEM - mw6 Current Load on mw6 is CRITICAL: CRITICAL - load average: 9.63, 7.52, 6.90 [22:57:44] PROBLEM - wiki.worldsofweary.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.worldsofweary.com could not be found [22:57:45] PROBLEM - hellointernet.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for hellointernet.miraheze.org could not be found [22:57:46] PROBLEM - wiki.s23.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.s23.org could not be found [22:58:37] PROBLEM - mw7 Current Load on mw7 is WARNING: WARNING - load average: 6.92, 7.83, 7.55 [22:58:55] [02miraheze/MatomoAnalytics] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JIPZ5 [22:58:56] [02miraheze/MatomoAnalytics] 07paladox 034d33cbb - Introduce wgMatomoAnalyticsForGetRequest config Allows users to force GET requests instead of POST. This also disables beacon and fallbacks to matomo 3 default method. [22:58:58] [02MatomoAnalytics] 07paladox created branch 03paladox-patch-1 - 13https://git.io/fN4LT [22:58:59] [02MatomoAnalytics] 07paladox opened pull request 03#28: Introduce wgMatomoAnalyticsForGetRequest config - 13https://git.io/JIPZd [23:00:02] [02miraheze/MatomoAnalytics] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JIPZp [23:00:03] [02miraheze/MatomoAnalytics] 07paladox 033104489 - Update extension.json [23:00:05] [02MatomoAnalytics] 07paladox synchronize pull request 03#28: Introduce wgMatomoAnalyticsForGetRequest config - 13https://git.io/JIPZd [23:00:39] PROBLEM - cp3 Stunnel Http for mw7 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:00:40] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 6 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb [23:00:50] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 8 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb, 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [23:01:05] PROBLEM - mw6 Current Load on mw6 is WARNING: WARNING - load average: 6.76, 7.25, 7.04 [23:01:24] [02miraheze/MatomoAnalytics] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JIPnf [23:01:26] [02miraheze/MatomoAnalytics] 07paladox 03901c280 - Update CHANGELOG [23:01:27] [02MatomoAnalytics] 07paladox synchronize pull request 03#28: Introduce wgMatomoAnalyticsForGetRequest config - 13https://git.io/JIPZd [23:01:36] [02miraheze/MatomoAnalytics] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JIPnU [23:01:38] [02miraheze/MatomoAnalytics] 07paladox 033387d19 - Update extension.json [23:01:39] [02MatomoAnalytics] 07paladox synchronize pull request 03#28: Introduce wgMatomoAnalyticsForGetRequest config - 13https://git.io/JIPZd [23:01:53] [02miraheze/MatomoAnalytics] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JIPnk [23:01:55] [02miraheze/MatomoAnalytics] 07paladox 03c40d684 - Update extension.json [23:01:56] [02MatomoAnalytics] 07paladox synchronize pull request 03#28: Introduce wgMatomoAnalyticsForGetRequest config - 13https://git.io/JIPZd [23:02:34] PROBLEM - gluster1 Current Load on gluster1 is WARNING: WARNING - load average: 7.90, 7.17, 5.86 [23:02:50] RECOVERY - cp3 Stunnel Http for mw7 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15955 bytes in 6.438 second response time [23:03:00] PROBLEM - mw6 Current Load on mw6 is CRITICAL: CRITICAL - load average: 8.23, 7.43, 7.12 [23:03:20] PROBLEM - cp9 Stunnel Http for mw6 on cp9 is CRITICAL: HTTP CRITICAL - No data received from host [23:03:23] PROBLEM - cp6 Stunnel Http for mw6 on cp6 is CRITICAL: HTTP CRITICAL - No data received from host [23:04:02] !ops Very slow page load times persist and encountering 502 Bad Gateways again [23:04:02] * SigmaBotMH yelps for help [23:04:27] RECOVERY - wiki.worldsofweary.com - reverse DNS on sslhost is OK: rDNS OK - wiki.worldsofweary.com reverse DNS resolves to cp7.miraheze.org [23:04:30] RECOVERY - hellointernet.miraheze.org - reverse DNS on sslhost is OK: rDNS OK - hellointernet.miraheze.org reverse DNS resolves to cp7.miraheze.org [23:04:31] RECOVERY - gluster1 Current Load on gluster1 is OK: OK - load average: 6.11, 6.61, 5.80 [23:04:32] RECOVERY - wiki.s23.org - reverse DNS on sslhost is OK: rDNS OK - wiki.s23.org reverse DNS resolves to cp6.miraheze.org [23:04:55] PROBLEM - mw6 Current Load on mw6 is WARNING: WARNING - load average: 7.40, 7.22, 7.07 [23:05:20] RECOVERY - cp9 Stunnel Http for mw6 on cp9 is OK: HTTP OK: HTTP/1.1 200 OK - 15955 bytes in 3.766 second response time [23:05:23] RECOVERY - cp6 Stunnel Http for mw6 on cp6 is OK: HTTP OK: HTTP/1.1 200 OK - 15955 bytes in 1.273 second response time [23:06:37] [02MatomoAnalytics] 07paladox closed pull request 03#28: Introduce wgMatomoAnalyticsForGetRequest config - 13https://git.io/JIPZd [23:06:38] [02miraheze/MatomoAnalytics] 07paladox pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/JIPnY [23:06:40] [02miraheze/MatomoAnalytics] 07paladox 03aca246e - Introduce wgMatomoAnalyticsForGetRequest config (#28) * Introduce wgMatomoAnalyticsForGetRequest config Allows users to force GET requests instead of POST. This also disables beacon and fallbacks to matomo 3 default method. * Update extension.json * Update CHANGELOG * Update extension.json * Update extension.json [23:06:41] [02MatomoAnalytics] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/fN4LT [23:06:43] [02miraheze/MatomoAnalytics] 07paladox deleted branch 03paladox-patch-1 [23:07:24] miraheze/mw-config/master/570fef0 - Universal Omega The build passed. https://travis-ci.org/miraheze/mw-config/builds/749336717 [23:07:43] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JIPn3 [23:07:44] [02miraheze/mediawiki] 07paladox 030fe059c - Update MatomoAnalytics [23:08:42] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [23:08:42] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [23:08:52] [02miraheze/MatomoAnalytics] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JIPnZ [23:08:53] [02miraheze/MatomoAnalytics] 07paladox 033fe950a - Fix config name [23:08:55] [02MatomoAnalytics] 07paladox created branch 03paladox-patch-1 - 13https://git.io/fN4LT [23:08:56] [02MatomoAnalytics] 07paladox opened pull request 03#29: Fix config name - 13https://git.io/JIPnn [23:09:20] [02miraheze/MatomoAnalytics] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JIPnW [23:09:21] [02miraheze/MatomoAnalytics] 07paladox 033e9cd5e - Update MatomoAnalyticsHooks.php [23:09:23] [02MatomoAnalytics] 07paladox synchronize pull request 03#29: Fix config name - 13https://git.io/JIPnn [23:09:54] [02miraheze/MatomoAnalytics] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JIPn8 [23:09:55] [02miraheze/MatomoAnalytics] 07paladox 03e5e0d09 - Update CHANGELOG [23:09:57] [02MatomoAnalytics] 07paladox synchronize pull request 03#29: Fix config name - 13https://git.io/JIPnn [23:10:05] [02miraheze/MatomoAnalytics] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JIPn4 [23:10:06] [02miraheze/MatomoAnalytics] 07paladox 03ce86662 - Update extension.json [23:10:08] [02MatomoAnalytics] 07paladox synchronize pull request 03#29: Fix config name - 13https://git.io/JIPnn [23:10:23] [02MatomoAnalytics] 07paladox closed pull request 03#29: Fix config name - 13https://git.io/JIPnn [23:10:24] [02miraheze/MatomoAnalytics] 07paladox pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/JIPnB [23:10:26] [02miraheze/MatomoAnalytics] 07paladox 030a71809 - Fix config name (#29) * Fix config name * Update MatomoAnalyticsHooks.php * Update CHANGELOG * Update extension.json [23:10:27] [02MatomoAnalytics] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/fN4LT [23:10:29] [02miraheze/MatomoAnalytics] 07paladox deleted branch 03paladox-patch-1 [23:10:47] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JIPnE [23:10:48] [02miraheze/mw-config] 07paladox 031a14fd1 - Set wgMatomoAnalyticsForceGetRequest to true [23:10:50] [02mw-config] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbvb3 [23:10:51] [02mw-config] 07paladox opened pull request 03#3561: Set wgMatomoAnalyticsForceGetRequest to true - 13https://git.io/JIPnu [23:10:56] PROBLEM - mw6 Current Load on mw6 is CRITICAL: CRITICAL - load average: 8.23, 7.53, 7.25 [23:11:14] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JIPnz [23:11:16] [02miraheze/mediawiki] 07paladox 0303ee74e - Update MatomoAnalytics [23:11:38] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JIPng [23:11:40] [02miraheze/mw-config] 07paladox 035ef56f1 - Update LocalSettings.php [23:11:41] [02mw-config] 07paladox synchronize pull request 03#3561: Set wgMatomoAnalyticsForceGetRequest to true - 13https://git.io/JIPnu [23:12:56] PROBLEM - mw6 Current Load on mw6 is WARNING: WARNING - load average: 6.61, 7.07, 7.11 [23:13:05] [02mw-config] 07paladox closed pull request 03#3561: Set wgMatomoAnalyticsForceGetRequest to true - 13https://git.io/JIPnu [23:13:07] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIPna [23:13:08] [02miraheze/mw-config] 07paladox 030371eb2 - Set wgMatomoAnalyticsForceGetRequest to true (#3561) * Set wgMatomoAnalyticsForceGetRequest to true * Update LocalSettings.php [23:13:10] [02mw-config] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbvb3 [23:13:11] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-1 [23:14:45] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 5 datacenters are down: 128.199.139.216/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [23:16:12] PROBLEM - gluster1 Current Load on gluster1 is WARNING: WARNING - load average: 7.33, 7.04, 6.26 [23:16:40] RECOVERY - mw7 Current Load on mw7 is OK: OK - load average: 4.89, 5.78, 6.54 [23:16:42] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [23:18:58] PROBLEM - mw6 Current Load on mw6 is CRITICAL: CRITICAL - load average: 9.24, 7.85, 7.43 [23:20:35] PROBLEM - mw7 Current Load on mw7 is WARNING: WARNING - load average: 6.92, 6.84, 6.83 [23:20:42] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 4 datacenters are down: 128.199.139.216/cpweb, 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb, 2607:5300:205:200::2ac4/cpweb [23:20:45] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [23:22:57] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [23:23:01] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [23:24:22] PROBLEM - gluster1 Current Load on gluster1 is CRITICAL: CRITICAL - load average: 8.88, 7.47, 6.70 [23:24:50] RECOVERY - mw7 Current Load on mw7 is OK: OK - load average: 4.78, 6.26, 6.64 [23:26:57] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 3 datacenters are down: 128.199.139.216/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb [23:27:03] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 2607:5300:205:200::2ac4/cpweb [23:27:07] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JIPnh [23:27:09] [02miraheze/mw-config] 07paladox 03bd51537 - Revert "Set wgMatomoAnalyticsForceGetRequest to true (#3561)" This reverts commit 0371eb26a444421b49989e329d03c63021c60bae. [23:28:48] PROBLEM - mw7 Current Load on mw7 is CRITICAL: CRITICAL - load average: 11.00, 7.95, 7.18 [23:30:12] PROBLEM - gluster1 Current Load on gluster1 is WARNING: WARNING - load average: 7.83, 7.38, 6.88 [23:30:47] PROBLEM - mw7 Current Load on mw7 is WARNING: WARNING - load average: 7.88, 7.81, 7.22 [23:31:43] !log restart mysql on db13 as it lost connection when running a query thus breaking it [23:31:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:32:42] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [23:33:02] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [23:33:24] PROBLEM - www.christipedia.nl - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.christipedia.nl could not be found [23:33:24] PROBLEM - www.baharna.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.baharna.org could not be found [23:33:27] PROBLEM - pa.gyaanipedia.co.in - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for pa.gyaanipedia.co.in could not be found [23:33:30] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-3 [+0/-0/±1] 13https://git.io/JIPcm [23:33:32] [02miraheze/mw-config] 07Universal-Omega 035abec1e - move configs from LW [23:33:33] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-3 - 13https://git.io/vbvb3 [23:33:43] [02mw-config] 07Universal-Omega opened pull request 03#3562: move configs from LW to LS for isvwiki - 13https://git.io/JIPcO [23:34:06] RECOVERY - gluster1 Current Load on gluster1 is OK: OK - load average: 4.93, 6.50, 6.67 [23:34:31] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-3 [+0/-0/±1] 13https://git.io/JIPcG [23:34:33] [02miraheze/mw-config] 07Universal-Omega 039fdd4aa - move configs to LS [23:34:34] [02mw-config] 07Universal-Omega synchronize pull request 03#3562: move configs from LW to LS for isvwiki - 13https://git.io/JIPcO [23:35:16] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 8.65, 7.87, 6.35 [23:37:17] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 5.33, 6.64, 6.07 [23:38:39] RECOVERY - mw7 Current Load on mw7 is OK: OK - load average: 4.41, 6.24, 6.72 [23:40:04] RECOVERY - www.baharna.org - reverse DNS on sslhost is OK: rDNS OK - www.baharna.org reverse DNS resolves to cp6.miraheze.org [23:40:12] RECOVERY - pa.gyaanipedia.co.in - reverse DNS on sslhost is OK: rDNS OK - pa.gyaanipedia.co.in reverse DNS resolves to cp6.miraheze.org [23:40:19] RECOVERY - www.christipedia.nl - reverse DNS on sslhost is OK: rDNS OK - www.christipedia.nl reverse DNS resolves to cp6.miraheze.org [23:40:55] PROBLEM - mw6 Current Load on mw6 is WARNING: WARNING - load average: 5.40, 7.03, 7.76 [23:44:55] RECOVERY - mw6 Current Load on mw6 is OK: OK - load average: 3.83, 4.93, 6.70 [23:53:23] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 17.11, 19.28, 23.75 [23:57:12] miraheze/MatomoAnalytics/paladox-patch-1/ce86662 - paladox The build has errored. https://travis-ci.org/miraheze/MatomoAnalytics/builds/749343370