[06:04:59] <icinga-miraheze>	 PROBLEM - misc3 Puppet on misc3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[nagios-plugins]
[06:12:55] <icinga-miraheze>	 RECOVERY - misc3 Puppet on misc3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[06:26:46] <icinga-miraheze>	 RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 2909 MB (12% inode=94%);
[10:08:47] <icinga-miraheze>	 PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2649 MB (10% inode=94%);
[10:10:47] <icinga-miraheze>	 RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 2689 MB (11% inode=94%);
[10:46:46] <icinga-miraheze>	 PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2623 MB (10% inode=94%);
[10:50:46] <icinga-miraheze>	 RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 2652 MB (11% inode=94%);
[10:54:46] <icinga-miraheze>	 PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2648 MB (10% inode=94%);
[14:09:43] <icinga-miraheze>	 PROBLEM - cp3 Stunnel Http for mw2 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[14:10:25] <icinga-miraheze>	 PROBLEM - cp4 Stunnel Http for mw2 on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[14:10:35] <icinga-miraheze>	 PROBLEM - mw1 MediaWiki Rendering on mw1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Backend fetch failed - 4210 bytes in 0.027 second response time
[14:10:38] <icinga-miraheze>	 PROBLEM - test1 MediaWiki Rendering on test1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Backend fetch failed - 4210 bytes in 0.024 second response time
[14:10:39] <icinga-miraheze>	 PROBLEM - lizardfs6 MediaWiki Rendering on lizardfs6 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Backend fetch failed - 4212 bytes in 0.050 second response time
[14:10:44] <icinga-miraheze>	 PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[14:10:54] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[14:11:04] <icinga-miraheze>	 PROBLEM - cp2 Stunnel Http for mw2 on cp2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[14:11:10] <icinga-miraheze>	 PROBLEM - cp2 Stunnel Http for mw1 on cp2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[14:11:26] <icinga-miraheze>	 PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 3 backends are down. mw1 mw2 mw3
[14:11:30] <icinga-miraheze>	 PROBLEM - cp3 Stunnel Http for mw1 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[14:11:39] <icinga-miraheze>	 PROBLEM - mw2 MediaWiki Rendering on mw2 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Backend fetch failed - 4212 bytes in 0.022 second response time
[14:11:50] <icinga-miraheze>	 PROBLEM - cp4 Stunnel Http for mw1 on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[14:12:06] <icinga-miraheze>	 PROBLEM - cp2 HTTP 4xx/5xx ERROR Rate on cp2 is WARNING: WARNING - NGINX Error Rate is 41%
[14:12:08] <icinga-miraheze>	 PROBLEM - cp4 Stunnel Http for mw3 on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[14:12:20] <icinga-miraheze>	 PROBLEM - mw3 MediaWiki Rendering on mw3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[14:12:23] <icinga-miraheze>	 RECOVERY - cp4 Stunnel Http for mw2 on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 15302 bytes in 0.005 second response time
[14:12:26] <icinga-miraheze>	 PROBLEM - cp4 Varnish Backends on cp4 is CRITICAL: 2 backends are down. lizardfs6 mw1
[14:12:36] <icinga-miraheze>	 RECOVERY - mw1 MediaWiki Rendering on mw1 is OK: HTTP OK: HTTP/1.1 200 OK - 18685 bytes in 2.152 second response time
[14:12:42] <icinga-miraheze>	 RECOVERY - test1 MediaWiki Rendering on test1 is OK: HTTP OK: HTTP/1.1 200 OK - 18685 bytes in 7.259 second response time
[14:12:52] <icinga-miraheze>	 PROBLEM - cp2 Varnish Backends on cp2 is CRITICAL: 2 backends are down. mw1 mw3
[14:12:58] <icinga-miraheze>	 PROBLEM - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is WARNING: WARNING - NGINX Error Rate is 41%
[14:13:11] <icinga-miraheze>	 RECOVERY - cp2 Stunnel Http for mw1 on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 15301 bytes in 3.296 second response time
[14:13:32] <icinga-miraheze>	 RECOVERY - cp3 Stunnel Http for mw1 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15295 bytes in 0.500 second response time
[14:13:42] <icinga-miraheze>	 RECOVERY - mw2 MediaWiki Rendering on mw2 is OK: HTTP OK: HTTP/1.1 200 OK - 18684 bytes in 0.267 second response time
[14:14:54] <icinga-miraheze>	 PROBLEM - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is CRITICAL: CRITICAL - NGINX Error Rate is 67%
[14:16:12] <icinga-miraheze>	 PROBLEM - cp2 HTTP 4xx/5xx ERROR Rate on cp2 is CRITICAL: CRITICAL - NGINX Error Rate is 69%
[14:16:51] <icinga-miraheze>	 PROBLEM - mw1 MediaWiki Rendering on mw1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Backend fetch failed - 4212 bytes in 0.021 second response time
[14:16:55] <icinga-miraheze>	 PROBLEM - test1 MediaWiki Rendering on test1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Backend fetch failed - 4210 bytes in 0.031 second response time
[14:18:06] <icinga-miraheze>	 RECOVERY - cp2 HTTP 4xx/5xx ERROR Rate on cp2 is OK: OK - NGINX Error Rate is 11%
[14:19:08] <icinga-miraheze>	 RECOVERY - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is OK: OK - NGINX Error Rate is 10%
[14:19:16] <icinga-miraheze>	 RECOVERY - lizardfs6 MediaWiki Rendering on lizardfs6 is OK: HTTP OK: HTTP/1.1 200 OK - 18684 bytes in 0.790 second response time
[14:19:27] <icinga-miraheze>	 RECOVERY - cp2 Stunnel Http for mw2 on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 15296 bytes in 0.371 second response time
[14:20:18] <icinga-miraheze>	 RECOVERY - cp3 Stunnel Http for mw2 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15296 bytes in 3.677 second response time
[14:20:18] <icinga-miraheze>	 RECOVERY - cp4 Stunnel Http for mw1 on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 15295 bytes in 0.008 second response time
[14:20:35] <icinga-miraheze>	 RECOVERY - cp4 Stunnel Http for mw3 on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 15288 bytes in 0.016 second response time
[14:20:42] <icinga-miraheze>	 RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online
[14:20:51] <icinga-miraheze>	 RECOVERY - mw3 MediaWiki Rendering on mw3 is OK: HTTP OK: HTTP/1.1 200 OK - 18685 bytes in 0.374 second response time
[14:21:02] <icinga-miraheze>	 RECOVERY - mw1 MediaWiki Rendering on mw1 is OK: HTTP OK: HTTP/1.1 200 OK - 18683 bytes in 0.841 second response time
[14:21:04] <icinga-miraheze>	 RECOVERY - cp2 Varnish Backends on cp2 is OK: All 7 backends are healthy
[14:21:09] <icinga-miraheze>	 RECOVERY - test1 MediaWiki Rendering on test1 is OK: HTTP OK: HTTP/1.1 200 OK - 18684 bytes in 0.440 second response time
[14:21:13] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[14:21:17] <icinga-miraheze>	 RECOVERY - cp3 Varnish Backends on cp3 is OK: All 6 backends are healthy
[14:22:28] <icinga-miraheze>	 RECOVERY - cp4 Varnish Backends on cp4 is OK: All 7 backends are healthy
[14:42:02] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 4 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[14:42:13] <icinga-miraheze>	 PROBLEM - mw3 MediaWiki Rendering on mw3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[14:42:31] <icinga-miraheze>	 PROBLEM - cp2 Stunnel Http for mw1 on cp2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[14:42:48] <icinga-miraheze>	 PROBLEM - test1 MediaWiki Rendering on test1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[14:43:01] <icinga-miraheze>	 PROBLEM - mw2 MediaWiki Rendering on mw2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[14:43:11] <icinga-miraheze>	 PROBLEM - cp4 Stunnel Http for mw1 on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[14:43:11] <icinga-miraheze>	 PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 5 datacenters are down: 107.191.126.23/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[14:44:00] <icinga-miraheze>	 PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 2 backends are down. mw2 mw3
[14:44:12] <icinga-miraheze>	 RECOVERY - mw3 MediaWiki Rendering on mw3 is OK: HTTP OK: HTTP/1.1 200 OK - 18685 bytes in 0.441 second response time
[14:44:27] <icinga-miraheze>	 RECOVERY - cp2 Stunnel Http for mw1 on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 15301 bytes in 0.423 second response time
[14:44:46] <icinga-miraheze>	 RECOVERY - test1 MediaWiki Rendering on test1 is OK: HTTP OK: HTTP/1.1 200 OK - 18684 bytes in 0.293 second response time
[14:44:56] <icinga-miraheze>	 RECOVERY - mw2 MediaWiki Rendering on mw2 is OK: HTTP OK: HTTP/1.1 200 OK - 18684 bytes in 0.214 second response time
[14:45:07] <icinga-miraheze>	 RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online
[14:45:07] <icinga-miraheze>	 RECOVERY - cp4 Stunnel Http for mw1 on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 15287 bytes in 0.011 second response time
[14:45:55] <icinga-miraheze>	 RECOVERY - cp3 Varnish Backends on cp3 is OK: All 6 backends are healthy
[14:46:00] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[15:45:27] <Not-5a0a>	 [02miraheze/dns] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://git.io/JvnYj
[15:45:29] <Not-5a0a>	 [02miraheze/dns] 07paladox 03a1ad5d2 - cloud1: Add ipv6 address
[15:45:30] <Not-5a0a>	 [02dns] 07paladox created branch 03paladox-patch-3 - 13https://git.io/vbQXl
[15:45:31] <Not-5a0a>	 [02dns] 07paladox opened pull request 03#120: cloud1: Add ipv6 address - 13https://git.io/JvnOe
[15:46:53] <Not-5a0a>	 [02dns] 07paladox closed pull request 03#120: cloud1: Add ipv6 address - 13https://git.io/JvnOe
[15:46:55] <Not-5a0a>	 [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvnOT
[15:46:56] <Not-5a0a>	 [02miraheze/dns] 07paladox 03cab5802 - cloud1: Add ipv6 address (#120)
[15:47:31] <Not-5a0a>	 [02miraheze/dns] 07paladox deleted branch 03paladox-patch-3
[15:47:33] <Not-5a0a>	 [02dns] 07paladox deleted branch 03paladox-patch-3 - 13https://git.io/vbQXl
[15:48:05] <travis-ci>	 miraheze/dns/master/cab5802 - paladox The build has errored. https://travis-ci.org/miraheze/dns/builds/647392679
[15:51:32] <RhinosF1>	 paladox|UKInEU: ^
[15:51:48] <RhinosF1>	 Just GeoLite
[15:51:54] * paladox|UKInEU currently busy, that will always fail now
[15:52:06] <paladox|UKInEU>	 we're going to have to remove travis
[15:52:31] <RhinosF1>	 paladox|UKInEU: that check will have to go
[15:52:40] <paladox|UKInEU>	 that's what i just said :)
[16:21:16] <Not-5a0a>	 [02miraheze/dns] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://git.io/Jvn3O
[16:21:18] <Not-5a0a>	 [02miraheze/dns] 07paladox 037bef2e7 - Add cp6.miraheze.org to dns
[16:21:19] <Not-5a0a>	 [02dns] 07paladox created branch 03paladox-patch-3 - 13https://git.io/vbQXl
[16:21:20] <Not-5a0a>	 [02dns] 07paladox opened pull request 03#121: Add cp6.miraheze.org to dns - 13https://git.io/Jvn33
[16:22:32] <travis-ci>	 miraheze/dns/paladox-patch-3/7bef2e7 - paladox The build has errored. https://travis-ci.org/miraheze/dns/builds/647407310
[16:38:20] <icinga-miraheze>	 PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 107.191.126.23/cpweb
[16:40:16] <icinga-miraheze>	 RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online
[16:42:14] <Not-5a0a>	 [02dns] 07paladox closed pull request 03#121: Add cp6.miraheze.org to dns - 13https://git.io/Jvn33
[16:42:15] <Not-5a0a>	 [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvnsm
[16:42:17] <Not-5a0a>	 [02miraheze/dns] 07paladox 036bf80f9 - Add cp6.miraheze.org to dns (#121)
[16:42:18] <Not-5a0a>	 [02miraheze/dns] 07paladox deleted branch 03paladox-patch-3
[16:42:20] <Not-5a0a>	 [02dns] 07paladox deleted branch 03paladox-patch-3 - 13https://git.io/vbQXl
[16:43:42] <travis-ci>	 miraheze/dns/master/6bf80f9 - paladox The build has errored. https://travis-ci.org/miraheze/dns/builds/647416622
[16:46:46] <icinga-miraheze>	 PROBLEM - browndust.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'browndust.wiki' expires in 15 day(s) (Sun 23 Feb 2020 04:42:50 PM GMT +0000).
[16:47:00] <Not-5a0a>	 [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvnsl
[16:47:02] <Not-5a0a>	 [02miraheze/ssl] 07MirahezeSSLBot 03109ecd5 - Bot: Update SSL cert for browndust.wiki
[16:52:45] <icinga-miraheze>	 RECOVERY - browndust.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'browndust.wiki' will expire on Thu 07 May 2020 03:46:54 PM GMT +0000.
[17:06:21] <hispano76>	 https://meta.miraheze.org/wiki/Special:CentralAuth/Ontzak Vandal Usurpation 
[17:06:23] <ZppixBot>	 [ Global account information for Ontzak - Miraheze Meta ] - meta.miraheze.org
[17:12:15] <RhinosF1>	 hispano76: that’s not Ontzak then?
[17:14:15] <RhinosF1>	 .wmca Ontzak
[17:14:15] <ZppixBot>	 https://meta.wikimedia.org/wiki/Special:CentralAuth/Ontzak
[17:20:51] <RhinosF1>	 hispano76: blocked and notified the real Ontzak
[17:21:06] <RhinosF1>	 JohnLewis, PuppyKun, SPF|Cloud: can you checkUser?
[17:21:28] <MH-Discord>	 <RhinosF1> @Stewards
[17:22:03] <RhinosF1>	 Voidwalker: see testwiki, hopefully ATS isnt back
[17:50:12] <Not-5a0a>	 [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvnZP
[17:50:14] <Not-5a0a>	 [02miraheze/dns] 07paladox 032e0b897 - cloud2: Add ipv6 address
[17:51:25] <travis-ci>	 miraheze/dns/master/2e0b897 - paladox The build has errored. https://travis-ci.org/miraheze/dns/builds/647443880
[17:53:23] * RhinosF1 thinks paladox|UKInEU is busy today
[17:53:40] <paladox|UKInEU>	 yup
[17:53:58] * RhinosF1 hopes it means new server prod soon!!
[17:54:12] * RhinosF1 is excieted and ready
[18:12:56] <Not-5a0a>	 [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvnnX
[18:12:57] <Not-5a0a>	 [02miraheze/dns] 07paladox 03b98320d - Add cloud.miraheze.org
[18:14:13] <RhinosF1>	 paladox|UKInEU: should we create dummy dbs to blacklist them in CW?
[18:14:32] <travis-ci>	 miraheze/dns/master/b98320d - paladox The build has errored. https://travis-ci.org/miraheze/dns/builds/647452128
[18:14:42] <paladox|UKInEU>	 i guess you could
[18:15:26] <RhinosF1>	 paladox|UKInEU: could I?
[18:15:47] <RhinosF1>	 I would be very concerned if I could
[18:15:49] <paladox|UKInEU>	 You wouldn't be able to, someone else with privs :)
[18:16:23] <RhinosF1>	 paladox|UKInEU: Who am I looking at or shall I find someone else?
[18:16:39] <paladox|UKInEU>	 you can find anyone, though i'm pretty busy with the cloud stuff.
[18:17:00] <RhinosF1>	 Zppix, Reception123: ^
[18:18:15] <RhinosF1>	 https://phabricator.miraheze.org/P259
[18:18:16] <ZppixBot>	 [ ✎ P259 New Server Setup ] - phabricator.miraheze.org
[18:18:59] <Not-5a0a>	 [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvnnd
[18:19:00] <Not-5a0a>	 [02miraheze/dns] 07paladox 030d45f92 - Update miraheze.org
[18:21:32] <travis-ci>	 miraheze/dns/master/0d45f92 - paladox The build has errored. https://travis-ci.org/miraheze/dns/builds/647454205
[19:15:04] <Zppix>	 RhinosF1:  i got it
[19:15:14] <Zppix>	 actually wait
[19:15:16] <Zppix>	 i dont know how
[19:17:48] <hispano76>	 paladox|UKInEU: excuse me, If you want to add a global interwiki, you do it in GitHub?
[19:18:06] <paladox|UKInEU>	 I thought that's done on meta?
[19:18:13] <RhinosF1>	 Zppix: sql.php --wiki=mhglobal
[19:18:24] <RhinosF1>	 CREATE database cloud1wiki;
[19:18:28] <RhinosF1>	 for example
[19:18:32] <RhinosF1>	 paladox|UKInEU: it is
[19:18:39] <RhinosF1>	 hispano76: no, ask a steward
[19:18:53] <Zppix>	 ok
[19:20:09] <Zppix>	 paladox|UKInEU:  are the new subdomains only cloud1 and cloud2?
[19:20:18] <paladox|UKInEU>	 yes
[19:20:20] <Zppix>	 k
[19:20:26] <Zppix>	 RhinosF1:  {{Done}}
[19:20:27] <hispano76>	  Good, because there is a colleague who has been requesting a global interwiki for some time and wanted to see if I could do it myself.   https://meta.miraheze.org/wiki/Community_noticeboard#Request_for_interwiki_of_simpleelectronics paladox|UKInEU  RhinosF1  
[19:20:30] <ZppixBot>	 [ Community noticeboard - Miraheze Meta ] - meta.miraheze.org
[19:20:39] <hispano76>	 user*
[19:21:08] <RhinosF1>	 paladox|UKInEU: won't mw[67]wiki and stuff be needed
[19:21:10] <RhinosF1>	 etc...
[19:21:15] <RhinosF1>	 Zppix: !log it
[19:21:28] <Zppix>	 !log created cloud1 and cloud2 dbs using sql.php
[19:21:29] <paladox|UKInEU>	 yes, i guess
[19:21:33] <RhinosF1>	 hispano76: someone'll see it
[19:21:37] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master
[19:21:47] <Zppix>	 ok ill do  that too
[19:21:55] <RhinosF1>	 Zppix: just do everything on https://phabricator.miraheze.org/P259
[19:21:56] <ZppixBot>	 [ ✎ P259 New Server Setup ] - phabricator.miraheze.org
[19:22:09] <Zppix>	 RhinosF1:  i only want to do what will actually affect anything
[19:22:15] <Zppix>	 otherwise its just a waste of db
[19:22:35] <hispano76>	 RhinosF1: I hope so.  :)
[19:22:43] <Zppix>	 !create mw[67]wiki using sql.php
[19:22:50] <Zppix>	 !log create mw[67]wiki using sql.php
[19:22:55] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master
[19:23:05] <RhinosF1>	 Zppix: in theory, if them wikis were created. they shouldn't be accessible anyway. So yes it prevents anyone creating an unusable wiki
[19:23:38] <RhinosF1>	 cloudwiki doesn't work either
[19:23:52] <RhinosF1>	 we need a better blacklist method than dummy dbs
[19:23:59] <Zppix>	 why wouldn't cloudwiki work?
[19:24:08] <RhinosF1>	 Zppix: try it
[19:24:13] <RhinosF1>	 cloud.miraheze.org
[19:24:29] <RhinosF1>	 click the link
[19:24:38] <Zppix>	 what are we doing on cloud.miraheze.org ?
[19:25:11] <RhinosF1>	 paladox|UKInEU: why's that refusing to connect?
[19:32:11] * hispano76 greetings
[19:37:39] <JohnLewis>	 cloud.mh.o isn't being used now
[19:40:23] <RhinosF1>	 JohnLewis: can you cu the user I blocked on test&meta
[20:04:07] <JohnLewis>	 RhinosF1: done
[20:05:54] <RhinosF1>	 JohnLewis: can you lock it
[20:06:01] <RhinosF1>	 As you have
[20:06:03] <JohnLewis>	 was already on it
[20:28:00] <RhinosF1>	 Good
[20:28:56] <RhinosF1>	 JohnLewis: confirmed to the Joaquinto farm then? I assume I’m fine to get that added to WMF files. What’s happening with the MH ban request?
[20:49:47] <JohnLewis>	 There's no technical no, and I do not know. I've not been asked to consider anything by Reception123 or Zppix who were the ones handling it
[20:50:20] <RhinosF1>	 JohnLewis: okay
[21:44:36] <RhinosF1>	 Voidwalker: while you’re at it, worth watching my tp to see who it is if i dont catch it
[21:53:15] <Voidwalker>	 watching
[22:32:27] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 2400:6180:0:d0::403:f001/cpweb
[22:34:31] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[22:59:05] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 2604:180:0:33b::2/cpweb, 2400:6180:0:d0::403:f001/cpweb
[23:03:08] <icinga-miraheze>	 PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 3 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[23:03:48] <icinga-miraheze>	 PROBLEM - cp3 Stunnel Http for mw3 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[23:03:56] <icinga-miraheze>	 PROBLEM - cp2 Stunnel Http for mw3 on cp2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[23:05:04] <icinga-miraheze>	 RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online
[23:05:57] <icinga-miraheze>	 RECOVERY - cp3 Stunnel Http for mw3 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15302 bytes in 7.354 second response time
[23:06:05] <icinga-miraheze>	 RECOVERY - cp2 Stunnel Http for mw3 on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 15302 bytes in 7.343 second response time
[23:07:27] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[23:13:05] <icinga-miraheze>	 PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 3 datacenters are down: 107.191.126.23/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb
[23:15:04] <icinga-miraheze>	 RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online
[23:24:20] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 2604:180:0:33b::2/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[23:25:06] <icinga-miraheze>	 PROBLEM - cp3 Stunnel Http for mw2 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[23:26:48] <icinga-miraheze>	 PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 4 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb
[23:27:08] <icinga-miraheze>	 RECOVERY - cp3 Stunnel Http for mw2 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15302 bytes in 3.419 second response time
[23:28:28] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[23:28:44] <icinga-miraheze>	 RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online
[23:53:21] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb
[23:54:23] <icinga-miraheze>	 PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 81.4.109.133/cpweb
[23:56:19] <icinga-miraheze>	 RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online
[23:57:21] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online