[00:02:01] RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 3759 MB (15% inode=93%); [00:30:20] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJE3K [00:30:21] [02miraheze/services] 07MirahezeSSLBot 03c614fe6 - BOT: Updating services config for wikis [01:55:14] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[disabled.life] [02:05:12] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:24:47] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJE8h [04:24:48] [02miraheze/puppet] 07Reception123 039f96906 - add lh3.googleusercontent.com to csp whitelist [04:33:49] PROBLEM - archive.stellurgists.wiki - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:33:52] PROBLEM - wiki.archlinuxfamily.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:33:55] PROBLEM - wiki.besuccess.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:33:58] PROBLEM - ta.gyaanipedia.co.in - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:33:58] PROBLEM - tradwiki.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:33:59] PROBLEM - www.permanentfuturelab.wiki - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:00] PROBLEM - wiki.titan.land - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:00] PROBLEM - wiki.velaan.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:02] PROBLEM - mg.gyaanipedia.co.in - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:02] PROBLEM - wiki.jafaron.net - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:03] PROBLEM - wiki.kourouklides.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:03] PROBLEM - wiki.animalrebellion.cz - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:05] PROBLEM - www.baharna.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:09] PROBLEM - kunwok.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:10] PROBLEM - pa.gyaanipedia.co.in - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:11] PROBLEM - wiki.earthmcbe.space - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:11] PROBLEM - wiki.shaazzz.ir - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:11] PROBLEM - wiki.kubixgames.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:11] PROBLEM - miraheze.ga - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:12] PROBLEM - wiki.thehall.xyz - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:12] PROBLEM - www.programming.red - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:14] PROBLEM - speleo.wiki - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:14] PROBLEM - museummiddelland.nl - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:15] PROBLEM - wiki.meeusen.net - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:16] PROBLEM - miraheze.ml - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:17] PROBLEM - www.museummiddelland.nl - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:17] PROBLEM - wiki.hanaoto.me - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:17] PROBLEM - wiki.rmbrk.sk - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:34:17] PROBLEM - www.reviwiki.info - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.89.160.142' [04:40:28] RECOVERY - archive.stellurgists.wiki - DNS on sslhost is OK: DNS OK: 0.110 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:34] RECOVERY - wiki.archlinuxfamily.org - DNS on sslhost is OK: DNS OK: 0.015 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:45] RECOVERY - ta.gyaanipedia.co.in - DNS on sslhost is OK: DNS OK: 0.012 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:47] RECOVERY - tradwiki.org - DNS on sslhost is OK: DNS OK: 0.019 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:48] RECOVERY - www.permanentfuturelab.wiki - DNS on sslhost is OK: DNS OK: 0.019 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:48] RECOVERY - kunwok.org - DNS on sslhost is OK: DNS OK: 0.057 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:51] RECOVERY - pa.gyaanipedia.co.in - DNS on sslhost is OK: DNS OK: 0.018 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:52] RECOVERY - wiki.titan.land - DNS on sslhost is OK: DNS OK: 0.035 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:52] RECOVERY - wiki.velaan.org - DNS on sslhost is OK: DNS OK: 0.067 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:52] RECOVERY - wiki.earthmcbe.space - DNS on sslhost is OK: DNS OK: 0.096 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:52] RECOVERY - wiki.shaazzz.ir - DNS on sslhost is OK: DNS OK: 0.010 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:54] RECOVERY - miraheze.ga - DNS on sslhost is OK: DNS OK: 0.010 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:54] RECOVERY - wiki.kubixgames.com - DNS on sslhost is OK: DNS OK: 0.058 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:54] RECOVERY - wiki.thehall.xyz - DNS on sslhost is OK: DNS OK: 0.021 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:54] RECOVERY - www.programming.red - DNS on sslhost is OK: DNS OK: 0.075 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:54] RECOVERY - wiki.besuccess.com - DNS on sslhost is OK: DNS OK: 0.142 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:55] RECOVERY - mg.gyaanipedia.co.in - DNS on sslhost is OK: DNS OK: 0.010 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:55] RECOVERY - wiki.jafaron.net - DNS on sslhost is OK: DNS OK: 0.067 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:56] RECOVERY - wiki.kourouklides.com - DNS on sslhost is OK: DNS OK: 0.040 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:57] RECOVERY - museummiddelland.nl - DNS on sslhost is OK: DNS OK: 0.133 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:40:57] RECOVERY - speleo.wiki - DNS on sslhost is OK: DNS OK: 0.156 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:41:15] RECOVERY - wiki.meeusen.net - DNS on sslhost is OK: DNS OK: 0.014 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [04:41:15] RECOVERY - www.reviwiki.info - DNS on sslhost is OK: DNS OK: 0.024 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [07:45:12] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJEz2 [07:45:13] [02miraheze/services] 07MirahezeSSLBot 0314e098b - BOT: Updating services config for wikis [08:20:22] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJEgy [08:20:23] [02miraheze/services] 07MirahezeSSLBot 033e2ea7c - BOT: Updating services config for wikis [13:32:30] !log add discord webhook for reverseunfavorablewikisanduserswiki [13:32:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:42:19] PROBLEM - gluster2 Puppet on gluster2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:21] PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:33] PROBLEM - cloud2 Puppet on cloud2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:39] PROBLEM - mw5 Puppet on mw5 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:41] PROBLEM - cp9 Puppet on cp9 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:41] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:41] PROBLEM - cloud1 Puppet on cloud1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:43] PROBLEM - phab1 Puppet on phab1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:45] PROBLEM - mail1 Puppet on mail1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:46] PROBLEM - cp6 Puppet on cp6 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:49] PROBLEM - rdb2 Puppet on rdb2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:49] PROBLEM - mw4 Puppet on mw4 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:50] PROBLEM - db12 Puppet on db12 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:51] PROBLEM - rdb1 Puppet on rdb1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:51] PROBLEM - jobrunner1 Puppet on jobrunner1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:53] PROBLEM - db7 Puppet on db7 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:43:01] PROBLEM - bacula2 Puppet on bacula2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:43:07] PROBLEM - ldap1 Puppet on ldap1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:43:10] PROBLEM - cloud3 Puppet on cloud3 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:43:14] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:43:15] PROBLEM - mon1 Puppet on mon1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:43:17] PROBLEM - services2 Puppet on services2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:43:27] PROBLEM - cp7 Puppet on cp7 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:43:35] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:43:41] PROBLEM - mw6 Puppet on mw6 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:43:41] PROBLEM - test2 Puppet on test2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:43:45] PROBLEM - gluster1 Puppet on gluster1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:43:45] PROBLEM - services1 Puppet on services1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:43:45] PROBLEM - mw7 Puppet on mw7 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:43:48] PROBLEM - puppet2 Puppet on puppet2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:43:49] my fault [13:44:01] PROBLEM - db11 Puppet on db11 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:44:05] PROBLEM - db13 Puppet on db13 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:44:08] Reception123: what you ruined? [13:44:28] RhinosF1: discord webhook, that yaml gets me every time [13:44:45] Reception123: ha! [13:51:35] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 3 seconds ago with 0 failures [13:51:44] RECOVERY - gluster1 Puppet on gluster1 is OK: OK: Puppet is currently enabled, last run 7 seconds ago with 0 failures [13:51:44] RECOVERY - services1 Puppet on services1 is OK: OK: Puppet is currently enabled, last run 13 seconds ago with 0 failures [13:51:48] RECOVERY - puppet2 Puppet on puppet2 is OK: OK: Puppet is currently enabled, last run 15 seconds ago with 0 failures [13:52:01] RECOVERY - db11 Puppet on db11 is OK: OK: Puppet is currently enabled, last run 34 seconds ago with 0 failures [13:52:05] RECOVERY - db13 Puppet on db13 is OK: OK: Puppet is currently enabled, last run 38 seconds ago with 0 failures [13:52:19] RECOVERY - gluster2 Puppet on gluster2 is OK: OK: Puppet is currently enabled, last run 39 seconds ago with 0 failures [13:52:21] RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 58 seconds ago with 0 failures [13:52:33] RECOVERY - cloud2 Puppet on cloud2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:52:40] RECOVERY - mw5 Puppet on mw5 is OK: OK: Puppet is currently enabled, last run 38 seconds ago with 0 failures [13:52:47] RECOVERY - phab1 Puppet on phab1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:52:48] RECOVERY - rdb2 Puppet on rdb2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:52:49] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:52:50] RECOVERY - mail1 Puppet on mail1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:52:51] RECOVERY - cloud1 Puppet on cloud1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:52:52] RECOVERY - cp6 Puppet on cp6 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:52:54] RECOVERY - db12 Puppet on db12 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:52:55] RECOVERY - cp9 Puppet on cp9 is OK: OK: Puppet is currently enabled, last run 28 seconds ago with 0 failures [13:52:55] RECOVERY - mw4 Puppet on mw4 is OK: OK: Puppet is currently enabled, last run 53 seconds ago with 0 failures [13:52:58] RECOVERY - rdb1 Puppet on rdb1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:53:00] RECOVERY - jobrunner1 Puppet on jobrunner1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:53:01] RECOVERY - db7 Puppet on db7 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:53:05] RECOVERY - bacula2 Puppet on bacula2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:53:08] RECOVERY - ldap1 Puppet on ldap1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:53:18] RECOVERY - mon1 Puppet on mon1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:53:20] RECOVERY - cloud3 Puppet on cloud3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:53:25] RECOVERY - services2 Puppet on services2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:53:27] RECOVERY - cp7 Puppet on cp7 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:53:41] RECOVERY - mw6 Puppet on mw6 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:53:41] RECOVERY - test2 Puppet on test2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:53:44] RECOVERY - mw7 Puppet on mw7 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:55:15] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:08:25] * hispano76 greetings [14:31:59] PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2636 MB (10% inode=93%); [14:33:58] RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 2658 MB (11% inode=93%); [14:36:00] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03RhinosF1-patch-1 [+0/-0/±1] 13https://git.io/JJENC [14:36:02] [02miraheze/mw-config] 07RhinosF1 03a4b5fa8 - block indexing of NS_SPECIAL resolves T5978 [14:36:03] [02mw-config] 07RhinosF1 created branch 03RhinosF1-patch-1 - 13https://git.io/vbvb3 [14:36:05] [02mw-config] 07RhinosF1 opened pull request 03#3191: block indexing of NS_SPECIAL - 13https://git.io/JJENW [14:36:23] Reception123: does ^ look sane [14:36:27] gonna check test2 [14:36:38] RhinosF1: it seems fine to me, but yes please do test2 first [14:40:11] rhinos@test2:/srv/mediawiki/config$ sudo -u www-data git cherry-pick a4b5fa8 [14:40:11] error: insufficient permission for adding an object to repository database .git/objects [14:40:11] error: failed to write commit object [14:40:11] fatal: cherry-pick failed [14:40:13] Reception123: ^ [14:40:37] why does www-data not have that [14:41:08] Reception123: someone's modified LS.php [14:41:41] PROBLEM - test2 Puppet on test2 is WARNING: WARNING: Puppet is currently disabled, message: RhinosF1, last run 9 minutes ago with 0 failures [14:41:59] no gits just being weird [14:42:52] now done [14:42:54] PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2647 MB (10% inode=93%); [14:44:16] $log cherry-picked a4b5fa8 to test2 [14:46:02] Reception123: LGTM, can you check you see noindex on special pages [14:47:41] I check 3 special pages and it seems to work [14:56:22] hi Cocopuff2018 [14:56:32] hi [14:56:48] * RhinosF1 is going to merge the search console fix [14:56:59] it looks sane, works for me and passes CI [14:57:06] [02mw-config] 07RhinosF1 closed pull request 03#3191: block indexing of NS_SPECIAL - 13https://git.io/JJENW [14:57:07] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJExL [14:57:09] [02miraheze/mw-config] 07RhinosF1 03b048932 - block indexing of NS_SPECIAL (#3191) resolves T5978 [14:57:12] [02mw-config] 07RhinosF1 deleted branch 03RhinosF1-patch-1 - 13https://git.io/vbvb3 [14:57:13] [02miraheze/mw-config] 07RhinosF1 deleted branch 03RhinosF1-patch-1 [14:59:41] RECOVERY - test2 Puppet on test2 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:02:13] Cocopuff2018: that's merged now and I've kicked off a validation job but it might take them a while to actually pick it up [15:06:24] RhinosF1: whoops sorry, back now [15:06:43] Reception123: just sense check a few wikis [15:06:43] and how am I supposed to "see" it? [15:06:51] Reception123: use developer console [15:06:58] In the html you'll see the head tag [15:06:58] ah [15:07:07] And in it there's a robots meta tag [15:07:29] Might be good to check something like hispanowiki as that had a rule on before [15:07:36] And check it's pages look fine [15:10:00] hmm not exactly sure where to find [15:15:37] Reception123: in the head of the raw html [15:16:07] ah [15:16:45] RhinosF1: I mean all I can see is at the bottom [15:17:00] Reception123: good [15:17:19] That's what you should see [15:17:31] ok then [15:19:29] Reception123: check the template namespace on Hispano wiki as well [15:19:55] As that had a custom noindex rule on [15:20:00] Make sure I didn't break that [15:20:44] RhinosF1: I'm afraid you could've [15:20:44] https://hispano.miraheze.org/wiki/Plantilla:Art%C3%ADculo_pa%C3%ADs [15:20:45] [ Plantilla:Artículo país - Hispano ] - hispano.miraheze.org [15:20:53] I don't see anything here, but I do see it when I click edit [15:21:03] however I have to go in about 5 mins so I can't do much [15:21:12] but something does seem wrong [15:21:18] Reception123: I'll check when at my pc [15:21:40] RhinosF1: ok, I'm not sure how that's related but it's definitely quite weird [15:21:54] No no no [15:22:04] That's the template code [15:22:11] Just check there's a noindex tag [15:22:49] PROBLEM - rdb2 APT on rdb2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [15:23:37] Uh oh [15:24:16] I think something went down [15:24:45] RECOVERY - rdb2 APT on rdb2 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [15:25:08] Paging Reception123 JohnLewis paladox SPF|Cloud Zppix [15:25:24] Whatever just happened caused a 502 [15:25:33] PROBLEM - cp7 Stunnel Http for mw7 on cp7 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 344 bytes in 0.003 second response time [15:26:00] PROBLEM - cp7 Varnish Backends on cp7 is CRITICAL: 1 backends are down. mw7 [15:26:41] PROBLEM - wiki.meeusen.net - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:43] PROBLEM - www.schulwiki.de - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:43] PROBLEM - wiki.lbcomms.co.za - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:44] PROBLEM - www.hinduwiki.online - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:44] PROBLEM - orain.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:45] PROBLEM - factorio-mods.tk - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:46] PROBLEM - wiki.gesamtschule-nordkirchen.de - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:48] PROBLEM - pol.wiki - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:48] PROBLEM - wiki.whentheycry.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:48] PROBLEM - rivworld.wiki - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:48] PROBLEM - wiki.galactee.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:53] PROBLEM - wiki.cdntennis.ca - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:54] PROBLEM - wiki.iufs.jp - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:54] PROBLEM - bh.gyaanipedia.co.in - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:54] PROBLEM - wiki.glaucuslinux.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:54] PROBLEM - index.pdcommunity.ir - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:54] PROBLEM - www.lab612.at - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:57] PROBLEM - wiki.worldsofweary.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:57] PROBLEM - cyberlaw.ccdcoe.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:26:58] PROBLEM - franchise.franchising.org.ua - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:01] PROBLEM - minetestserver-cc.ddns.net - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:04] PROBLEM - wiki.serwerwanilia.pl - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:04] PROBLEM - dariawiki.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:05] PROBLEM - wiki.8b8t.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:06] PROBLEM - wiki.arcolinuxclub.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:06] PROBLEM - wiki.warframestat.us - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:09] PROBLEM - wiki.hrznstudio.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:09] PROBLEM - duepedia.uk.to - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:09] PROBLEM - www.eerstelijnszones.be - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:09] PROBLEM - wiki.s23.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:09] PROBLEM - public.revi.wiki - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:10] PROBLEM - wiki.tallguysfree.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:10] PROBLEM - www.evanswiki.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:11] PROBLEM - studynotekr.miraheze.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:11] PROBLEM - www.netazar.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:12] PROBLEM - wiki.climatechange.ai - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:12] PROBLEM - wiki.campaign-labour.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210,51.89.160.142' [15:27:29] RECOVERY - cp7 Stunnel Http for mw7 on cp7 is OK: HTTP OK: HTTP/1.1 200 OK - 15595 bytes in 0.003 second response time [15:27:58] RECOVERY - cp7 Varnish Backends on cp7 is OK: All 7 backends are healthy [15:33:31] RECOVERY - wiki.lbcomms.co.za - DNS on sslhost is OK: DNS OK: 0.311 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:31] RECOVERY - www.schulwiki.de - DNS on sslhost is OK: DNS OK: 0.034 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:34] RECOVERY - www.hinduwiki.online - DNS on sslhost is OK: DNS OK: 0.012 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:34] RECOVERY - orain.org - DNS on sslhost is OK: DNS OK: 0.052 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:36] RECOVERY - factorio-mods.tk - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:37] RECOVERY - wiki.gesamtschule-nordkirchen.de - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:38] RECOVERY - wiki.worldsofweary.com - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:38] RECOVERY - cyberlaw.ccdcoe.org - DNS on sslhost is OK: DNS OK: 0.043 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:40] RECOVERY - wiki.meeusen.net - DNS on sslhost is OK: DNS OK: 0.010 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:41] RECOVERY - franchise.franchising.org.ua - DNS on sslhost is OK: DNS OK: 0.009 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:42] RECOVERY - pol.wiki - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:42] RECOVERY - wiki.whentheycry.org - DNS on sslhost is OK: DNS OK: 0.013 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:42] RECOVERY - rivworld.wiki - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:43] RECOVERY - wiki.galactee.org - DNS on sslhost is OK: DNS OK: 0.016 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:47] RECOVERY - minetestserver-cc.ddns.net - DNS on sslhost is OK: DNS OK: 0.010 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:49] RECOVERY - wiki.cdntennis.ca - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:51] RECOVERY - wiki.iufs.jp - DNS on sslhost is OK: DNS OK: 0.012 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:51] RECOVERY - bh.gyaanipedia.co.in - DNS on sslhost is OK: DNS OK: 0.034 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:51] RECOVERY - wiki.glaucuslinux.org - DNS on sslhost is OK: DNS OK: 0.041 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:51] RECOVERY - index.pdcommunity.ir - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:52] RECOVERY - www.lab612.at - DNS on sslhost is OK: DNS OK: 0.010 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:52] RECOVERY - wiki.serwerwanilia.pl - DNS on sslhost is OK: DNS OK: 0.009 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:53] RECOVERY - dariawiki.org - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:55] RECOVERY - wiki.8b8t.com - DNS on sslhost is OK: DNS OK: 0.016 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:57] RECOVERY - wiki.arcolinuxclub.org - DNS on sslhost is OK: DNS OK: 0.010 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:33:57] RECOVERY - wiki.warframestat.us - DNS on sslhost is OK: DNS OK: 0.175 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:34:01] RECOVERY - wiki.hrznstudio.com - DNS on sslhost is OK: DNS OK: 0.022 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:34:01] RECOVERY - duepedia.uk.to - DNS on sslhost is OK: DNS OK: 0.145 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:34:01] RECOVERY - www.eerstelijnszones.be - DNS on sslhost is OK: DNS OK: 0.014 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:34:02] RECOVERY - wiki.s23.org - DNS on sslhost is OK: DNS OK: 0.107 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:34:03] RECOVERY - public.revi.wiki - DNS on sslhost is OK: DNS OK: 0.014 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:34:03] RECOVERY - wiki.tallguysfree.com - DNS on sslhost is OK: DNS OK: 0.018 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:34:05] RECOVERY - www.evanswiki.org - DNS on sslhost is OK: DNS OK: 0.017 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:34:05] RECOVERY - studynotekr.miraheze.org - DNS on sslhost is OK: DNS OK: 0.012 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:34:05] RECOVERY - www.netazar.org - DNS on sslhost is OK: DNS OK: 0.069 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:34:08] RECOVERY - wiki.campaign-labour.org - DNS on sslhost is OK: DNS OK: 0.012 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:34:08] RECOVERY - wiki.climatechange.ai - DNS on sslhost is OK: DNS OK: 0.054 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [15:42:49] [02puppet] 07RhinosF1 commented on commit 035cc6209b339deec804cfc23f9740fc8efce72f69 - 13https://git.io/JJEh0 [17:17:22] !log add pcpartswiki discord webhook [17:17:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:25:07] PROBLEM - db7 Check MariaDB Replication on db7 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 162s [17:45:25] Hello Waggi3! If you have any questions, feel free to ask and someone should answer soon. [17:48:27] .kb Waggi3 impersonation [17:48:28] Please wait... [18:10:21] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJutl [18:10:22] [02miraheze/services] 07MirahezeSSLBot 033b5429c - BOT: Updating services config for wikis [18:27:01] Jobs seems kinda slow right now, see also GlobalRenameProgress (not stuck, just slow) [18:28:15] Yes, i saw any delays of page moves after rename, about 2 weeks ago. [18:28:44] 7-12 hours i think [18:28:57] > Jobs seems kinda slow right now, see also GlobalRenameProgress (not stuck, just slow) guessing the global renames run on the jobrunner1 server as well [18:37:42] Yes [18:37:47] Rename is a job [18:38:23] Anything that doesn't happen on page load, probably goes via the JobQueue [18:59:44] Ah, that's really helpful content, thanks, @RhinosF1. No wonder it is so busy...basically, any log action, unless it's creating a page (or maybe deleting a page?). [19:07:50] Deleting a page happens on load [19:08:11] If as soon as you click a button and the page loads, it happens. Then it's not the JobQueue [19:08:28] Anything that isn't on load and happens in the background probably is [20:13:23] thanks! [21:02:46] PROBLEM - mw5 Current Load on mw5 is WARNING: WARNING - load average: 7.93, 6.17, 4.40 [21:04:46] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 2.62, 4.72, 4.08 [22:14:47] anyone else having problems reaching meta? [22:15:01] y [22:15:24] PROBLEM - wiki.ciptamedia.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:25] PROBLEM - www.iceria.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:25] PROBLEM - madzebrascience.wiki - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:25] PROBLEM - hif.gyaanipedia.co.in - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:25] PROBLEM - wiki.lbcomms.co.za - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:26] PROBLEM - theatlas.pw - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:26] PROBLEM - www.schulwiki.de - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:26] PROBLEM - factorio-mods.tk - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:27] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 11.63, 8.61, 4.34 [22:15:27] PROBLEM - www.allthetropes.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:28] PROBLEM - pol.wiki - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:28] PROBLEM - www.portalsofphereon.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:29] PROBLEM - wiki.whentheycry.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:29] PROBLEM - garrettcountyguide.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:38] Now okay [22:15:41] PROBLEM - dariawiki.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:44] PROBLEM - wiki.8b8t.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:45] PROBLEM - wiki.hrznstudio.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:45] PROBLEM - duepedia.uk.to - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:45] PROBLEM - wiki.tallguysfree.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:46] PROBLEM - mon1 Current Load on mon1 is CRITICAL: CRITICAL - load average: 9.97, 6.79, 3.62 [22:15:46] PROBLEM - www.eerstelijnszones.be - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:49] PROBLEM - wiki.teamrelectric.ca - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:49] PROBLEM - www.evanswiki.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:49] PROBLEM - wiki.s23.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:49] PROBLEM - studynotekr.miraheze.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:49] PROBLEM - www.netazar.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:50] PROBLEM - public.revi.wiki - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:50] PROBLEM - wiki.campaign-labour.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:50] PROBLEM - wiki.climatechange.ai - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:51] PROBLEM - wiki.cloudytheology.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:15:51] PROBLEM - wiki.twilightsignal.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:02] PROBLEM - wiki.joust.ro - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:03] PROBLEM - www.tradwiki.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:03] PROBLEM - yellowiki.xyz - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:04] PROBLEM - documentation.aqfer.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:04] PROBLEM - wiki.apap04.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:05] PROBLEM - wiki.warfra.ml - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:05] PROBLEM - wiki.thesimswiki.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:06] PROBLEM - pastport.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:06] PROBLEM - vitriol.spoon.army - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:06] I'm looking at over 200 criticals in icinga right now [22:16:07] PROBLEM - savage-wiki.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:07] PROBLEM - enc.for.uz - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:08] PROBLEM - reviwiki.info - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:08] PROBLEM - sahitya.shaunak.in - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:35] PROBLEM - monarchists.wiki - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:35] PROBLEM - permanentfuturelab.wiki - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:36] Voidwalker: this is why I hate that checks [22:16:36] PROBLEM - www.wikimicrofinanza.it - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:37] PROBLEM - www.thesimswiki.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:37] PROBLEM - allthetropes.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:37] PROBLEM - hellointernet.miraheze.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:37] PROBLEM - stablestate.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:37] PROBLEM - wiki.openhatch.org - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:37] PROBLEM - wiki.dobots.nl - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:38] PROBLEM - wiki.invinciblewellbeing.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:38] PROBLEM - wiki.archlinuxfamily.de - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:39] PROBLEM - www.publictestwiki.com - DNS on sslhost is CRITICAL: DNS CRITICAL - expected '2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142' but got '2001:41d0:800:1056::2,51.77.107.210' [22:16:42] Alert to Miraheze Staff: It looks like the icinga-miraheze bot has stopped! Ping !sre. [22:16:43] https://meta.miraheze.org is 03UP03 [22:16:43] [ Meta ] - meta.miraheze.org [22:16:45] https://icinga.miraheze.org is 03UP03 [22:16:46] [ Icinga Web 2 Login ] - icinga.miraheze.org [22:16:48] fuck [22:18:04] yeah, but it happened at the same time I had problems accessing the site, so I'm assuming it's related, and definitely still a problem [22:18:10] Voidwalker: Load surge on a few servers [22:18:22] mon1 and misc1 [22:20:10] Voidwalker: still an issue? [22:20:55] no access is fine, but possibly still up and down [22:21:10] Voidwalker: yeah I see it now [22:21:29] PROBLEM - mon1 Current Load on mon1 is WARNING: WARNING - load average: 1.21, 3.55, 3.24 [22:22:06] RECOVERY - www.allthetropes.org - DNS on sslhost is OK: DNS OK: 0.117 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:06] RECOVERY - www.portalsofphereon.com - DNS on sslhost is OK: DNS OK: 0.018 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:14] RECOVERY - theatlas.pw - DNS on sslhost is OK: DNS OK: 0.023 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:15] RECOVERY - wiki.lbcomms.co.za - DNS on sslhost is OK: DNS OK: 0.012 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:15] RECOVERY - www.schulwiki.de - DNS on sslhost is OK: DNS OK: 0.019 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:15] RECOVERY - chouverse.ncps.xyz - DNS on sslhost is OK: DNS OK: 0.093 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:16] RECOVERY - garrettcountyguide.com - DNS on sslhost is OK: DNS OK: 0.165 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:16] RECOVERY - www.iceria.org - DNS on sslhost is OK: DNS OK: 0.107 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:16] RECOVERY - factorio-mods.tk - DNS on sslhost is OK: DNS OK: 0.087 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:16] RECOVERY - madzebrascience.wiki - DNS on sslhost is OK: DNS OK: 0.095 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:17] RECOVERY - kkutu.wiki - DNS on sslhost is OK: DNS OK: 0.052 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:19] RECOVERY - www.hinduwiki.online - DNS on sslhost is OK: DNS OK: 0.012 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:19] RECOVERY - orain.org - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:20] RECOVERY - cyberlaw.ccdcoe.org - DNS on sslhost is OK: DNS OK: 0.013 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:21] RECOVERY - wiki.worldsofweary.com - DNS on sslhost is OK: DNS OK: 0.010 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:21] RECOVERY - wiki.gesamtschule-nordkirchen.de - DNS on sslhost is OK: DNS OK: 0.063 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:21] RECOVERY - pol.wiki - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:21] RECOVERY - wiki.whentheycry.org - DNS on sslhost is OK: DNS OK: 0.060 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:21] RECOVERY - wiki.mxlinuxusers.de - DNS on sslhost is OK: DNS OK: 0.057 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:35] RECOVERY - wiki.autocountsoft.com - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:35] RECOVERY - wiki.8b8t.com - DNS on sslhost is OK: DNS OK: 0.014 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:37] RECOVERY - es.gyaanipedia.co.in - DNS on sslhost is OK: DNS OK: 0.012 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:37] RECOVERY - unmade.miraheze.org - DNS on sslhost is OK: DNS OK: 0.030 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:37] RECOVERY - wiki.twilightsignal.com - DNS on sslhost is OK: DNS OK: 0.068 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:37] RECOVERY - poserdazfreebies.orain.org - DNS on sslhost is OK: DNS OK: 0.113 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:38] RECOVERY - cnt.miraheze.org - DNS on sslhost is OK: DNS OK: 0.187 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:38] RECOVERY - wiki.hrznstudio.com - DNS on sslhost is OK: DNS OK: 0.111 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:38] RECOVERY - duepedia.uk.to - DNS on sslhost is OK: DNS OK: 0.071 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:39] RECOVERY - wiki.macc.nyc - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:39] RECOVERY - wiki.valentinaproject.org - DNS on sslhost is OK: DNS OK: 0.016 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:40] RECOVERY - www.eerstelijnszones.be - DNS on sslhost is OK: DNS OK: 0.010 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:40] RECOVERY - wiki.tallguysfree.com - DNS on sslhost is OK: DNS OK: 0.052 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:41] RECOVERY - wiki.casadocarvalho.net - DNS on sslhost is OK: DNS OK: 0.013 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:41] RECOVERY - wiki.joust.ro - DNS on sslhost is OK: DNS OK: 0.015 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:42] RECOVERY - encyclopediaofastrobiology.org - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:53] RECOVERY - vitriol.spoon.army - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:54] RECOVERY - baharna.org - DNS on sslhost is OK: DNS OK: 0.010 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:55] RECOVERY - wiki.thesimswiki.com - DNS on sslhost is OK: DNS OK: 0.010 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:55] RECOVERY - pastport.org - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:56] RECOVERY - savage-wiki.com - DNS on sslhost is OK: DNS OK: 0.012 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:56] RECOVERY - www.lab612.at - DNS on sslhost is OK: DNS OK: 0.023 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:57] RECOVERY - www.tradwiki.org - DNS on sslhost is OK: DNS OK: 0.022 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:57] RECOVERY - sahitya.shaunak.in - DNS on sslhost is OK: DNS OK: 0.027 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:59] RECOVERY - enc.for.uz - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:22:59] RECOVERY - reviwiki.info - DNS on sslhost is OK: DNS OK: 0.051 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:16] RECOVERY - monarchists.wiki - DNS on sslhost is OK: DNS OK: 0.021 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:17] RECOVERY - infectowiki.com - DNS on sslhost is OK: DNS OK: 0.010 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:17] RECOVERY - www.wikimicrofinanza.it - DNS on sslhost is OK: DNS OK: 0.118 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:17] RECOVERY - www.thesimswiki.com - DNS on sslhost is OK: DNS OK: 0.054 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:18] RECOVERY - spiral.wiki - DNS on sslhost is OK: DNS OK: 0.019 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:19] RECOVERY - miraheze.tk - DNS on sslhost is OK: DNS OK: 0.021 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:20] RECOVERY - wiki.bullshit.systems - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:20] RECOVERY - permanentfuturelab.wiki - DNS on sslhost is OK: DNS OK: 0.013 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:21] RECOVERY - wiki.openhatch.org - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:21] RECOVERY - wiki.mikrodev.com - DNS on sslhost is OK: DNS OK: 0.017 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:21] RECOVERY - piwik.miraheze.org - DNS on sslhost is OK: DNS OK: 0.080 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:21] RECOVERY - wiki.exnihilolinux.org - DNS on sslhost is OK: DNS OK: 0.085 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:21] RECOVERY - www.dariawiki.org - DNS on sslhost is OK: DNS OK: 0.090 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:22] RECOVERY - studentwiki.ddns.net - DNS on sslhost is OK: DNS OK: 0.011 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:22] RECOVERY - dreamsit.com.br - DNS on sslhost is OK: DNS OK: 0.032 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:23] RECOVERY - wikiverte.pl - DNS on sslhost is OK: DNS OK: 0.066 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:35] RECOVERY - wiki.mobilityengineer.com - DNS on sslhost is OK: DNS OK: 0.022 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:35] RECOVERY - munwiki.info - DNS on sslhost is OK: DNS OK: 0.014 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:36] RECOVERY - www.publictestwiki.com - DNS on sslhost is OK: DNS OK: 0.010 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:36] RECOVERY - vault.aics.cf - DNS on sslhost is OK: DNS OK: 0.013 seconds response time. sslhost returns 2001:41d0:800:1056::2,2001:41d0:800:105a::10,51.77.107.210,51.89.160.142 [22:23:37] Alert to Miraheze Staff: It looks like the icinga-miraheze bot has stopped! Ping !sre. [22:23:38] https://meta.miraheze.org is 03UP03 [22:23:39] [ Meta ] - meta.miraheze.org [22:23:39] https://icinga.miraheze.org is 03UP03 [22:23:40] [ Icinga Web 2 Login ] - icinga.miraheze.org [22:25:17] RECOVERY - mon1 Current Load on mon1 is OK: OK - load average: 1.24, 2.85, 3.06 [22:25:44] PROBLEM - misc1 Current Load on misc1 is CRITICAL: CRITICAL - load average: 2.18, 2.50, 1.41 [22:27:43] RECOVERY - misc1 Current Load on misc1 is OK: OK - load average: 0.32, 1.70, 1.25 [22:28:53] [02mw-config] 07Amanda-Catherine opened pull request 03#3192: Change worldtrainwiki license to Creative Commons 3.0 T5983 - 13https://git.io/JJuWy [22:29:06] RECOVERY - db7 Check MariaDB Replication on db7 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 0s [22:30:24] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJuW7 [22:30:26] [02miraheze/puppet] 07paladox 0392a851d - Nutcracker: Switch to rdb1 [22:31:33] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 14.58, 17.94, 23.85 [22:31:54] paladox: ^ [22:34:23] Not sure why you pinged need for that, I’m already aware of the high load :) [22:39:21] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 8.78, 13.08, 19.33 [23:00:23] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJu8m [23:00:25] [02miraheze/services] 07MirahezeSSLBot 0357493a4 - BOT: Updating services config for wikis [23:50:15] [02puppet] 07RhinosF1 opened pull request 03#1460: add a throttle for BingBot - 13https://git.io/JJu4j [23:50:30] [02puppet] 07RhinosF1 edited pull request 03#1460: add a throttle for BingBot - 13https://git.io/JJu4j [23:50:35] paladox: ^ [23:54:11] RhinosF1: see -staff [23:54:54] [02puppet] 07paladox closed pull request 03#1460: add a throttle for BingBot - 13https://git.io/JJu4j [23:54:55] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJuBn [23:54:57] [02miraheze/puppet] 07RhinosF1 034af2f58 - add a throttle for BingBot (#1460) It's hitting at an insane rate of more than one query per second on a single mw