[00:58:57] <Hispano76>	 https://prnt.sc/ncgs95 Special:MassEditRegex mediateca.miraheze.org
[00:58:58] <ZppixBot>	 Title: [ Screenshot by Lightshot ] - prnt.sc
[01:00:23] <Hispano76>	 Um, it looks like it was a false positive. 
[01:03:31] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[01:04:05] <icinga-miraheze>	 PROBLEM - cp3 HTTPS on cp3 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Backend fetch failed - 3871 bytes in 0.733 second response time
[01:04:09] <icinga-miraheze>	 PROBLEM - mw1 HTTPS on mw1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[01:04:35] <icinga-miraheze>	 PROBLEM - cp4 Varnish Backends on cp4 is CRITICAL: 3 backends are down. mw1 mw2 mw3
[01:04:43] <icinga-miraheze>	 PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 3 backends are down. mw1 mw2 mw3
[01:05:11] <icinga-miraheze>	 PROBLEM - cp2 Varnish Backends on cp2 is CRITICAL: 3 backends are down. mw1 mw2 mw3
[01:05:15] <icinga-miraheze>	 PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[01:05:40] <icinga-miraheze>	 PROBLEM - cp2 HTTP 4xx/5xx ERROR Rate on cp2 is CRITICAL: CRITICAL - NGINX Error Rate is 72%
[01:06:01] <icinga-miraheze>	 PROBLEM - mw3 HTTPS on mw3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[01:06:08] <icinga-miraheze>	 PROBLEM - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is CRITICAL: CRITICAL - NGINX Error Rate is 77%
[01:06:17] <icinga-miraheze>	 PROBLEM - mw2 HTTPS on mw2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[01:06:38] <icinga-miraheze>	 PROBLEM - cp4 HTTPS on cp4 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Backend fetch failed - 3858 bytes in 0.007 second response time
[01:06:42] <icinga-miraheze>	 PROBLEM - cp4 HTTP 4xx/5xx ERROR Rate on cp4 is CRITICAL: CRITICAL - NGINX Error Rate is 88%
[01:07:07] <icinga-miraheze>	 PROBLEM - cp2 HTTPS on cp2 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Backend fetch failed - 3860 bytes in 0.403 second response time
[01:07:54] <icinga-miraheze>	 PROBLEM - mw3 Puppet on mw3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki config]
[01:07:59] <icinga-miraheze>	 RECOVERY - mw3 HTTPS on mw3 is OK: HTTP OK: HTTP/1.1 200 OK - 23221 bytes in 0.008 second response time
[01:08:01] <icinga-miraheze>	 RECOVERY - cp3 HTTPS on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 24074 bytes in 1.390 second response time
[01:08:08] <icinga-miraheze>	 RECOVERY - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is OK: OK - NGINX Error Rate is 35%
[01:08:11] <icinga-miraheze>	 RECOVERY - mw2 HTTPS on mw2 is OK: HTTP OK: HTTP/1.1 200 OK - 23221 bytes in 0.008 second response time
[01:08:13] <icinga-miraheze>	 RECOVERY - mw1 HTTPS on mw1 is OK: HTTP OK: HTTP/1.1 200 OK - 23221 bytes in 0.011 second response time
[01:08:24] <icinga-miraheze>	 PROBLEM - mw1 Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki config],Exec[git_pull_MediaWiki core]
[01:08:35] <icinga-miraheze>	 RECOVERY - cp4 Varnish Backends on cp4 is OK: All 5 backends are healthy
[01:08:38] <icinga-miraheze>	 RECOVERY - cp4 HTTPS on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 24051 bytes in 0.012 second response time
[01:08:42] <icinga-miraheze>	 RECOVERY - cp4 HTTP 4xx/5xx ERROR Rate on cp4 is OK: OK - NGINX Error Rate is 3%
[01:08:43] <icinga-miraheze>	 RECOVERY - cp3 Varnish Backends on cp3 is OK: All 5 backends are healthy
[01:09:07] <icinga-miraheze>	 RECOVERY - cp2 HTTPS on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 24085 bytes in 0.504 second response time
[01:09:10] <icinga-miraheze>	 RECOVERY - cp2 Varnish Backends on cp2 is OK: All 5 backends are healthy
[01:09:15] <icinga-miraheze>	 RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online
[01:09:31] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[01:09:40] <icinga-miraheze>	 RECOVERY - cp2 HTTP 4xx/5xx ERROR Rate on cp2 is OK: OK - NGINX Error Rate is 3%
[01:10:49] <icinga-miraheze>	 PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_puppet]
[01:11:41] <ZppixBot>	 Hello Guest68778! If you have any questions feel free to ask and someone should answer soon.
[01:13:55] <icinga-miraheze>	 RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 45 seconds ago with 0 failures
[01:14:18] <icinga-miraheze>	 RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[01:14:49] <icinga-miraheze>	 RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[01:19:43] <icinga-miraheze>	 PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki config]
[01:23:10] <icinga-miraheze>	 PROBLEM - cp2 Varnish Backends on cp2 is CRITICAL: 3 backends are down. mw1 mw2 mw3
[01:23:13] <icinga-miraheze>	 PROBLEM - cp2 HTTPS on cp2 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Backend fetch failed - 3860 bytes in 0.404 second response time
[01:23:15] <icinga-miraheze>	 PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[01:23:31] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[01:23:44] <icinga-miraheze>	 RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[01:24:01] <icinga-miraheze>	 PROBLEM - cp3 HTTPS on cp3 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Backend fetch failed - 3871 bytes in 0.732 second response time
[01:24:13] <icinga-miraheze>	 PROBLEM - mw3 HTTPS on mw3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[01:24:15] <icinga-miraheze>	 PROBLEM - mw1 HTTPS on mw1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[01:24:17] <icinga-miraheze>	 PROBLEM - mw2 HTTPS on mw2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[01:24:35] <icinga-miraheze>	 PROBLEM - cp4 Varnish Backends on cp4 is CRITICAL: 3 backends are down. mw1 mw2 mw3
[01:24:42] <icinga-miraheze>	 PROBLEM - cp4 HTTP 4xx/5xx ERROR Rate on cp4 is CRITICAL: CRITICAL - NGINX Error Rate is 84%
[01:24:43] <icinga-miraheze>	 PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 3 backends are down. mw1 mw2 mw3
[01:26:11] <icinga-miraheze>	 RECOVERY - mw2 HTTPS on mw2 is OK: HTTP OK: HTTP/1.1 200 OK - 23221 bytes in 0.007 second response time
[01:26:12] <icinga-miraheze>	 RECOVERY - mw3 HTTPS on mw3 is OK: HTTP OK: HTTP/1.1 200 OK - 23221 bytes in 0.017 second response time
[01:26:12] <icinga-miraheze>	 RECOVERY - mw1 HTTPS on mw1 is OK: HTTP OK: HTTP/1.1 200 OK - 23221 bytes in 0.035 second response time
[01:26:33] <icinga-miraheze>	 PROBLEM - cp2 HTTP 4xx/5xx ERROR Rate on cp2 is WARNING: WARNING - NGINX Error Rate is 44%
[01:26:43] <icinga-miraheze>	 RECOVERY - cp3 Varnish Backends on cp3 is OK: All 5 backends are healthy
[01:27:08] <icinga-miraheze>	 RECOVERY - cp2 HTTPS on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 24075 bytes in 0.797 second response time
[01:27:11] <icinga-miraheze>	 RECOVERY - cp2 Varnish Backends on cp2 is OK: All 5 backends are healthy
[01:27:15] <icinga-miraheze>	 RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online
[01:27:31] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[01:27:58] <PuppyKun>	 uhhhhhhhhh
[01:28:02] <icinga-miraheze>	 RECOVERY - cp3 HTTPS on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 24074 bytes in 1.388 second response time
[01:28:30] <icinga-miraheze>	 RECOVERY - cp2 HTTP 4xx/5xx ERROR Rate on cp2 is OK: OK - NGINX Error Rate is 8%
[01:28:35] <icinga-miraheze>	 RECOVERY - cp4 Varnish Backends on cp4 is OK: All 5 backends are healthy
[01:28:42] <icinga-miraheze>	 RECOVERY - cp4 HTTP 4xx/5xx ERROR Rate on cp4 is OK: OK - NGINX Error Rate is 2%
[01:54:39] <icinga-miraheze>	 PROBLEM - absurdopedia.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[01:56:35] <icinga-miraheze>	 RECOVERY - absurdopedia.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'absurdopedia.wiki' will expire on Sat 15 Jun 2019 12:09:11 PM GMT +0000.
[02:55:10] <Not-7290>	 [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjYt7
[02:55:12] <Not-7290>	 [02miraheze/services] 07MirahezeSSLBot 0369b1fa5 - BOT: Updating services config for wikis
[03:55:32] <icinga-miraheze>	 PROBLEM - cp2 HTTPS on cp2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[03:59:31] <icinga-miraheze>	 RECOVERY - cp2 HTTPS on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 24075 bytes in 0.600 second response time
[04:01:14] <icinga-miraheze>	 PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 5061 MB (20% inode=95%);
[05:20:11] <Not-7290>	 [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjYqK
[05:20:13] <Not-7290>	 [02miraheze/services] 07MirahezeSSLBot 034ad74f9 - BOT: Updating services config for wikis
[05:32:34] <icinga-miraheze>	 PROBLEM - mw3 HTTPS on mw3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[05:32:43] <icinga-miraheze>	 PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 3 backends are down. mw1 mw2 mw3
[05:33:05] <icinga-miraheze>	 PROBLEM - cp4 Varnish Backends on cp4 is CRITICAL: 3 backends are down. mw1 mw2 mw3
[05:33:11] <icinga-miraheze>	 PROBLEM - cp2 Varnish Backends on cp2 is CRITICAL: 3 backends are down. mw1 mw2 mw3
[05:33:31] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[05:34:11] <icinga-miraheze>	 PROBLEM - mw1 HTTPS on mw1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[05:34:17] <icinga-miraheze>	 PROBLEM - mw2 HTTPS on mw2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[05:34:42] <icinga-miraheze>	 PROBLEM - cp4 HTTP 4xx/5xx ERROR Rate on cp4 is CRITICAL: CRITICAL - NGINX Error Rate is 74%
[05:35:15] <icinga-miraheze>	 PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[05:36:01] <icinga-miraheze>	 PROBLEM - cp3 HTTPS on cp3 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Backend fetch failed - 3871 bytes in 0.656 second response time
[05:36:38] <icinga-miraheze>	 PROBLEM - cp4 HTTPS on cp4 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Backend fetch failed - 3858 bytes in 0.014 second response time
[05:37:07] <icinga-miraheze>	 PROBLEM - cp2 HTTPS on cp2 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Backend fetch failed - 3860 bytes in 0.401 second response time
[05:38:02] <icinga-miraheze>	 RECOVERY - cp3 HTTPS on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 24074 bytes in 1.362 second response time
[05:38:15] <icinga-miraheze>	 RECOVERY - mw1 HTTPS on mw1 is OK: HTTP OK: HTTP/1.1 200 OK - 23221 bytes in 0.010 second response time
[05:38:15] <icinga-miraheze>	 RECOVERY - mw2 HTTPS on mw2 is OK: HTTP OK: HTTP/1.1 200 OK - 23221 bytes in 0.030 second response time
[05:38:38] <icinga-miraheze>	 RECOVERY - cp4 HTTPS on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 24083 bytes in 0.010 second response time
[05:38:42] <icinga-miraheze>	 RECOVERY - cp4 HTTP 4xx/5xx ERROR Rate on cp4 is OK: OK - NGINX Error Rate is 5%
[05:38:43] <icinga-miraheze>	 RECOVERY - cp3 Varnish Backends on cp3 is OK: All 5 backends are healthy
[05:38:49] <icinga-miraheze>	 RECOVERY - mw3 HTTPS on mw3 is OK: HTTP OK: HTTP/1.1 200 OK - 23221 bytes in 0.008 second response time
[05:39:02] <icinga-miraheze>	 RECOVERY - cp4 Varnish Backends on cp4 is OK: All 5 backends are healthy
[05:39:07] <icinga-miraheze>	 RECOVERY - cp2 HTTPS on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 24085 bytes in 0.499 second response time
[05:39:11] <icinga-miraheze>	 RECOVERY - cp2 Varnish Backends on cp2 is OK: All 5 backends are healthy
[05:39:15] <icinga-miraheze>	 RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online
[05:39:31] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[06:04:52] <icinga-miraheze>	 PROBLEM - misc2 Puppet on misc2 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): Package[php7.3-apcu],Package[php7.3-redis]
[06:12:52] <icinga-miraheze>	 RECOVERY - misc2 Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[06:27:14] <icinga-miraheze>	 RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 5510 MB (22% inode=95%);
[08:19:25] <ZppixBot>	 Hello MDead_! If you have any questions feel free to ask and someone should answer soon.
[08:40:12] <Not-7290>	 [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjYOX
[08:40:14] <Not-7290>	 [02miraheze/services] 07MirahezeSSLBot 038754921 - BOT: Updating services config for wikis
[10:25:11] <Not-7290>	 [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjYsx
[10:25:12] <Not-7290>	 [02miraheze/services] 07MirahezeSSLBot 03fadbcbd - BOT: Updating services config for wikis
[13:15:12] <Not-7290>	 [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjYcq
[13:15:14] <Not-7290>	 [02miraheze/services] 07MirahezeSSLBot 03451cb7a - BOT: Updating services config for wikis
[13:17:24] <Not-7290>	 [02mw-config] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbvb3
[13:17:25] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjYc3
[13:17:27] <Not-7290>	 [02miraheze/mw-config] 07paladox 0366aecf6 - Make TemplateData a global extension
[13:17:28] <Not-7290>	 [02mw-config] 07paladox opened pull request 03#2669: Make TemplateData a global extension - 13https://git.io/fjYcs
[13:17:46] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjYcG
[13:17:48] <Not-7290>	 [02miraheze/mw-config] 07paladox 03ba8afa1 - Update LocalExtensions.php
[13:17:49] <Not-7290>	 [02mw-config] 07paladox synchronize pull request 03#2669: Make TemplateData a global extension - 13https://git.io/fjYcs
[13:18:00] <Not-7290>	 [02mw-config] 07paladox closed pull request 03#2669: Make TemplateData a global extension - 13https://git.io/fjYcs
[13:18:01] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/fjYcZ
[13:18:03] <Not-7290>	 [02miraheze/mw-config] 07paladox 03680fb98 - Make TemplateData a global extension (#2669) * Make TemplateData a global extension * Update LocalExtensions.php
[13:18:05] <Not-7290>	 [02mw-config] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbvb3
[13:18:07] <Not-7290>	 [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-1
[13:18:48] <travis-ci>	 miraheze/mw-config/paladox-patch-1/66aecf6 - paladox The build is still failing. https://travis-ci.org/miraheze/mw-config/builds/520765114
[13:20:03] <travis-ci>	 miraheze/mw-config/paladox-patch-1/ba8afa1 - paladox The build is still failing. https://travis-ci.org/miraheze/mw-config/builds/520765255
[15:49:32] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 032 commits to 03master [+0/-0/±3] 13https://git.io/fjY81
[15:49:34] <Not-7290>	 [02miraheze/mw-config] 07paladox 03a4668e4 - Revert "Make TemplateData a global extension (#2669)" This reverts commit 680fb986e2e7f0babcbdf0c459962558174e73f3.
[15:49:35] <Not-7290>	 [02miraheze/mw-config] 07paladox 031b3756b - Load TemplateData under TemplateWizard Suggested by @MacFan4000 and @John Change-Id: Ic0088c979170ff0b911db969f55160107728afe0
[16:00:01] <MH-Discord>	 <TheInterloperMage> paladox
[16:00:27] <paladox>	 hi
[16:01:36] <Not-7290>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjY4J
[16:01:37] <Not-7290>	 [02miraheze/puppet] 07paladox 03f3255fa - Raise redis max memory to 1150mb
[16:02:52] <MH-Discord>	 <TheInterloperMage> I want an image that I uploaded deleted from the Toxic Fandoms and Hatedoms Wiki (https://toxicfandomsandhatedoms.miraheze.org/wiki/Toxic_Fandoms_%26_Hatedoms_Wiki) deleted.  The file I uploaded (https://toxicfandomsandhatedoms.miraheze.org/wiki/File:ExcuseThis.png) is something that I regret, as it was done impulsively without considering that the YouTube channel whose video had this thumbnail might be harassed for a
[16:02:53] <MH-Discord>	 thumbnail that is over six years old.  I would delete it, but it seems that deleting images is only for admins, and the only admin of the Wiki has been inactive for a long time now.
[16:04:11] <MH-Discord>	 <TheInterloperMage> The best way I could "delete" it was remove it from the page it was on and replace the file basically stating that I disowned it for uploading it impulsively out of anger.
[16:05:53] <MH-Discord>	 <TheInterloperMage> However, that didn't work, because the image keeps going back to the original image that I uploaded and wish to be deleted.
[16:09:09] <icinga-miraheze>	 PROBLEM - db4 Current Load on db4 is CRITICAL: CRITICAL - load average: 10.13, 6.78, 4.82
[16:09:29] <icinga-miraheze>	 PROBLEM - misc2 HTTPS on misc2 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 374 bytes in 0.007 second response time
[16:11:29] <icinga-miraheze>	 RECOVERY - misc2 HTTPS on misc2 is OK: HTTP OK: HTTP/1.1 200 OK - 45693 bytes in 0.241 second response time
[16:13:09] <icinga-miraheze>	 PROBLEM - db4 Current Load on db4 is WARNING: WARNING - load average: 6.28, 6.95, 5.40
[16:17:09] <icinga-miraheze>	 RECOVERY - db4 Current Load on db4 is OK: OK - load average: 6.08, 6.63, 5.65
[16:22:25] <paladox>	 @TheInterloperMage hi, it's been done! (by a steward)
[16:24:59] <MH-Discord>	 <TheInterloperMage> Thanks!
[16:58:35] <icinga-miraheze>	 PROBLEM - cp4 Varnish Backends on cp4 is CRITICAL: 1 backends are down. mw2
[16:58:43] <icinga-miraheze>	 PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 1 backends are down. mw2
[16:59:11] <icinga-miraheze>	 PROBLEM - cp2 Varnish Backends on cp2 is CRITICAL: 2 backends are down. mw2 mw3
[16:59:39] <Mikeee|>	 I'm getting that "Something went wrong Unknown error" message every time i save from the visual editor. How do I view the error logs to see what is causing the error?
[17:00:14] <Mikeee|>	 i mean, I'm getting 503 for any editing at the moment
[17:00:14] <MacFan4000>	 paladox: ^
[17:00:26] <Mikeee|>	 but i assume that's not just me and will be fixed soon
[17:00:33] <paladox>	 Mikeee| hi
[17:00:39] <MacFan4000>	 Mikeee|: only sysadmins can view the logs
[17:00:39] <paladox>	 which wiki and which page?
[17:00:52] <Mikeee|>	 every page, industrialrobotics.miraheze.org
[17:01:10] <MH-Discord>	 <k6ka> Can confirm, thesimswiki.com is experiencing downtime too.
[17:01:15] <icinga-miraheze>	 PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 2 datacenters are down: 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[17:01:31] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 3 datacenters are down: 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[17:02:00] <paladox>	 hmm
[17:02:07] <paladox>	 so the VE issue returns
[17:02:08] <icinga-miraheze>	 PROBLEM - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is WARNING: WARNING - NGINX Error Rate is 43%
[17:02:17] <Mikeee|>	 I had this issue with the visual editor back when I ran my own mediawiki instance, I just don't know how to fix it when someone else hosts it
[17:02:51] <paladox>	 Mikeee| did you manage to fix it for you when you ran your instance?
[17:03:32] <Mikeee|>	 with mine it was when mediawiki didn't play nice with captcha stuff
[17:03:38] <Mikeee|>	 don't know if they fixed it since
[17:04:08] <icinga-miraheze>	 RECOVERY - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is OK: OK - NGINX Error Rate is 28%
[17:04:35] <icinga-miraheze>	 RECOVERY - cp4 Varnish Backends on cp4 is OK: All 5 backends are healthy
[17:04:43] <icinga-miraheze>	 RECOVERY - cp3 Varnish Backends on cp3 is OK: All 5 backends are healthy
[17:05:31] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[17:07:15] <icinga-miraheze>	 RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online
[17:07:22] <Mikeee|>	 503 is fixed nicely
[17:07:57] <Mikeee|>	 i just initiated another save from the visual editor, should be the last entry in the error log for my instance
[17:08:37] <Mikeee|>	 it does actually save fine, it just says something went wrong and you have to refresh the page
[17:09:11] <icinga-miraheze>	 RECOVERY - cp2 Varnish Backends on cp2 is OK: All 5 backends are healthy
[18:04:57] <icinga-miraheze>	 PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 554
[18:20:57] <icinga-miraheze>	 RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs
[18:38:35] <icinga-miraheze>	 PROBLEM - cp4 Varnish Backends on cp4 is CRITICAL: 3 backends are down. mw1 mw2 mw3
[18:38:43] <icinga-miraheze>	 PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 2 backends are down. mw1 mw2
[18:39:10] <icinga-miraheze>	 PROBLEM - cp2 Varnish Backends on cp2 is CRITICAL: 1 backends are down. mw1
[18:39:15] <icinga-miraheze>	 PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 4 datacenters are down: 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb
[18:39:31] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb
[18:40:35] <icinga-miraheze>	 RECOVERY - cp4 Varnish Backends on cp4 is OK: All 5 backends are healthy
[18:40:43] <icinga-miraheze>	 RECOVERY - cp3 Varnish Backends on cp3 is OK: All 5 backends are healthy
[18:41:11] <icinga-miraheze>	 RECOVERY - cp2 Varnish Backends on cp2 is OK: All 5 backends are healthy
[18:41:15] <icinga-miraheze>	 RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online
[18:41:31] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[19:44:26] <Not-7290>	 [02MirahezeMagic] 07paladox created branch 03paladox-patch-2 - 13https://git.io/fQRGX
[19:44:27] <Not-7290>	 [02miraheze/MirahezeMagic] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/fjYEA
[19:44:29] <Not-7290>	 [02miraheze/MirahezeMagic] 07paladox 03ec82e34 - Add ""settings-links" to i18n
[19:44:30] <Not-7290>	 [02MirahezeMagic] 07paladox opened pull request 03#74: Add ""settings-links" to i18n - 13https://git.io/fjYEx
[19:44:35] <Not-7290>	 [02MirahezeMagic] 07paladox edited pull request 03#74: Add "settings-links" to i18n - 13https://git.io/fjYEx
[19:45:14] <Not-7290>	 [02miraheze/MirahezeMagic] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/fjYEp
[19:45:16] <Not-7290>	 [02miraheze/MirahezeMagic] 07paladox 0393f5fd3 - Update qqq.json
[19:45:17] <Not-7290>	 [02MirahezeMagic] 07paladox synchronize pull request 03#74: Add "settings-links" to i18n - 13https://git.io/fjYEx
[19:45:35] <Not-7290>	 [02MirahezeMagic] 07paladox closed pull request 03#74: Add "settings-links" to i18n - 13https://git.io/fjYEx
[19:45:37] <Not-7290>	 [02miraheze/MirahezeMagic] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/fjYue
[19:45:38] <Not-7290>	 [02miraheze/MirahezeMagic] 07paladox 038fdf5cb - Add "settings-links" to i18n (#74) * Add ""settings-links" to i18n * Update qqq.json
[19:45:40] <Not-7290>	 [02miraheze/MirahezeMagic] 07paladox deleted branch 03paladox-patch-2
[19:45:41] <Not-7290>	 [02MirahezeMagic] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/fQRGX
[19:45:46] <travis-ci>	 miraheze/MirahezeMagic/paladox-patch-2/ec82e34 - paladox The build was fixed. https://travis-ci.com/miraheze/MirahezeMagic/builds/108580050
[19:46:43] <travis-ci>	 miraheze/MirahezeMagic/paladox-patch-2/93f5fd3 - paladox The build is still failing. https://travis-ci.com/miraheze/MirahezeMagic/builds/108580153
[19:48:30] <Not-7290>	 [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+0/-0/±1] 13https://git.io/fjYuU
[19:48:31] <Not-7290>	 [02miraheze/mediawiki] 07paladox 03387f3ed - Update MM
[19:50:26] <paladox>	 !log rebuilding lc on mw*
[19:50:31] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master
[19:58:00] <Not-7290>	 [02mw-config] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbvb3
[19:58:02] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjYu3
[19:58:03] <Not-7290>	 [02miraheze/mw-config] 07paladox 0335e7776 - Move wgExternalLinkTarget to ManageWiki
[19:58:05] <Not-7290>	 [02mw-config] 07paladox opened pull request 03#2670: Move wgExternalLinkTarget to ManageWiki - 13https://git.io/fjYus
[19:59:30] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjYuW
[19:59:32] <Not-7290>	 [02miraheze/mw-config] 07paladox 03e62d4db - Update LocalSettings.php
[19:59:33] <Not-7290>	 [02mw-config] 07paladox synchronize pull request 03#2670: Move wgExternalLinkTarget to ManageWiki - 13https://git.io/fjYus
[19:59:41] <travis-ci>	 miraheze/mw-config/paladox-patch-1/35e7776 - paladox The build was fixed. https://travis-ci.org/miraheze/mw-config/builds/520935686
[20:01:58] <Not-7290>	 [02mw-config] 07paladox closed pull request 03#2670: Move wgExternalLinkTarget to ManageWiki - 13https://git.io/fjYus
[20:01:59] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/fjYu4
[20:02:01] <Not-7290>	 [02miraheze/mw-config] 07paladox 03c37c2b8 - Move wgExternalLinkTarget to ManageWiki (#2670) * Move wgExternalLinkTarget to ManageWiki * Update LocalSettings.php
[20:02:56] <Not-7290>	 [02mw-config] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbvb3
[20:02:57] <Not-7290>	 [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-1
[20:03:50] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjYuR
[20:03:51] <Not-7290>	 [02miraheze/mw-config] 07paladox 03783d1ff - Update ManageWiki.php
[20:04:32] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjYuE
[20:04:33] <Not-7290>	 [02miraheze/mw-config] 07paladox 035fa9107 - Update ManageWiki.php
[20:06:22] <icinga-miraheze>	 PROBLEM - mw1 Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki config]
[20:10:38] <Not-7290>	 [02mw-config] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbvb3
[20:10:39] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjYua
[20:10:41] <Not-7290>	 [02miraheze/mw-config] 07paladox 03395d5f9 - Move wgAllowExternalImages to ManageWiki
[20:10:42] <Not-7290>	 [02mw-config] 07paladox opened pull request 03#2671: Move wgAllowExternalImages to ManageWiki - 13https://git.io/fjYuV
[20:11:50] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjYuw
[20:11:52] <Not-7290>	 [02miraheze/mw-config] 07paladox 03fb1f823 - Update LocalSettings.php
[20:11:53] <Not-7290>	 [02mw-config] 07paladox synchronize pull request 03#2671: Move wgAllowExternalImages to ManageWiki - 13https://git.io/fjYuV
[20:13:23] <Not-7290>	 [02mw-config] 07paladox closed pull request 03#2671: Move wgAllowExternalImages to ManageWiki - 13https://git.io/fjYuV
[20:13:24] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/fjYuK
[20:13:25] <Not-7290>	 [02miraheze/mw-config] 07paladox 03cae92a8 - Move wgAllowExternalImages to ManageWiki (#2671) * Move wgAllowExternalImages to ManageWiki * Update LocalSettings.php
[20:13:27] <Not-7290>	 [02mw-config] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbvb3
[20:13:28] <Not-7290>	 [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-1
[20:14:18] <icinga-miraheze>	 RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[20:15:20] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjYuP
[20:15:22] <Not-7290>	 [02miraheze/mw-config] 07paladox 030831e12 - Update ManageWiki.php
[20:34:55] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjYzJ
[20:34:57] <Not-7290>	 [02miraheze/mw-config] 07paladox 03ae22804 - Move wgPopupsBetaFeature to the beta section
[20:38:48] <Not-7290>	 [02mw-config] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbvb3
[20:38:49] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjYzO
[20:38:51] <Not-7290>	 [02miraheze/mw-config] 07paladox 03d38ee32 - Move wgAllowImageTag to ManageWiki
[20:39:00] <Not-7290>	 [02mw-config] 07paladox opened pull request 03#2672: Move wgAllowImageTag to ManageWiki - 13https://git.io/fjYz3
[20:40:32] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjYzG
[20:40:34] <Not-7290>	 [02miraheze/mw-config] 07paladox 030b8404d - Update LocalSettings.php
[20:40:35] <Not-7290>	 [02mw-config] 07paladox synchronize pull request 03#2672: Move wgAllowImageTag to ManageWiki - 13https://git.io/fjYz3
[20:40:46] <Not-7290>	 [02mw-config] 07paladox closed pull request 03#2672: Move wgAllowImageTag to ManageWiki - 13https://git.io/fjYz3
[20:40:48] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/fjYzZ
[20:40:49] <Not-7290>	 [02miraheze/mw-config] 07paladox 038f4d405 - Move wgAllowImageTag to ManageWiki (#2672) * Move wgAllowImageTag to ManageWiki * Update LocalSettings.php
[20:40:51] <Not-7290>	 [02mw-config] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbvb3
[20:40:52] <Not-7290>	 [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-1
[20:41:56] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjYzc
[20:41:58] <Not-7290>	 [02miraheze/mw-config] 07paladox 033fbe4dd - Fix wgAllowImageTag description
[20:42:36] <travis-ci>	 miraheze/mw-config/paladox-patch-1/d38ee32 - paladox The build was broken. https://travis-ci.org/miraheze/mw-config/builds/520951599
[20:43:41] <travis-ci>	 miraheze/mw-config/paladox-patch-1/0b8404d - paladox The build was broken. https://travis-ci.org/miraheze/mw-config/builds/520952374
[20:44:02] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjYzl
[20:44:04] <Not-7290>	 [02miraheze/mw-config] 07paladox 033840153 - Fix wgAllowImageTag part two
[20:49:10] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjYzR
[20:49:12] <Not-7290>	 [02miraheze/mw-config] 07paladox 03470e328 - Update ManageWiki.php
[20:59:21] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjYzM
[20:59:23] <Not-7290>	 [02miraheze/mw-config] 07paladox 039cfcfa6 - Remove wgAllowTitlesInSVG
[21:06:35] <icinga-miraheze>	 PROBLEM - cp4 Varnish Backends on cp4 is CRITICAL: 1 backends are down. mw3
[21:06:43] <icinga-miraheze>	 PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 2 backends are down. mw2 mw3
[21:07:15] <icinga-miraheze>	 PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[21:07:31] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[21:11:15] <icinga-miraheze>	 RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online
[21:11:31] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[21:12:35] <icinga-miraheze>	 RECOVERY - cp4 Varnish Backends on cp4 is OK: All 5 backends are healthy
[21:12:43] <icinga-miraheze>	 RECOVERY - cp3 Varnish Backends on cp3 is OK: All 5 backends are healthy
[21:31:21] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjYgW
[21:31:23] <Not-7290>	 [02miraheze/mw-config] 07paladox 03fb85ae4 - Add wgShowArchiveThumbnails to ManageWiki
[21:57:29] <Hispano76>	 paladox: ping
[21:57:36] <paladox>	 hi
[21:58:03] <Hispano76>	 Hey, you know any regular expressions?
[21:58:18] <paladox>	 I know some yup
[22:02:44] <Hispano76>	 All right, I'll take care of it. I'm just using the ReplaceText extension to change a template used in images. But I have the problem that each image has a different url and I would like to keep them. For example, this would be the change: From {{source|https://URL|mediateca.inah.gob.mx}} to | fuenteurl = https://URLYou know how I can do that?  paladox  
[22:04:10] <paladox>	 in js regex it's "[-a-zA-Z0-9@:%_\+.~#?&//=]{2,256}\.[a-z]{2,4}\b(\/[-a-zA-Z0-9@:%_\+.~#?&//=]*)?" see https://stackoverflow.com/questions/3809401/what-is-a-good-regular-expression-to-match-a-url
[22:04:40] <JohnLewis>	 "in js regex" if that's not how RT works then that's an irrelevant answer :)
[22:06:00] <paladox>	 you probably want MassEditRegex  instead of ReplaceText for regex Hispano76 
[22:06:07] <paladox>	 See https://www.mediawiki.org/wiki/Extension:MassEditRegex
[22:06:47] * paladox looks at JohnLewis 
[22:07:04] <Hispano76>	 Um, I have it enabled. But I have the same inconvenience that I don't know how to do it keeping the web url in each image. 
[22:13:11] <Hispano76>	 https://prnt.sc/ncxfnt paladox 
[22:14:30] <paladox>	 hmm
[22:16:36] <paladox>	 Hispano76 i guess Buscar is "search for"?
[22:17:04] <paladox>	 that regex looks wrong though
[22:17:06] <paladox>	 you may want
[22:17:19] <paladox>	 https://(.*)$
[22:17:25] <Hispano76>	 Yes
[22:17:57] <Hispano76>	 All right, I'm gonna run a test.
[22:21:03] <Hispano76>	 paladox: https://prnt.sc/ncxinj I guess that's not how it is. :)
[22:21:04] <ZppixBot>	 Title: [ Screenshot by Lightshot ] - prnt.sc
[22:22:00] <paladox>	 Hispano76 do: /https://(.*)$/
[22:26:09] <Hispano76>	 hum, paladox  https://prnt.sc/ncxki4 
[22:26:10] <ZppixBot>	 Title: [ Screenshot by Lightshot ] - prnt.sc
[22:27:45] <paladox>	 the second column should be $1
[22:27:52] <paladox>	 ie https://$1
[22:29:20] <Hispano76>	 No, I get internal error. 
[22:33:14] <Voidwalker>	 here's hoping global user merges work today :P
[22:35:06] <JohnLewis>	 glhf :P
[22:36:51] <Not-7290>	 [02mw-config] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbvb3
[22:36:52] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjY2O
[22:36:54] <Not-7290>	 [02miraheze/mw-config] 07paladox 03c0ba60b - Move wgSVGConverter to ManageWIki
[22:36:55] <Not-7290>	 [02mw-config] 07paladox opened pull request 03#2673: Move wgSVGConverter to ManageWIki - 13https://git.io/fjY23
[22:37:13] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjY2s
[22:37:14] <Not-7290>	 [02miraheze/mw-config] 07paladox 03fbdcb99 - Update LocalSettings.php
[22:37:16] <Not-7290>	 [02mw-config] 07paladox synchronize pull request 03#2673: Move wgSVGConverter to ManageWIki - 13https://git.io/fjY23
[22:37:54] <Voidwalker>	 outlook not so good, but I'll give it a few more minutes
[22:39:43] <Hispano76>	 https://prnt.sc/ncxp15
[22:39:44] <ZppixBot>	 Title: [ Screenshot by Lightshot ] - prnt.sc
[22:39:52] <travis-ci>	 miraheze/mw-config/paladox-patch-1/fbdcb99 - paladox The build was fixed. https://travis-ci.org/miraheze/mw-config/builds/521002755
[22:43:57] <Not-7290>	 [02mw-config] 07paladox synchronize pull request 03#2673: Move wgSVGConverter to ManageWIki - 13https://git.io/fjY23
[22:43:59] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjY2Z
[22:44:00] <Not-7290>	 [02miraheze/mw-config] 07paladox 03afa63ac - Update ManageWiki.php
[22:44:51] <paladox>	 Hispano76 i think some how the extension needs the regex at the start
[22:52:15] <Not-7290>	 [02mw-config] 07paladox closed pull request 03#2673: Move wgSVGConverter to ManageWIki - 13https://git.io/fjY23
[22:52:17] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/fjY24
[22:52:18] <Not-7290>	 [02miraheze/mw-config] 07paladox 03bae4bf9 - Move wgSVGConverter to ManageWIki (#2673) * Move wgSVGConverter to ManageWIki * Update LocalSettings.php * Update ManageWiki.php
[22:52:20] <Not-7290>	 [02mw-config] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbvb3
[22:52:21] <Not-7290>	 [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-1
[22:56:32] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjY2R
[22:56:33] <Not-7290>	 [02miraheze/mw-config] 07paladox 03420b584 - Fix value name in wgSVGConverter
[23:01:37] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjY2z
[23:01:38] <Not-7290>	 [02miraheze/mw-config] 07paladox 038fcc99d - Update ManageWiki.php
[23:02:57] <icinga-miraheze>	 PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 456
[23:05:31] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjY22
[23:05:32] <Not-7290>	 [02miraheze/mw-config] 07paladox 0331ec50e - Exempting wgImportSources from ManageWiki Reason is it would be some huge list if we figure out how to get all wiki's db..
[23:11:10] <icinga-miraheze>	 PROBLEM - cp2 Varnish Backends on cp2 is CRITICAL: 3 backends are down. mw1 mw2 mw3
[23:11:16] <icinga-miraheze>	 PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[23:11:31] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[23:11:48] <Hispano76>	 503 mediatecawiki
[23:12:02] <icinga-miraheze>	 PROBLEM - cp4 HTTP 4xx/5xx ERROR Rate on cp4 is WARNING: WARNING - NGINX Error Rate is 51%
[23:12:35] <icinga-miraheze>	 PROBLEM - cp4 Varnish Backends on cp4 is CRITICAL: 1 backends are down. mw3
[23:12:43] <icinga-miraheze>	 PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 1 backends are down. mw3
[23:13:31] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[23:13:31] <JohnLewis>	 paladox: ^ put MW on hold please
[23:13:38] <paladox>	 ok
[23:14:01] <icinga-miraheze>	 RECOVERY - cp4 HTTP 4xx/5xx ERROR Rate on cp4 is OK: OK - NGINX Error Rate is 1%
[23:14:20] <paladox>	 JohnLewis things recovered.
[23:14:38] <JohnLewis>	 yeah but this is like the 30th time in the past 48 hours
[23:14:44] <icinga-miraheze>	 PROBLEM - db4 Disk Space on db4 is WARNING: DISK WARNING - free space: / 41574 MB (11% inode=95%);
[23:14:55] <JohnLewis>	 I didn't ask you to stop because it was down, but rather that is the priority now
[23:14:57] <icinga-miraheze>	 RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs
[23:15:15] <icinga-miraheze>	 RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online
[23:15:16] <paladox>	 ok
[23:15:57] <paladox>	 I doin't see anything in php-error.log or exception.log which would explain this, so /me checks nginx logs
[23:16:21] <JohnLewis>	 ofc you wouldn't they're not relevant logs?
[23:17:00] <Hispano76>	 :/
[23:17:31] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 5 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[23:18:34] <paladox>	 JohnLewis you woulden't know how i can use grep to only show the lines which contain 503?
[23:18:57] <icinga-miraheze>	 PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 382
[23:19:31] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[23:19:31] <JohnLewis>	 cat | grep
[23:20:35] <icinga-miraheze>	 RECOVERY - cp4 Varnish Backends on cp4 is OK: All 5 backends are healthy
[23:20:36] <paladox>	 ah right
[23:20:43] <icinga-miraheze>	 RECOVERY - cp3 Varnish Backends on cp3 is OK: All 5 backends are healthy
[23:21:11] <icinga-miraheze>	 RECOVERY - cp2 Varnish Backends on cp2 is OK: All 5 backends are healthy
[23:24:02] <paladox>	 i can see that "- - [16/Apr/2019:23:09:44 +0000] "GET /healthcheck HTTP/1.0" 503 xx "-" "gdnsd-monitor" 0.001 -/-" which is 00:09 our time
[23:24:16] <paladox>	 i can also see the problem started specifically at 23:05
[23:24:46] <paladox>	 oh
[23:24:53] <paladox>	 no, 23:09 sorry
[23:24:57] <icinga-miraheze>	 RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs
[23:25:29] <paladox>	 23:09:08
[23:41:44] <Hispano76>	 paladox: It didn't work. https://mediateca.miraheze.org/w/index.php?title=Archivo:Abraham_Gonz%C3%A1lez_Casavantes.jpg&diff=8899&oldid=8898
[23:41:46] <ZppixBot>	 Title: [ Diferencia entre revisiones de «Archivo:Abraham González Casavantes.jpg» - Mediateca Wiki ] - mediateca.miraheze.org
[23:42:03] <paladox>	 oh :(