[00:03:07] <icinga-miraheze>	 PROBLEM - Puppet on misc1 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues  
[00:03:15] <icinga-miraheze>	 PROBLEM - Puppet on cp4 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues  
[00:03:17] <icinga-miraheze>	 PROBLEM - Puppet on test1 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues  
[00:04:07] <icinga-miraheze>	 PROBLEM - Puppet on mw1 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues  
[00:04:17] <icinga-miraheze>	 PROBLEM - Puppet on cp5 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues  
[00:05:07] <icinga-miraheze>	 RECOVERY - Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 38 seconds ago with 0 failures  
[00:05:15] <icinga-miraheze>	 RECOVERY - Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 29 seconds ago with 0 failures  
[00:08:05] <icinga-miraheze>	 PROBLEM - Bacula - Private Git on bacula1 is WARNING: WARNING: Full, 1372 files, 1.467MB, 2018-07-08 00:05:00 (1.1 weeks ago)  
[00:08:30] <Not-683e>	 [02miraheze/services] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNYrk
[00:08:31] <Not-683e>	 [02miraheze/services] 07paladox 03260c98d - Enable services for allwikis fixes #T3359
[00:11:16] <icinga-miraheze>	 RECOVERY - Puppet on test1 is OK: OK: Puppet is currently enabled, last run 34 seconds ago with 0 failures  
[00:12:16] <icinga-miraheze>	 RECOVERY - Puppet on cp5 is OK: OK: Puppet is currently enabled, last run 4 seconds ago with 0 failures  
[00:14:06] <icinga-miraheze>	 RECOVERY - Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 58 seconds ago with 0 failures  
[02:17:36] <Not-683e>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNY68
[02:17:37] <Not-683e>	 [02miraheze/puppet] 07paladox 03dfdfd73 - Update proxy-server.conf.erb
[02:21:00] <Not-683e>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNY6u
[02:21:01] <Not-683e>	 [02miraheze/puppet] 07paladox 038e35cee - Update container-server.conf.erb
[02:22:39] <Not-683e>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNY6g
[02:22:41] <Not-683e>	 [02miraheze/puppet] 07paladox 0375e6484 - Update object-server.conf.erb
[06:13:37] <icinga-miraheze>	 PROBLEM - JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 5508  
[08:03:36] <icinga-miraheze>	 RECOVERY - JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs  
[10:39:56] <icinga-miraheze>	 PROBLEM - wiki.exnihilolinux.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate wiki.exnihilolinux.org expires in 15 day(s) (Wed 01 Aug 2018 10:36:40 AM GMT +0000).  
[10:40:11] <Not-683e>	 [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNOJz
[10:40:13] <Not-683e>	 [02miraheze/ssl] 07MirahezeSSLBot 0341fb35f - Bot: Update SSL cert for wiki.exnihilolinux.org
[10:42:00] <icinga-miraheze>	 RECOVERY - wiki.exnihilolinux.org - LetsEncrypt on sslhost is OK: OK - Certificate wiki.exnihilolinux.org will expire on Sun 14 Oct 2018 09:40:02 AM GMT +0000.  
[13:16:36] <Wiki-1776>	 Hi
[13:27:21] <icinga-miraheze>	 PROBLEM - Puppet on test1 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 1 minute ago with 0 failures  
[15:00:19] <Not-683e>	 [02miraheze/ManageWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±16] 13https://git.io/fNO2l
[15:00:20] <Not-683e>	 [02miraheze/ManageWiki] 07translatewiki 031c329e7 - Localisation updates from https://translatewiki.net.
[15:00:22] <Not-683e>	 [02miraheze/CreateWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±12] 13https://git.io/fNO28
[15:00:23] <Not-683e>	 [02miraheze/CreateWiki] 07translatewiki 03ae86acd - Localisation updates from https://translatewiki.net.
[15:00:25] <Not-683e>	 [02miraheze/WikiDiscover] 07translatewiki pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/fNO24
[15:00:26] <Not-683e>	 [02miraheze/WikiDiscover] 07translatewiki 03207bbd0 - Localisation updates from https://translatewiki.net.
[19:39:24] <Not-683e>	 [02miraheze/CreateWiki] 07JohnFLewis pushed 032 commits to 03master [+0/-0/±2] 13https://git.io/fNOAr
[19:39:26] <Not-683e>	 [02miraheze/CreateWiki] 07JohnFLewis 03b82d026 - add CreateWikiCreation hook
[19:39:28] <Not-683e>	 [02miraheze/CreateWiki] 07JohnFLewis 03c81a7e0 - remove Miraheze specific code from SpecialCreateWiki
[19:39:31] <Not-683e>	 [02miraheze/MirahezeMagic] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/fNOAo
[19:39:32] <Not-683e>	 [02miraheze/MirahezeMagic] 07JohnFLewis 03a61be7e - use Hook for swift CW
[19:39:35] <JohnLewis>	 paladox: ^^
[19:40:01] <paladox>	 JohnLewis ok, also syntax error!
[19:40:07] <paladox>	 in the json ext file
[19:40:33] <paladox>	 brb
[19:40:45] <Not-683e>	 [02miraheze/MirahezeMagic] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/fNOAM
[19:40:46] <Not-683e>	 [02miraheze/MirahezeMagic] 07JohnFLewis 03bb0d9db - use Hook for swift CW
[19:40:47] <JohnLewis>	 fixed
[19:41:39] <JohnLewis>	 now additional steps can be added to wiki creation by any extensions
[19:42:50] <JohnLewis>	 which is key because I don't want analytics to depend on CreateWiki
[20:04:39] <paladox>	 JohnLewis back and ok
[20:05:36] <JohnLewis>	 paladox: https://github.com/miraheze/MatomoAnalytics do you like the stump :P
[20:05:37] <ZppixBot>	 Title: [ GitHub - miraheze/MatomoAnalytics ] - github.com
[20:06:00] <paladox>	 heh
[20:06:04] <paladox>	 JohnLewis what stump?
[20:06:19] <JohnLewis>	 it's an extension that adds one i18n message :P
[20:06:30] <paladox>	 ah i see
[20:06:32] <paladox>	 lol 
[20:06:37] <paladox>	 JohnLewis :D
[20:15:57] <JohnLewis>	 https://phabricator.miraheze.org/T680#63472
[20:15:58] <ZppixBot>	 Title: [ ⚓ T680 Add statistics interface for founders/users ] - phabricator.miraheze.org
[21:05:32] <icinga-miraheze>	 PROBLEM - JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 2665  
[21:11:16] <JohnLewis>	 bah need to add the new repo to IRC stuff
[21:14:14] <JohnLewis>	 meh anyway with https://github.com/miraheze/MatomoAnalytics/commit/f64c51b84f1d5fd8f8be6a94d702fb4b8de44bd0 we can deploy it technically and remove the MirahezeMagic code for it
[21:14:15] <ZppixBot>	 Title: [ add matomo JS script · miraheze/MatomoAnalytics@f64c51b · GitHub ] - github.com
[21:14:22] <icinga-miraheze>	 PROBLEM - decrypted.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate decrypted.wiki expires in 15 day(s) (Wed 01 Aug 2018 09:11:05 PM GMT +0000).  
[21:14:32] <Not-683e>	 [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fN3Te
[21:14:34] <Not-683e>	 [02miraheze/ssl] 07MirahezeSSLBot 038afbc4c - Bot: Update SSL cert for decrypted.wiki
[21:15:21] <JohnLewis>	 though I think I might hold off it until it's somewhat more usable as MirahezeMagic is fine at what it's doing right now, not really changed anything that it can't handle it
[21:22:23] <icinga-miraheze>	 RECOVERY - decrypted.wiki - LetsEncrypt on sslhost is OK: OK - Certificate decrypted.wiki will expire on Sun 14 Oct 2018 08:14:27 PM GMT +0000.  
[22:23:30] <MH-Discord>	 <NeoMahler> I'm excited for this new extension! 😄
[22:23:49] <JohnLewis>	 aren't we all :P
[22:32:43] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is WARNING: WARNING - load average: 3.60, 3.00, 2.36  
[22:34:41] <icinga-miraheze>	 RECOVERY - Current Load on swift1 is OK: OK - load average: 2.20, 2.67, 2.31  
[23:07:42] <icinga-miraheze>	 PROBLEM - Varnish Backends on cp4 is CRITICAL: 2 backends are down.  mw2 mw3  
[23:07:44] <icinga-miraheze>	 PROBLEM - GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb, 172.104.111.8/cpweb, 2400:8902::f03c:91ff:fe07:444e/cpweb  
[23:08:22] <icinga-miraheze>	 PROBLEM - Varnish Backends on cp2 is CRITICAL: 1 backends are down.  mw3  
[23:09:32] <icinga-miraheze>	 PROBLEM - Varnish Backends on cp5 is CRITICAL: 1 backends are down.  mw2  
[23:09:42] <icinga-miraheze>	 RECOVERY - GDNSD Datacenters on ns1 is OK: OK - all datacenters are online  
[23:12:22] <icinga-miraheze>	 RECOVERY - Varnish Backends on cp2 is OK: All 3 backends are healthy  
[23:13:32] <icinga-miraheze>	 RECOVERY - Varnish Backends on cp5 is OK: All 3 backends are healthy  
[23:13:42] <icinga-miraheze>	 RECOVERY - Varnish Backends on cp4 is OK: All 3 backends are healthy  
[23:19:02] <Not-683e>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fN33g
[23:19:03] <Not-683e>	 [02miraheze/puppet] 07paladox 033673232 - Update proxy-server.conf.erb
[23:28:03] <icinga-miraheze>	 PROBLEM - Puppet on mw3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core]  
[23:34:03] <icinga-miraheze>	 RECOVERY - Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures