[00:17:01] <icinga-miraheze>	 PROBLEM - JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 5488  
[01:24:24] <ZppixBot>	 Hello smallsnail! If you have any questions feel free to ask and someone should answer soon.
[01:24:40] <smallsnail>	 hi, um... so i curious problem for ambox etc
[01:25:32] <smallsnail>	 https://i.imgur.com/wCSX0fP.png so ambox is showing this, and idk what's wrong to this (lua, old template is damm)
[01:25:51] <smallsnail>	 and all the browser is same output...
[01:46:27] <smallsnail>	 hmmm... even module is not launched - module:tabletools
[02:13:00] <icinga-miraheze>	 RECOVERY - JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs  
[03:04:48] <smallsnail>	 sorry, common.css is missing
[05:02:39] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is CRITICAL: CRITICAL - load average: 4.05, 3.33, 2.96  
[05:04:40] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is WARNING: WARNING - load average: 3.43, 3.33, 3.01  
[05:08:40] <icinga-miraheze>	 RECOVERY - Current Load on swift1 is OK: OK - load average: 3.18, 3.37, 3.10  
[05:37:09] <Not-683e>	 [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/fNf46
[05:37:11] <Not-683e>	 [02miraheze/dns] 07Reception123 036f2d75a - add toonpedia.cf zone
[05:47:26] <Not-683e>	 [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/fNf4h
[05:47:27] <Not-683e>	 [02miraheze/ssl] 07Reception123 03bc9be3e - add toonpedia.cf cert
[05:54:27] <Not-683e>	 [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNfBs
[05:54:29] <Not-683e>	 [02miraheze/mw-config] 07Reception123 0309cd7ff - add toonpedia.cf custom domain per T3330 resolves T3330
[05:57:19] <Not-683e>	 [02miraheze/services] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNfBZ
[05:57:20] <Not-683e>	 [02miraheze/services] 07Reception123 033b75df4 - add "aisg" to services
[06:07:06] <icinga-miraheze>	 PROBLEM - Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki config]  
[06:09:36] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is WARNING: WARNING - load average: 3.50, 3.51, 3.11  
[06:11:34] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is CRITICAL: CRITICAL - load average: 4.07, 3.72, 3.23  
[06:13:34] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is WARNING: WARNING - load average: 3.80, 3.72, 3.28  
[08:15:26] <Reception123>	 !log sudo -u www-data php importDump.php --wiki xenharmonicwiki /home/reception/current.xml on mw1
[08:15:30] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master
[08:26:25] <icinga-miraheze>	 PROBLEM - Disk Space on db4 is WARNING: DISK WARNING - free space: / 76900 MB (20% inode=95%):  
[08:26:30] <Not-683e>	 [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNfg3
[08:26:31] <Not-683e>	 [02miraheze/mw-config] 07Reception123 039c69d57 - namespaces for wikibookwiki per T3334 resolves T3334
[08:37:05] <Reception123>	 ^ paladox importing a (very) large wiki, so db4 space is bound to go down again
[08:41:03] <Not-683e>	 [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNf23
[08:41:05] <Not-683e>	 [02miraheze/mw-config] 07Reception123 03d7c33b8 - rm sitenotice It's been 2 days, images seem to be working now
[08:43:09] <Not-683e>	 [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNf20
[08:43:11] <Not-683e>	 [02miraheze/mw-config] 07Reception123 03a1480a3 - ruberwiki default skin pivot per T3326 resolves T3326
[08:53:05] <icinga-miraheze>	 PROBLEM - JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 5782  
[09:03:35] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is WARNING: WARNING - load average: 3.55, 3.24, 2.86  
[09:05:35] <icinga-miraheze>	 RECOVERY - Current Load on swift1 is OK: OK - load average: 3.14, 3.18, 2.88  
[11:01:05] <icinga-miraheze>	 RECOVERY - JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs  
[11:31:05] <icinga-miraheze>	 PROBLEM - JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 2806  
[12:19:26] <icinga-miraheze>	 PROBLEM - Varnish Backends on cp4 is CRITICAL: 1 backends are down.  mw1  
[12:20:26] <icinga-miraheze>	 PROBLEM - Varnish Backends on cp5 is CRITICAL: 1 backends are down.  mw1  
[12:21:24] <icinga-miraheze>	 RECOVERY - MediaWiki Rendering on mw1 is OK: HTTP OK: HTTP/1.1 200 OK - 30099 bytes in 0.147 second response time  
[12:21:26] <icinga-miraheze>	 RECOVERY - Varnish Backends on cp5 is OK: All 3 backends are healthy  
[12:33:35] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is WARNING: WARNING - load average: 3.53, 3.48, 3.06  
[12:35:35] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is CRITICAL: CRITICAL - load average: 4.09, 3.64, 3.16  
[12:37:35] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is WARNING: WARNING - load average: 3.39, 3.51, 3.17  
[12:45:38] <MH-Discord>	 <NeoMahler> Hi, does piwik have an API to retrieve specific stats such as page views and unique visitors?
[12:46:02] <Reception123>	 paladox: JohnLewis ^ do you know? (I doubt it)
[12:46:25] <JohnLewis>	 yes
[12:47:33] <Reception123>	 JohnLewis: so then would it not be possible for us to create a public service or something without IPs being visible?
[12:47:54] <JohnLewis>	 probably
[12:49:36] <Reception123>	 Maybe we should look into that
[12:50:10] <JohnLewis>	 I have and my comment of "we store all data together so it's impossible"
[12:51:38] <Reception123>	 JohnLewis: well we did manage to separate ATT and all websites in Piwik 
[12:51:57] <JohnLewis>	 because that was done manually
[12:56:23] <Reception123>	 well stats could just be requested as any other feature request
[12:57:10] <JohnLewis>	 you do it right or not at all tbh
[12:57:27] <JohnLewis>	 even ATT don't have access to the stats
[12:57:47] <paladox>	 !log PURGE BINARY LOGS BEFORE DATE(NOW() - INTERVAL 0 DAY) + INTERVAL 0 SECOND; on db4
[12:57:51] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master
[12:58:24] <icinga-miraheze>	 RECOVERY - Disk Space on db4 is OK: DISK OK - free space: / 84868 MB (23% inode=95%):  
[12:58:41] <Reception123>	 JohnLewis: for me if we'd resolve the IPs being public issue, we could just make Piwik accounts for who wants them
[12:58:47] <Reception123>	 JohnLewis: and how else can we really do stats?
[12:58:52] <JohnLewis>	 paladox: is it really too hard to just do "PURGE BINARY LOGS BEFORE '2018-07-09 00:00:00'?
[12:59:07] <JohnLewis>	 well, within MW for a first
[12:59:08] <Reception123>	 paladox: that large import is still going, so wait for more :P
[12:59:09] <paladox>	 JohnLewis nope.
[12:59:16] <JohnLewis>	 otherwise you're not really solving the issue
[12:59:25] <paladox>	 Reception123 oh
[12:59:33] <Reception123>	 well all the already existing extensions have load related issues
[12:59:38] <Reception123>	 6000 (0.35 pages/sec 0.35 revs/sec)
[12:59:39] <JohnLewis>	 and piwik is not too user friendly and hard to get what you need
[12:59:45] <Reception123>	 and I don't think it's close to being done unfortunately, paladox 
[12:59:54] <Reception123>	 yeah, and it's pretty slow too
[12:59:54] <paladox>	 ok
[13:00:18] <Reception123>	 paladox: 750MB xml file so it is going to take a while
[13:00:25] <paladox>	 ah yeh
[13:00:32] <paladox>	 may take a week i guess
[13:00:51] <Reception123>	 lol definitely not a week
[13:01:15] <Reception123>	 paladox: maybe you can help with this tool https://phabricator.miraheze.org/T3302#63014?
[13:01:16] <ZppixBot>	 Title: [ ⚓ T3302 Import Wiki ] - phabricator.miraheze.org
[13:01:45] <paladox>	 Reception123 hmm
[13:02:06] <Reception123>	 we've had many wikispaces users who wanted to migrate so this tool would be really useful
[13:03:02] <JohnLewis>	 https://phabricator.miraheze.org/T2740 ?
[13:03:03] <ZppixBot>	 Title: [ ⚓ T2740 Install Html2Wiki Extension ] - phabricator.miraheze.org
[13:03:32] <paladox>	 JohnLewis we can install it ?
[13:03:57] <JohnLewis>	 no it requires a review
[13:04:05] <paladox>	 ah ok
[13:05:09] <paladox>	 I want to try and setup mathoid for private wiki's.
[13:05:38] <paladox>	 i think that will work anyways.
[13:09:39] <MH-Discord>	 <NeoMahler> I guess that if I ask for an account on the piwik, the answer will be no, right?
[13:09:58] <MH-Discord>	 <NeoMahler> (I already use it for another site, so the user-friendlyness is not a problem)
[13:10:21] <MH-Discord>	 <Reception123> @NeoMahler I would make one, but unfortunately the IP issue makes it a no
[13:10:29] <MH-Discord>	 <Reception123> If we resolve that, I'd have no problem in making one for oyu
[13:10:30] <MH-Discord>	 <Reception123> *you
[13:11:03] <MH-Discord>	 <NeoMahler> Mmh ok
[13:11:46] <MH-Discord>	 <NeoMahler> Would it be possible to at least separate nonbinary.miraheze.org from the rest, so I can ask you the number of visitors every month? :P
[13:14:35] <icinga-miraheze>	 PROBLEM - Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Apt_key[Add key: F86AA916A2195E121AEDB11437BBEE3F7AD95B3F from Apt::Source grafana_apt]  
[13:15:37] <paladox>	 Reception123 ^^ we can do that?
[13:15:49] <paladox>	 by that i am asking for permission
[13:15:51] <paladox>	 or JohnLewis ^^
[13:16:52] <JohnLewis>	 I guess
[13:18:28] <paladox>	 ok
[13:18:47] <Not-683e>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNfNu
[13:18:48] <Not-683e>	 [02miraheze/mw-config] 07paladox 030e0ff6d - Setup piwik for nonbinary
[13:18:50] <paladox>	 NeoMahler ^^ done.
[13:19:25] <paladox>	 JohnLewis i also changed att domain to have https, i have no idea why it was http
[13:22:26] <icinga-miraheze>	 RECOVERY - Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures  
[13:26:26] <JohnLewis>	 8.9k visitors
[13:27:28] <paladox>	 JohnLewis yep, alot!
[13:29:24] <icinga-miraheze>	 RECOVERY - JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs  
[13:30:23] <JohnLewis>	 https://phabricator.miraheze.org/T680 is our oldest open task too
[13:30:24] <ZppixBot>	 Title: [ ⚓ T680 Add statistics interface for founders/users ] - phabricator.miraheze.org
[13:32:03] <paladox>	 heh
[13:32:35] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is WARNING: WARNING - load average: 3.70, 3.44, 3.05  
[13:34:20] <MH-Discord>	 <NeoMahler> paladox, do statistics from previous months appear on piwik, or only from now on?
[13:34:30] <paladox>	 NeoMahler only from now on
[13:35:15] <Wiki-1776>	 Hello
[13:35:29] <MH-Discord>	 <NeoMahler> Ok
[13:40:17] <paladox>	 hi
[13:40:35] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is CRITICAL: CRITICAL - load average: 4.05, 3.67, 3.30  
[13:44:35] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is WARNING: WARNING - load average: 3.67, 3.91, 3.50  
[14:22:56] <icinga-miraheze>	 PROBLEM - Current Load on test1 is WARNING: WARNING - load average: 1.89, 1.56, 1.21  
[14:24:56] <icinga-miraheze>	 RECOVERY - Current Load on test1 is OK: OK - load average: 1.00, 1.42, 1.20  
[14:43:29] <Not-683e>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNJUV
[14:43:30] <Not-683e>	 [02miraheze/mw-config] 07paladox 035682512 - Update FileBackend.php
[15:40:03] <Not-683e>	 [02miraheze/mw-config] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNJYI
[15:40:04] <Not-683e>	 [02miraheze/mw-config] 07MacFan4000 03086a36f - disable anon edits on civclassicwiki
[15:59:36] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is WARNING: WARNING - load average: 3.66, 3.41, 2.97  
[16:01:36] <icinga-miraheze>	 RECOVERY - Current Load on swift1 is OK: OK - load average: 3.36, 3.36, 3.00  
[16:17:25] <icinga-miraheze>	 PROBLEM - JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 2679  
[16:18:53] <Not-683e>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fNJGr
[16:18:55] <Not-683e>	 [02miraheze/puppet] 07paladox 03399077d - Add defaulter.py This file is from https://github.com/tipabu/swift_defaulter/
[16:18:55] <ZppixBot>	 Title: [ GitHub - tipabu/swift_defaulter: Allow the setting of default headers/values for PUT requests in Swift ] - github.com
[16:18:56] <Not-683e>	 [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS
[16:18:58] <Not-683e>	 [02puppet] 07paladox opened pull request 03#774: Add defaulter.py - 13https://git.io/fNJGo
[16:19:43] <Not-683e>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fNJGP
[16:19:44] <Not-683e>	 [02miraheze/puppet] 07paladox 03a3326f5 - Update proxy.pp
[16:19:46] <Not-683e>	 [02puppet] 07paladox synchronize pull request 03#774: Add defaulter.py - 13https://git.io/fNJGo
[16:20:05] <Not-683e>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+1/-0/±0] 13https://git.io/fNJGM
[16:20:07] <Not-683e>	 [02miraheze/puppet] 07paladox 03f24ab59 - Create defaulter.py
[16:20:08] <Not-683e>	 [02puppet] 07paladox synchronize pull request 03#774: Add defaulter.py - 13https://git.io/fNJGo
[16:20:21] <Not-683e>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+1/-0/±0] 13https://git.io/fNJGS
[16:20:22] <Not-683e>	 [02miraheze/puppet] 07paladox 036fc4ee1 - Create LICENSE
[16:20:24] <Not-683e>	 [02puppet] 07paladox synchronize pull request 03#774: Add defaulter.py - 13https://git.io/fNJGo
[16:42:34] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is CRITICAL: CRITICAL - load average: 4.12, 3.49, 2.91  
[16:44:34] <icinga-miraheze>	 RECOVERY - Current Load on swift1 is OK: OK - load average: 3.11, 3.37, 2.93  
[16:53:05] <icinga-miraheze>	 PROBLEM - Puppet on misc4 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[php7.2-gettext]  
[17:03:06] <icinga-miraheze>	 RECOVERY - Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures  
[18:58:11] <Not-683e>	 [02miraheze/mw-config] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNJVU
[18:58:13] <Not-683e>	 [02miraheze/mw-config] 07MacFan4000 0324c4241 - Enable subpages for main js by default Not sure why you wouldn’t subpages in the main NS, given that nearly every entry is to enable subpages in the main ns :)
[19:06:50] <MacFan4000>	 .mangle
[19:07:09] <ZppixBot>	 MacFan4000: Non-683e says: "[02miraheze / mw-config] 07MacFan4000 0324c4241 - Create default sub-pages for basic js I do not know why there are no sub-pages in the main NS because almost all the articles refer to important sub-pages to enable :)
[19:39:10] <ZppixBot>	 Hello longerstaff13! If you have any questions feel free to ask and someone should answer soon.
[19:51:14] <Not-683e>	 [02miraheze/WikiDiscover] 07MacFan4000 deleted branch 03paladox-patch-1
[19:51:15] <Not-683e>	 [02WikiDiscover] 07MacFan4000 deleted branch 03paladox-patch-1 - 13https://git.io/vhUAp
[20:14:13] <Not-683e>	 [02miraheze/ManageWiki] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNJMJ
[20:14:15] <Not-683e>	 [02miraheze/ManageWiki] 07JohnFLewis 03755fc15 - allow URLs in helptips
[20:59:25] <icinga-miraheze>	 RECOVERY - JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs  
[21:31:36] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is WARNING: WARNING - load average: 3.97, 3.01, 2.55  
[21:33:34] <icinga-miraheze>	 RECOVERY - Current Load on swift1 is OK: OK - load average: 2.71, 2.88, 2.56  
[21:51:05] <icinga-miraheze>	 PROBLEM - wiki.pablojprieto.com.es - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds  
[21:54:36] <icinga-miraheze>	 PROBLEM - Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Apt_key[Add key: F86AA916A2195E121AEDB11437BBEE3F7AD95B3F from Apt::Source grafana_apt]  
[22:02:24] <icinga-miraheze>	 RECOVERY - Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures  
[22:09:35] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is WARNING: WARNING - load average: 3.72, 3.47, 3.07  
[23:13:35] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is CRITICAL: CRITICAL - load average: 3.52, 4.00, 4.15  
[23:19:35] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is WARNING: WARNING - load average: 3.17, 3.56, 3.91  
[23:27:36] <icinga-miraheze>	 RECOVERY - Current Load on swift1 is OK: OK - load average: 2.70, 2.75, 3.37  
[23:33:36] <icinga-miraheze>	 PROBLEM - Current Load on swift1 is WARNING: WARNING - load average: 3.53, 3.46, 3.48