[00:00:21] <Not-7f3d>	 [02puppet] 07JohnFLewis commented on pull request 03#811: Remove slow-query-log - 13https://git.io/fAesr
[00:01:04] <Not-7f3d>	 [02puppet] 07paladox synchronize pull request 03#811: Remove slow-query-log - 13https://git.io/fAesV
[00:01:05] <Not-7f3d>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://git.io/fAes6
[00:01:07] <Not-7f3d>	 [02miraheze/puppet] 07paladox 0360c7b05 - Update mw.cnf.erb
[00:01:12] <Not-7f3d>	 [02puppet] 07paladox edited pull request 03#811: Set slow-query-log to 0 - 13https://git.io/fAesV
[00:02:06] <Not-7f3d>	 [02puppet] 07paladox closed pull request 03#811: Set slow-query-log to 0 - 13https://git.io/fAesV
[00:02:07] <Not-7f3d>	 [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAesP
[00:02:09] <Not-7f3d>	 [02miraheze/puppet] 07paladox 03f0fa281 - Remove slow-query-log (#811) Set slow-query-log to 0
[00:02:38] <paladox>	 !log restarting mariadb to apply config change
[00:02:43] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master
[00:03:34] <icinga-miraheze>	 PROBLEM - cp5 Puppet on cp5 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues  
[00:05:50] <icinga-miraheze>	 PROBLEM - cp5 Varnish Backends on cp5 is CRITICAL: 3 backends are down.  mw1 mw2 mw3  
[00:06:18] <icinga-miraheze>	 PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - check plugin (check_jobqueue) or PHP errors  
[00:06:22] <icinga-miraheze>	 PROBLEM - cp2 Varnish Backends on cp2 is CRITICAL: 3 backends are down.  mw1 mw2 mw3  
[00:06:42] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb, 172.104.111.8/cpweb, 2400:8902::f03c:91ff:fe07:444e/cpweb  
[00:07:37] <icinga-miraheze>	 PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb, 172.104.111.8/cpweb, 2400:8902::f03c:91ff:fe07:444e/cpweb  
[00:07:44] <icinga-miraheze>	 PROBLEM - cp2 HTTP 4xx/5xx ERROR Rate on cp2 is WARNING: WARNING - NGINX Error Rate is 51%  
[00:07:51] <icinga-miraheze>	 RECOVERY - cp5 Varnish Backends on cp5 is OK: All 3 backends are healthy  
[00:07:55] <icinga-miraheze>	 RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online  
[00:08:15] <icinga-miraheze>	 RECOVERY - cp4 HTTP 4xx/5xx ERROR Rate on cp4 is OK: OK - NGINX Error Rate is 32%  
[00:08:19] <icinga-miraheze>	 RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs  
[00:08:22] <icinga-miraheze>	 RECOVERY - cp2 Varnish Backends on cp2 is OK: All 3 backends are healthy  
[00:08:43] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online  
[00:09:07] <icinga-miraheze>	 PROBLEM - cp5 HTTP 4xx/5xx ERROR Rate on cp5 is WARNING: WARNING - NGINX Error Rate is 42%  
[00:10:11] <icinga-miraheze>	 RECOVERY - cp2 HTTP 4xx/5xx ERROR Rate on cp2 is OK: OK - NGINX Error Rate is 4%  
[00:11:07] <icinga-miraheze>	 RECOVERY - cp5 HTTP 4xx/5xx ERROR Rate on cp5 is OK: OK - NGINX Error Rate is 0%  
[00:11:07] <paladox>	 !log upgrade phabricator on misc4
[00:11:11] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master
[00:13:33] <icinga-miraheze>	 RECOVERY - cp5 Puppet on cp5 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures  
[01:03:03] <AmandaCatherine>	 Good evening
[01:04:56] <AmandaCatherine>	 I’ve been asked to work the overnight shift for tonight, so I’ll be here in on and off doses over the course of the night
[01:05:16] <AmandaCatherine>	 Not sure what nighttime in Ottawa is equivalent to in Europe or wherever else people are
[01:05:42] <JohnLewis>	 its 2am right now
[01:05:48] <paladox>	 Heh
[01:06:02] <AmandaCatherine>	 JohnLewis: and you’re awake at that time?
[01:06:08] <paladox>	 JohnLewis: be precise it’s 02:06am :D
[01:06:12] <AmandaCatherine>	 Unless you’re working overnight too, shouldn’t you be asleep?
[01:06:28] <paladox>	 AmandaCatherine: we are night owls
[01:06:34] <JohnLewis>	 I never work nigjts
[01:06:50] <AmandaCatherine>	 Oh so you’re nocturnal
[01:06:51] <paladox>	 Heh
[01:06:59] <AmandaCatherine>	 You just sleep during the day because you want to
[01:07:01] <paladox>	 JohnLewis: why?
[01:07:07] <AmandaCatherine>	 And then stay up all night
[01:07:13] <paladox>	 You say you produce you best work st night 
[01:07:37] <JohnLewis>	 paladox: how many schools do you know active at night?
[01:07:52] <paladox>	 JohnLewis: night schools?
[01:07:59] <AmandaCatherine>	 JohnLewis: you’re a teacher?
[01:08:20] <JohnLewis>	 no, could never teach
[01:08:20] <paladox>	 Yes he works in a classroom 
[01:08:28] <paladox>	 That is a jk
[01:08:44] <paladox>	 He works in a office with other staff I presume
[01:10:54] <paladox>	 AmandaCatherine: what time does your shift finish at?
[01:11:35] <AmandaCatherine>	 paladox: on a smooth night, 5AM
[01:11:36] <AmandaCatherine>	 But things don’t always go smoothly
[01:11:43] <paladox>	 Oh lol
[01:11:55] <paladox>	 That is very late.....
[01:12:08] <paladox>	 Doint you have to get up to work during the day too?
[01:12:11] <JohnLewis>	 not for a night shift :P
[01:12:35] <paladox>	 Oh
[01:12:42] <paladox>	 JohnLewis: how would you know?
[01:12:48] <paladox>	 You doint work nights :D
[01:13:16] <JohnLewis>	 a) used to b) on call is night too :P
[01:13:37] <paladox>	 JohnLewis: why did you use to work nights?
[01:13:43] <AmandaCatherine>	 A typical night shift on a smooth night is 9PM to 5AM
[01:13:52] <AmandaCatherine>	 But as I said, things aren’t always smooth
[01:13:54] <JohnLewis>	 university
[01:13:56] <paladox>	 Please do explain why you would be on call
[01:14:10] <paladox>	 JohnLewis: my grandad worked at the uni
[01:16:07] <paladox>	 JohnLewis: why would you need to work late nights at uni?
[01:16:26] <JohnLewis>	 because theyre 24 hours
[01:16:50] <JohnLewis>	 unis are very rarely locked to students
[01:16:56] <paladox>	 JohnLewis: really?
[01:17:09] <paladox>	 Who would be learning at 1am?
[01:17:29] <JohnLewis>	 anyone who chooses to
[01:18:02] <paladox>	 JohnLewis: and so you did lectures at 1am?
[01:18:19] <AmandaCatherine>	 Some people are nocturnal-ish by nature
[01:18:21] <JohnLewis>	 why... would I do lectures?
[01:18:41] <paladox>	 JohnLewis: you said you use to work st the uni at nights
[01:18:52] <AmandaCatherine>	 They work better at night and sleep better during the day
[01:19:13] <JohnLewis>	 yes but I'm not a teacher
[01:19:27] <paladox>	 JohnLewis: what are you then?
[01:19:35] <JohnLewis>	 I've done like a weeks worth of teaching over a 4 year career
[01:20:03] <JohnLewis>	 IT Technican
[01:20:16] <paladox>	 JohnLewis: heh so you were doing computer stuff 
[01:20:19] <paladox>	 Fixing?
[01:20:21] <AmandaCatherine>	 Oh, we have two IT people
[01:20:25] <paladox>	 Or computer coding?
[01:20:51] <JohnLewis>	 fixing, maintenance, software, hatdware etc
[01:21:14] <paladox>	 Heh
[01:21:20] <JohnLewis>	 at the uni, software upgrades could only take place on a Saturday or a Sunday between 1-5am
[01:21:35] <paladox>	 JohnLewis: what were you doing before your 4 year career 
[01:21:39] <AmandaCatherine>	 That’s common of many businesses
[01:21:50] <JohnLewis>	 learning?
[01:21:52] <AmandaCatherine>	 Because that’s the “common time” when nobody is using the system
[01:21:59] <AmandaCatherine>	 And the system can’t upgrade while it is being used
[01:22:13] <paladox>	 JohnLewis: learning?
[01:22:17] <AmandaCatherine>	 I’ve had to explain that to various people and groups hundreds of times
[01:22:45] <paladox>	 AmandaCatherine: you should type “reboot” then say sorry :D
[01:22:45] <AmandaCatherine>	 “I get an error message saying that mandatory updates are available but it won’t install in the background” is something I get a lot
[01:23:19] <JohnLewis>	 paladox: yes, you need qualifications to get good jobs
[01:23:23] <AmandaCatherine>	 I just have to explain to them that in order for the software to be upgraded, people can’t be using it at the same time
[01:23:38] <paladox>	 JohnLewis: yes I know that I’m currently at college :)
[01:24:07] <JohnLewis>	 so why was learning a shock :P
[01:24:34] <paladox>	 JohnLewis: just you manage to progress quite quickly up the ranks
[01:24:49] <PuppyKun>	 +b *!*@*
[01:25:03] <AmandaCatherine>	 PuppyKun ?
[01:25:11] <AmandaCatherine>	 Doesn’t that ban every and any user
[01:25:13] <paladox>	 What
[01:25:18] <paladox>	 No
[01:25:24] <PuppyKun>	 That'll fix the spam.
[01:25:25] <paladox>	 PuppyKun: sorry
[01:25:30] <paladox>	 That was a mistake
[01:25:34] <AmandaCatherine>	 That’s actually what PuppyKun deserves for banning everyone
[01:25:35] <AmandaCatherine>	 :)
[01:25:35] <paladox>	 I put a space
[01:25:45] <paladox>	 And my client ate the white space
[01:25:45] <PuppyKun>	  /ban paladox 
[01:25:48] <JohnLewis>	 2 years in a job before getting into management isn't too shocking in IT if you have the skills and experience before hand
[01:26:03] <paladox>	 JohnLewis: it is to me
[01:26:21] <paladox>	 Young person getting a management job is quite shocking :D
[01:26:55] <JohnLewis>	 not for me :P
[01:27:09] <paladox>	 Heh
[01:27:41] <paladox>	 JohnLewis: do you like bossing people about or do you code?
[01:27:55] <paladox>	 Filling out the paper work will be boring
[01:28:01] <JohnLewis>	 I don't code
[01:28:13] <paladox>	 JohnLewis: why?
[01:28:41] <JohnLewis>	 because you pay people who are better than you to do it
[01:29:34] <paladox>	 JohnLewis: so your saying someone is better then you at coding?
[01:29:43] <paladox>	 Have you seen your php work?
[01:30:03] <JohnLewis>	 I'm a network person
[01:30:30] <paladox>	 JohnLewis: so then you work in traffic :D
[01:31:33] <paladox>	 JohnLewis: I’m not a network person, but then how can you do coding in php?
[01:31:45] <paladox>	 Last I checked networking does not use php
[01:33:16] <JohnLewis>	 because I can do it?
[01:33:36] <paladox>	 JohnLewis: but you are very good at it
[01:33:43] <paladox>	 That is called coding :D
[01:33:55] <paladox>	 Can you write c or c# code?
[01:34:01] <JohnLewis>	 but I dont at work
[01:34:15] <JohnLewis>	 I did a bit of c
[01:34:28] <JohnLewis>	 but not touched it in years
[01:34:38] <paladox>	 Oh
[01:34:41] <AmandaCatherine>	 Yikes! I can move the main page on Miraheze Login Wiki https://i.imgur.com/l45p3Sr.jpg
[01:34:55] <paladox>	 JohnLewis: so you do networking at work, what type of networking?
[01:35:03] <AmandaCatherine>	 That should be fixed ASAP
[01:35:05] <AmandaCatherine>	 I think I can PR a fix if you want me to
[01:35:28] <JohnLewis>	 paladox: guess :P
[01:36:43] <AmandaCatherine>	 JohnLewis: did you see above
[01:36:47] <paladox>	 JohnLewis: errr
[01:36:49] <paladox>	 TCP?
[01:36:54] <paladox>	 WiFi?
[01:36:57] <paladox>	 Ethernet?
[01:37:02] <paladox>	 Udp 
[01:38:03] <paladox>	 AmandaCatherine: needs protection
[01:45:21] <Not-7f3d>	 [02miraheze/mw-config] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAecI
[01:45:23] <Not-7f3d>	 [02miraheze/mw-config] 07JohnFLewis 03b8c3767 - loginwiki: rvk mv
[01:46:03] <JohnLewis>	 paladox: routers, IP management, network connectivity, dns etc
[01:46:13] <paladox>	 Heh
[01:52:33] <AmandaCatherine>	 *sigh* database lag
[01:58:16] * PuppyKun talks about miraheze all the time to family
[01:58:22] <PuppyKun>	 tech savvy cousin: I still have no idea what miraheze is
[01:58:25] <PuppyKun>	 >:(
[01:59:14] <JohnLewis>	 we're called 'heze' now since the WMF decom'd mira
[02:02:56] <PuppyKun>	 anyone have any idea how big att db is recently?
[02:03:38] <JohnLewis>	 37
[02:03:47] <PuppyKun>	 GB?
[02:03:54] <JohnLewis>	 TB
[02:03:58] <PuppyKun>	 kek
[02:12:16] <AmandaCatherine>	 How bout 75+ TB at IT firm headquarters?
[02:12:25] <AmandaCatherine>	 That’s why our databases keep lagging/crashing all the time!
[02:28:58] <AmandaCatherine>	 Err... there’s a typo in the topic
[02:28:58] <AmandaCatherine>	 Welcome to the IRC channel of Miraheze, a free non-profit wiki hosting provider! | https://meta.miraheze.org | Status: INVESTING issues with a server that affects users in Asia  | This channel is publicly logged at https://botbot.me/freenode/miraheze/ | By participating in this channel, you agree to abide by our Code of Conduct: https://meta.miraheze.org/m/Np
[02:28:58] <AmandaCatherine>	 Should be INVESTIGATING
[02:29:01] <ZppixBot>	 Title: [ Meta ] - meta.miraheze.org
[02:29:02] <ZppixBot>	 Title: [ IRC Logs for #miraheze | BotBot.me [o__o] ] - botbot.me
[02:29:03] <ZppixBot>	 Title: [ Code of Conduct - Miraheze Meta ] - meta.miraheze.org
[02:29:03] <AmandaCatherine>	 (Caps for emphasis)
[02:32:21] <PuppyKun>	 thanks
[02:32:57] <AmandaCatherine>	 Np
[02:55:36] <MacFan4000>	 That’s not even true anymore (topic)
[08:32:21] <icinga-miraheze>	 PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 5921  
[10:46:19] <icinga-miraheze>	 RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs  
[13:09:25] <Wiki-1776>	 Hello 
[14:16:21] <icinga-miraheze>	 PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 2865  
[14:50:51] <Zppix>	 `hello
[14:52:04] <paladox>	 Hello zappy 
[14:52:13] <Zppix>	 paladox:  lol
[14:52:19] <paladox>	 :)
[14:57:56] <Zppix>	 paladox: i wish we had more documentation on server specs for mh
[15:00:11] <paladox>	 Zppix: we do
[15:00:18] <Zppix>	 paladox: link?
[15:00:33] <paladox>	 Zppix https://meta.miraheze.org/wiki/Tech:Server_usage
[15:00:33] <ZppixBot>	 Title: [ Tech:Server usage - Miraheze Meta ] - meta.miraheze.org
[15:00:36] <Zppix>	 Ty
[15:25:15] <Reception123>	 ^ paladox  could you please?
[15:25:20] <Reception123>	 I'm on mobile with no access currently
[15:25:38] <Zppix>	 ?
[15:25:47] <paladox>	 Reception123: I carnt
[15:25:56] <Reception123>	 ok
[15:25:59] <Reception123>	 I will have to do it later yet
[15:26:03] <Zppix>	 what you on about :P
[15:26:05] <paladox>	 Remember my vacation starts today :)
[15:26:10] <Reception123>	 *later then 
[15:26:15] <paladox>	 So am mobile only
[15:26:19] <Reception123>	 Zppix: lol you can't see the message because it's moderation
[15:26:20] <Reception123>	 due to the spam
[15:26:24] <Reception123>	 a user is asking us to import some images
[15:26:33] <Zppix>	 oh lol
[15:29:49] <Reception123>	 Zppix: blame the spammers
[15:30:07] * paladox blames zppax
[15:30:12] <paladox>	 Jks
[15:32:45] <Zppix>	 lol
[16:22:19] <icinga-miraheze>	 RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs  
[17:08:57] <icinga-miraheze>	 PROBLEM - encyclopedie.didactiqueprofessionnelle.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'encyclopedie.didactiqueprofessionnelle.org' expires in 15 day(s) (Mon 03 Sep 2018 05:06:11 PM GMT +0000).  
[17:09:09] <Not-7f3d>	 [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAeMQ
[17:09:11] <Not-7f3d>	 [02miraheze/ssl] 07MirahezeSSLBot 033c1b3f5 - Bot: Update SSL cert for encyclopedie.didactiqueprofessionnelle.org
[17:10:58] <icinga-miraheze>	 RECOVERY - encyclopedie.didactiqueprofessionnelle.org - LetsEncrypt on sslhost is OK: OK - Certificate 'encyclopedie.didactiqueprofessionnelle.org' will expire on Fri 16 Nov 2018 04:09:04 PM GMT +0000.  
[17:26:37] <Not-7f3d>	 [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAeDM
[17:26:38] <Not-7f3d>	 [02miraheze/mw-config] 07Reception123 03cfd76c4 - wmincubator settings for T3492
[17:32:08] <Not-7f3d>	 [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/fAeDp
[17:32:09] <Not-7f3d>	 [02miraheze/ssl] 07Reception123 03aae85d0 - add wiki.doverdistrictscouts.co.uk cert
[17:33:22] <Reception123>	 hamza: hey, sorry again for the delay, but the images are importing now
[17:44:51] <Reception123>	 Zppix: lol zppix I was just about to check out Special:RequestWikiQueue/5631
[17:45:00] <Reception123>	 then I just see "Zppix" :44
[17:45:02] <Zppix>	 Reception123:  too slow :P
[17:45:10] <Reception123>	 Zppix: yep, getting too slow for everyone
[17:45:16] <Zppix>	 Reception123:  Zppix :44?
[17:45:19] <Reception123>	 Zppix: in the old days of Miraheze, I was the fastest
[17:45:33] <Reception123>	 Zppix: the minute.. now paladox beats me on GitHub, you beat me on wiki
[17:46:01] <Zppix>	 Reception123:  your just getting old thats all :P jk
[17:46:19] <Reception123>	 Zppix: lol yes
[17:48:25] <Reception123>	 Zppix: how dare you miss https://meta.miraheze.org/wiki/Special:RequestWikiQueue/5624
[17:48:26] <ZppixBot>	 Title: [ Wiki requests queue - Miraheze Meta ] - meta.miraheze.org
[17:48:46] <Reception123>	 Zppix: that will be a 10% cut on your salary
[17:48:50] <Reception123>	 :P
[17:49:55] <Zppix>	 Reception123:  So ill have to pay 10% of 0$ to miraheze now?
[17:51:31] <Reception123>	 Zppix: indeed
[17:52:19] <Zppix>	 Reception123: for every wiki request i do i think my salary should increase 10% :P 
[17:53:14] <Reception123>	 Zppix: you'll have to take that up with the CFO
[17:53:20] <Reception123>	 :D
[17:53:51] <Zppix>	 Reception123:  or you could just slide some money under the doormat?
[17:56:54] * Reception123 slides 0$ to Zppix 
[17:58:50] <Zppix>	 xD
[18:12:18] <paladox>	 Reception123: who’s the cfo?
[18:12:48] <Reception123>	 Labster 
[18:12:58] <paladox>	 Oh
[18:12:59] <Reception123>	 paladox: you really don't like reading docs do you? :P
[19:12:06] <Reception123>	 hamza: You're welcome. And again, sorry for the lateness
[19:12:17] <Reception123>	 hamza: If you need anything else, please feel free to ask me at any time :)
[19:35:09] <Not-7f3d>	 [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAeQN
[19:35:11] <Not-7f3d>	 [02miraheze/services] 07MirahezeSSLBot 03a3e0097 - BOT: Updating services config for wikis
[19:50:11] <Not-7f3d>	 [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAe7w
[19:50:12] <Not-7f3d>	 [02miraheze/services] 07MirahezeSSLBot 03c0c55dc - BOT: Updating services config for wikis
[19:55:09] <Not-7f3d>	 [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAe7Q
[19:55:11] <Not-7f3d>	 [02miraheze/services] 07MirahezeSSLBot 03e43e09f - BOT: Updating services config for wikis
[21:56:33] <Wiki-1776>	 Hello
[21:56:40] <Voidwalker>	 hi