[00:03:07] PROBLEM - misc2 Puppet on misc2 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:11:08] RECOVERY - misc2 Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 27 seconds ago with 0 failures [00:46:00] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:56:45] Voidwalker: I tested your version of the script [00:56:49] https://www.irccloud.com/pastebin/0OzshtVv/ [00:56:50] Title: [ Snippet | IRCCloud ] - www.irccloud.com [00:57:36] one sec [00:59:12] wait, i didn't activate the env [00:59:16] https://www.irccloud.com/pastebin/jLYsoOeF/ [00:59:17] Title: [ Snippet | IRCCloud ] - www.irccloud.com [01:02:44] Voidwalker: ^ [01:03:10] MacFan4000, https://github.com/Pix1234/mh-signbot/commit/7e1dc07f47abcd28c5985822059b823bd3744824#diff-f289368d1ae6d2d58bec9f799832d932 [01:03:11] Title: [ fix indenting · Pix1234/mh-signbot@7e1dc07 · GitHub ] - github.com [01:03:37] ok [01:08:59] https://www.irccloud.com/pastebin/H20JYF0Y/ [01:09:00] Title: [ Snippet | IRCCloud ] - www.irccloud.com [01:10:50] Voidwalker: ^ [01:15:43] hmm it looks like the RC generator automatically ignores bot edits [01:19:16] but it's not making much sense because the docs are saying the opposite [01:20:50] this should filter out bot edits anyway https://github.com/Pix1234/mh-signbot/commit/3a7b21110695f86d124d1126f42c3f7326f3a1b1 [01:20:50] Title: [ ignore bots at the RC gen stage · Pix1234/mh-signbot@3a7b211 · GitHub ] - github.com [01:26:28] https://www.irccloud.com/pastebin/EHazMiJR [01:26:28] Title: [ Snippet | IRCCloud ] - www.irccloud.com [01:28:06] Voidwalker: ^ [01:30:51] now that's frustrating [01:31:07] I was hoping not to touch BotThread [01:38:08] https://github.com/Pix1234/mh-signbot/commit/73fedd19a55cc1f880277ebc9e0316d40114986e [01:38:08] Title: [ attempt to populate revision key · Pix1234/mh-signbot@73fedd1 · GitHub ] - github.com [01:42:01] https://www.irccloud.com/pastebin/fqW3RgsD [01:42:02] Title: [ Snippet | IRCCloud ] - www.irccloud.com [01:42:46] Voidwalker: ^ [01:47:50] I'm going out on a limb with d5ecdbd [01:51:02] Same KeyError [01:51:37] Voidwalker: ^ [02:00:44] oh, I think I've got it now [02:07:29] https://github.com/Pix1234/mh-signbot/commit/f2e7bd384ec0c215171065bdf699610ec38df9ea [02:07:29] Title: [ account for new revision possibly being current · Pix1234/mh-signbot@f2e7bd3 · GitHub ] - github.com [02:10:51] https://www.irccloud.com/pastebin/wOMNttMr [02:10:52] Title: [ Snippet | IRCCloud ] - www.irccloud.com [02:10:58] Voidwalker: ^ [02:12:06] * Voidwalker facepalms for not reading the line numbers [02:13:26] but at the same time, I don't know if it's something on our end [02:22:37] I just ran a similar test that worked so I have no idea [02:33:27] I'll try it again tomorrow [02:33:52] well, work on it again tomorrow [04:10:13] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAGml [04:10:15] [02miraheze/services] 07MirahezeSSLBot 0350bbeb2 - BOT: Updating services config for wikis [06:03:50] PROBLEM - misc4 Puppet on misc4 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[php7.2-gettext] [06:11:51] RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 41 seconds ago with 0 failures [07:14:52] !log reception@mw3:/srv/mediawiki/w/maintenance$ sudo -u www-data php importDump.php --wiki electowikiwiki /home/reception/Electowiki-20180826232629.xml [07:14:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [12:15:12] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAGPd [12:15:13] [02miraheze/services] 07MirahezeSSLBot 03a0840cb - BOT: Updating services config for wikis [12:18:40] Reception123: don’t know if you saw from earlier, but void rewrite the signbot script [12:18:58] It does still need fixes unfortunately [12:19:10] rewrote* [12:20:55] [02puppet] 07CnocBride opened pull request 03#816: Fortnightly backup for schattenvonskeloswiki - 13https://git.io/fAGXg [12:23:33] I remember with dumps we needed to create a cron job I believe it is called? Is that redundant now? [12:24:11] paladox: ^ [12:24:23] Oh, that's great :) [12:24:29] Though we can't use Toolforge [12:24:48] @CnocBride: all you need is the PR [12:24:55] The tool still exists, which is how we’re testing it [12:25:17] [02puppet] 07Reception123 closed pull request 03#816: Fortnightly backup for schattenvonskeloswiki - 13https://git.io/fAGXg [12:25:19] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAGXQ [12:25:20] [02miraheze/puppet] 07CnocBride 036582899 - Fortnightly backup for schattenvonskeloswiki (#816) Per request T3527 (The XML dump process has changed, am I even doing this right?) [12:25:24] WOO [12:25:26] I did it right. [12:25:29] And I didn't fuck up ) [12:25:36] MacFan4000: it is against the policy though so I don't know [12:25:49] @CnocBride yeah :D [12:26:52] https://phabricator.miraheze.org/T3527 These guys want the dump emailed to them though. [12:26:53] Title: [ ⚓ T3527 Requesting scheduled backup of https://schattenvonskelos.miraheze.org every 2 weeks ] - phabricator.miraheze.org [12:26:57] Sooo, I believe that is out of my scope. [12:27:13] Reception123: ^^ [12:29:06] That must be reverted then since it's private [12:29:38] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03revert-816-patch-2 [+0/-0/±1] 13https://git.io/fAG1e [12:29:39] [02miraheze/puppet] 07Reception123 03ee4e6c6 - Revert "Fortnightly backup for schattenvonskeloswiki (#816)" This reverts commit 6582899706b45b53cfc065636138ade54e95fc63. [12:29:41] [02puppet] 07Reception123 created branch 03revert-816-patch-2 - 13https://git.io/vbiAS [12:29:47] [02puppet] 07Reception123 opened pull request 03#817: Revert "Fortnightly backup for schattenvonskeloswiki" - 13https://git.io/fAG1v [12:29:53] [02puppet] 07Reception123 closed pull request 03#817: Revert "Fortnightly backup for schattenvonskeloswiki" - 13https://git.io/fAG1v [12:29:55] [02miraheze/puppet] 07Reception123 pushed 032 commits to 03master [+0/-0/±2] 13https://git.io/fAG1J [12:29:56] [02miraheze/puppet] 07Reception123 03e79c001 - Merge pull request #817 from miraheze/revert-816-patch-2 Revert "Fortnightly backup for schattenvonskeloswiki" [12:30:00] [02puppet] 07Reception123 deleted branch 03revert-816-patch-2 - 13https://git.io/vbiAS [12:30:02] [02miraheze/puppet] 07Reception123 deleted branch 03revert-816-patch-2 [12:37:51] MacFan4000: yep [12:38:04] It’s in a yaml file now so really easy to add [12:38:10] Just need to pick which one [12:38:16] Ok [12:54:25] Why is normal fit so bad with binary files [12:54:59] Private dumps [12:55:08] Can be added to our private repo [12:55:11] On puppet1 [12:55:33] Started using phabricator as a lfs store. Old repo: 853 mb on host after very aggressive gc. New repo: 59 mb including phabricator file storage [12:55:44] Lol [12:55:46] (Not here btw) [12:55:53] PuppyKun: who’s using phab for lfs? [12:55:57] Me [12:56:05] And my dad I suppose in the near future [12:56:20] Heh [12:56:38] PuppyKun: your dad is helping you maintain your phab infrastructure? [12:56:48] Not at all [12:57:03] He knows the basics of git which we now use instead of svn. I maintain the server stuff [12:57:11] Heh [12:57:18] PuppyKun: is this a family server? [12:57:28] Work [12:57:45] Ok [12:57:57] Why are you using repos at your work? [12:58:08] You host the source code in there? [12:58:12] But I've been having both of us use normal git without lfs since may 2017. Theres only like 130 commits across like 30 branches but I somehow managed to turn like 4 mb into 800 mb [12:58:17] Is the repo publicly accessible? [12:58:20] Lots of stuff? Why does anyone use version control [12:58:21] Or even phab? [12:58:24] And no it's not public [12:58:26] Yes its phab [12:58:44] PuppyKun: I mean is phab public :D [12:59:18] I use git repos for phab, mediawiki, open-source but not publicly accessible source code, binary plc control files, binary inspection files, and program config files [12:59:22] Not this instance of phab [12:59:49] Heh [13:00:02] PuppyKun: did you apply customisations to your install? [13:00:35] To phab? Not outside of setting some of the config vars and enabling/disabling applications. theres no custom code [13:00:47] Ah ok [13:01:10] I'm very confused as to how over 850 mb is now less than 60 mb [13:01:15] The entire history is still there [13:01:18] I applied customisations to wmf one (adding a github download button) (with help from the wmf phab maintainer) [13:01:25] PuppyKun: easy [13:01:28] Objects [13:01:36] Loose references too [13:02:04] But the objects still exist and the old repo (the 850 mb one) should not have any objects the new one doesnt [13:02:08] If you do not run regularly gc then you will find it happening again [13:02:31] The old repo is 850 mb after deleting the entire reflog and running a very aggressive gc [13:02:37] PuppyKun: So all the object files still exist? [13:02:57] Yes they're just not stored in the .git folder they're in phab file store [13:03:13] But it went from 850 mb in one place to 3 mb repo + 59 mb file store [13:03:18] Still doesnt add up [13:03:20] They should be in .git [13:03:32] Not with an external lfs? [13:03:35] Maybe? [13:03:36] Oh [13:03:38] Lfs [13:03:44] Yes [13:03:44] Yeh that will be in the file thingy [13:03:55] But even the new file thingy isnt the size of the old repo [13:04:00] I’ve never used lfs in phab [13:04:15] I think the old repo was maybe saving pointless diffs of binary files (I had no gitattribs or anything) [13:04:45] Hmm [14:48:08] Hello [14:48:30] Hi [14:50:53] hi [14:52:38] Heellooo [14:56:50] Hi [14:57:09] Hi everyone [14:57:46] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 2988 [15:25:01] Don’t know if someone can try to decode https://phabricator.miraheze.org/T3512 [15:25:03] Title: [ ⚓ T3512 Adding Open Graph data to HTML ] - phabricator.miraheze.org [15:25:04] I can’t figure it out [15:26:03] Reception123: were you going to do something here? https://phabricator.miraheze.org/T3505#66914 [15:26:03] Title: [ ⚓ T3505 Extension:Moderation ] - phabricator.miraheze.org [15:26:07] If not I think the task is resolved [15:28:41] [02CreateWiki] 07Southparkfan edited pull request 03#85: Rewrite manageInactiveWikis.php - 13https://git.io/fAsYx [15:35:09] !log storing sql dump of mussmanwissenwiki and removing from Miraheze & ban.req on cp* for the domain [15:35:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [15:59:49] *wired something and it no longer works* [16:11:32] [02MatomoAnalytics] 07Southparkfan created branch 03Southparkfan-patch-1 - 13https://git.io/fN4LT [16:11:33] [02miraheze/MatomoAnalytics] 07Southparkfan pushed 031 commit to 03Southparkfan-patch-1 [+0/-0/±1] 13https://git.io/fAZqR [16:11:35] [02miraheze/MatomoAnalytics] 07Southparkfan 032df3069 - Tidy up MatomoAnalytics class [16:12:23] Hmm.. increasingly argumentative responses here https://meta.miraheze.org/wiki/Requests_for_global_rights#Pkbwcgs.27s_Request_for_CVT [16:12:58] And then immediately applying for wiki creator while nearly copy-pasting the suggested requirements https://meta.miraheze.org/wiki/Community_noticeboard#Request_for_Wiki_creator [16:13:05] Why do I sense hat collecting here? [16:13:21] [02miraheze/MatomoAnalytics] 07Southparkfan pushed 031 commit to 03Southparkfan-patch-1 [+0/-0/±1] 13https://git.io/fAZqP [16:13:22] [02miraheze/MatomoAnalytics] 07Southparkfan 03ce44e1c - Update extension.json [16:13:24] [02MatomoAnalytics] 07Southparkfan opened pull request 03#5: Tidy up MatomoAnalytics class - 13https://git.io/fAZqX [16:13:25] miraheze/MatomoAnalytics/Southparkfan-patch-1/2df3069 - Southparkfan The build passed. https://travis-ci.org/miraheze/MatomoAnalytics/builds/422147015 [16:23:43] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [16:25:35] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZY4 [16:25:37] [02miraheze/puppet] 07paladox 037711149 - Update matomo to 3.6.0 [16:33:08] PROBLEM - misc2 Puppet on misc2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_checkout_piwik] [16:38:10] RECOVERY - misc2 Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [16:40:07] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZOA [16:40:08] [02miraheze/puppet] 07paladox 037e8a380 - Piwik: set force_ssl = 1 [17:25:25] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZca [17:25:27] [02miraheze/puppet] 07Southparkfan 0399d2178 - install piwik on test1 T3518 [17:26:35] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZci [17:26:37] [02miraheze/puppet] 07Southparkfan 03b80398e - Remove role, webserver conflict [17:27:40] !log disable puppet on cp4 [17:27:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:27:59] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [17:28:35] !log disable puppet on misc2 [17:28:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:29:55] PROBLEM - cp4 Puppet on cp4 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 8 minutes ago with 0 failures [17:31:07] PROBLEM - misc2 Puppet on misc2 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 10 minutes ago with 0 failures [17:37:24] Socks on Weather Wiki: https://weather.miraheze.org/wiki/Special:Contributions/NOAA and https://weather.miraheze.org/wiki/Special:Contributions/The_Weather_Channel [17:37:26] Title: [ Login required - Weather Wiki ] - weather.miraheze.org [17:37:27] Title: [ Login required - Weather Wiki ] - weather.miraheze.org [17:37:38] JohnLewis PuppyKun SPF|Cloud Voidwalker please CU ^ [17:37:40] PROBLEM - cp4 Varnish Backends on cp4 is WARNING: No backends detected. If this is an error, see readme.txt [17:39:00] let's see.. [17:39:45] Hello SPF|Cloud AmandaCatherine :) [17:39:56] Hi Wiki-1776 [17:45:21] actually, nevermind, I'll let someone else handle it [17:45:51] hi [17:46:06] Hi Voidwalker [17:51:15] !log restarted misc2 apache [17:51:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [18:01:55] RECOVERY - cp4 Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:02:35] * paladox wonders what changes SPF|Cloud was testing on cp4 and misc2 :) [18:02:49] !log enable puppet on cp4 and misc2 [18:02:51] T3518 [18:02:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [18:03:07] RECOVERY - misc2 Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:03:26] but nginx makes replacing cp4's IP with the real IP so much easier when it comes to PHP handling as well [18:03:41] RECOVERY - cp4 Varnish Backends on cp4 is OK: All 3 backends are healthy [18:04:17] I think replacing apache with nginx takes less effort and is the better long term choice [18:05:37] Heh, I just mass imported pages from my WikiCanada archive file, and some of them imported with the original author being one of LP’s old socks :P [18:06:55] Also all of the years are now inaccurate (everything from 2017 is now stamped as 2018) [18:06:58] But meh, whatever [18:10:31] [02miraheze/MatomoAnalytics] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZ41 [18:10:33] [02miraheze/MatomoAnalytics] 07MacFan4000 03890e196 - Update README.md [18:12:49] SPF|Cloud: +1 [18:13:59] Wiki-1776: hola [18:14:50] AlvaroMolina: estas? [18:14:56] hola Reception123 :) [18:15:24] Wiki-1776: "analytics" en español es analitica, verdad? [18:20:28] Reception123: Lo he visto traducido como "Análisis" siempre acompañado de otra palabra [18:21:05] Wiki-1776: De verdad? Analytics and analysis are different things :P [18:21:44] Pero analitica sería la traducción correcta [18:21:54] lo siento Reception123 tengo que salir [18:22:38] [02miraheze/MatomoAnalytics] 07Reception123 pushed 032 commits to 03master [+4/-0/±1] 13https://git.io/fAZBj [18:22:39] [02miraheze/MatomoAnalytics] 07Reception123 03e49f32b - add a few translations (ro,fr,de,es) [18:22:41] [02miraheze/MatomoAnalytics] 07Reception123 03aa55f8a - Merge branch 'master' of https://github.com/miraheze/MatomoAnalytics [18:22:41] Title: [ GitHub - miraheze/MatomoAnalytics: MediaWiki extension for integration with matomo for analytics ] - github.com [18:22:54] blame the second part of the commit on MacFan4000 :D [18:23:20] Wiki-1776: Ok, se puede arreglar mas tarde. Adios [18:23:22] Heh [18:24:21] Pulling the extension now and hoping since it's a copy/paste that there won't be JSON errors [18:24:59] and that there's no Spanish mistakes :P [18:25:55] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_31 [+0/-0/±1] 13https://git.io/fAZR4 [18:25:56] [02miraheze/mediawiki] 07Reception123 03979c3b5 - Update MatomoAnalytics [18:28:18] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZRi [18:28:19] [02miraheze/puppet] 07Southparkfan 0361cab17 - Fix role removal from wrong server [18:31:05] !log sudo -u www-data php rebuildLocalisationCache.php --wiki test1wiki on mw* [18:31:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [18:33:31] MacFan4000: yay https://es.publictestwiki.com/wiki/Especial:Analytics [18:33:31] Title: [ MatomoAnalytics - PruebaWiki ] - es.publictestwiki.com [18:33:49] JohnLewis: I wonder, can we translate the individual vars though? [18:34:22] which? [18:34:35] I wonder if it’s a good idea for everyone to be able to see the analytics [18:34:54] Agreed ^ [18:35:14] I have also wondered that [18:35:20] JohnLewis: the stuff like "Browser Usage", etc. [18:35:31] but I guess that's controlled by Matomo so I'd have to ask there? [18:35:40] It’s very easy to restrict it [18:35:43] As long as it dosent get Users ip or location [18:36:14] Or any info that you could use to identify them, then we are fine. [18:36:24] MacFan4000: yeah, it's been discussed in staff when it was first advertised, and John and SPF said there was no reason for it to be restricted [18:36:33] Ok [18:38:00] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:39:05] Hay discrepancias en la traducción, revisando veo que han traducido Analytic function a Función analítica, Web analytics a Web analytics Reception123 [18:39:29] en eswikipedia* [18:39:54] Análisis web* [18:40:49] Oh, tal vez podrias ayudar a solucionarlos? [18:41:04] JohnLewis: so is the stuff like "Browser Usage", etc. controlled by Upstream? [18:41:09] (as in translation-wise) [18:43:18] Uh [18:43:35] It’s probably translated to site language [18:46:10] JohnLewis: unfortunately nope [18:46:14] https://es.publictestwiki.com/wiki/Especial:Analytics [18:46:15] Title: [ MatomoAnalytics - PruebaWiki ] - es.publictestwiki.com [18:46:18] only my i18n changes were, not the actual vars [18:46:38] No veo problema que se traduzca como Especial:Analítica (y quizás haciendo una redirección creando Especial:Análisis) [18:47:57] see image https://prnt.sc/kod3dx of Matomo [18:47:57] Title: [ Screenshot by Lightshot ] - prnt.sc [18:48:47] pero si gustas, podemos esperar la opinión de AlvaroMolina [18:52:50] [02mw-config] 07Amanda-Catherine opened pull request 03#2407: Userrights changes for weatherwiki #1 - 13https://git.io/fAZum [18:54:00] [02mw-config] 07Amanda-Catherine opened pull request 03#2408: Userrights changes for weatherwiki #2 - 13https://git.io/fAZuW [18:57:00] [02mw-config] 07MacFan4000 closed pull request 03#2408: Userrights changes for weatherwiki #2 - 13https://git.io/fAZuW [18:57:01] [02miraheze/mw-config] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZuo [18:57:03] [02miraheze/mw-config] 07Amanda-Catherine 0357141a7 - Userrights changes for weatherwiki #2 (#2408) [18:57:10] [02mw-config] 07MacFan4000 closed pull request 03#2407: Userrights changes for weatherwiki #1 - 13https://git.io/fAZum [18:57:11] [02miraheze/mw-config] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZu6 [18:57:13] [02miraheze/mw-config] 07Amanda-Catherine 030321daf - Userrights changes for weatherwiki #1 (#2407) [18:58:02] !m MacFan4000 [18:58:03] <[d__d]> You're doing good work, MacFan4000! [19:03:32] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+2/-2/±2] 13https://git.io/fAZzc [19:03:34] [02miraheze/puppet] 07Southparkfan 03577d75c - Prepare nginx module for misc sites [19:04:03] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki config] [19:05:33] * SPF|Cloud is going to move misc2 to nginx now :) [19:05:45] SPF|Cloud: great :) [19:05:51] !log disable misc2 puppet [19:05:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [19:09:08] PROBLEM - misc2 Puppet on misc2 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 8 minutes ago with 0 failures [19:09:40] SPF|Cloud: you should put “SPF” in the disable message :) [19:09:51] eh right [19:10:27] paladox: meh, he logged it so I think it's fine [19:10:42] Yep [19:10:51] yeah, but if paladox prefers it sure [19:11:20] SPF|Cloud: oh I doint mind just JohnLewis told us to do that :) [19:11:29] ah okay [19:11:44] SPF|Cloud: it was mostly for testing on test1, so we could know who's doing what [19:11:53] since no logging was required for what we were doing there [19:11:55] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/fAZgf [19:11:57] [02miraheze/puppet] 07Southparkfan 036e7c266 - use misc2 as piwik backend [19:13:00] heh? [19:13:54] PROBLEM - misc3 Puppet on misc3 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [19:13:56] PROBLEM - cp4 Puppet on cp4 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [19:14:17] if that's referring to what I said, multiple sysadmins were disabling Puppet to edit files locally, and it got confusing so that's why we used names in the reason [19:14:22] PROBLEM - cp5 Puppet on cp5 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [19:14:51] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+0/-1/±1] 13https://git.io/fAZg4 [19:14:53] [02miraheze/puppet] 07Southparkfan 032f59551 - kill nginx.conf.erb in varnish module [19:15:16] You need .probe SPF|Cloud I think [19:15:38] misc2 has no redundancy so a probe is not useful [19:15:59] Hmm ok [19:16:06] SPF|Cloud: why’s puppet failing? :) [19:16:12] SPF|Cloud: um since you're doing stuff with misc/Piwik I think it's about time we also change it from piwik.mh.o to matomo.mh.o [19:16:19] because there were double nginx.conf.erb stuff [19:16:30] (since you said there was an issue with doing it before) [19:16:30] Ah ok [19:17:30] Reception123: if you can get a PR for mw-config and DNS ready, sure [19:17:43] And puppet and mm [19:17:50] And ma [19:17:54] RECOVERY - cp4 Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [19:18:11] MA no, as that’s controles by wg cars [19:18:11] there is a config var for that in mw-config I guess? also /me will do puppet as necessary [19:18:16] vars* [19:18:33] ok [19:19:04] MM I don’t know what needs a change, since ma adds the tracking code [19:19:28] Reception123: search for piwik.miraheze.org in the miraheze organisation [19:19:34] Should reveal where it is in all the places [19:19:41] if someone can add matomo.miraheze.org to DNS immediately, please [19:20:28] Reception123: ^^ [19:20:45] SPF|Cloud: ok, I'll do DNS first now and then the Puppet PR [19:20:55] puppet can wait [19:21:08] I need to write the nginx configuration first [19:21:10] [02dns] 07MacFan4000 opened pull request 03#95: Update miraheze.org - 13https://git.io/fAZ2L [19:21:14] SPF|Cloud: Do I also leave Piwik? [19:21:29] Reception123: SPF|Cloud you can merge that [19:21:32] keep piwik.mh.o intact please [19:21:46] Kay [19:21:46] [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZ2s [19:21:48] [02miraheze/dns] 07Reception123 03715c6a9 - add matomo [19:21:54] also they should point to the varnish lb [19:21:54] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 41 seconds ago with 0 failures [19:22:00] [02dns] 07MacFan4000 closed pull request 03#95: Update miraheze.org - 13https://git.io/fAZ2L [19:22:03] MacFan4000: ah that doesn't work since we also need Piwik for now [19:22:20] RECOVERY - cp5 Puppet on cp5 is OK: OK: Puppet is currently enabled, last run 9 seconds ago with 0 failures [19:22:26] SPF|Cloud: oh, varnish lb? [19:22:37] In dns [19:22:39] In the config [19:22:56] Which points it to varnish [19:23:00] it should be like this: piwik DYNA geoip!mw [19:23:13] ah ok so like we do custom domains [19:23:20] Replace mw with misc2 an define misc2 in the config [19:23:22] (and eventually 'mw' should be renamed as it cp won't be mediawiki-only anymore..) [19:23:40] [02miraheze/mw-config] 07MacFan4000 pushed 031 commit to 03MacFan4000-patch-1 [+0/-0/±1] 13https://git.io/fAZ2z [19:23:41] [02miraheze/mw-config] 07MacFan4000 034a2d8d2 - Update LocalSettings.php [19:23:43] [02mw-config] 07MacFan4000 created branch 03MacFan4000-patch-1 - 13https://git.io/vbvb3 [19:23:44] [02mw-config] 07MacFan4000 opened pull request 03#2409: Update LocalSettings.php - 13https://git.io/fAZ2g [19:23:45] Oh mw the [19:24:05] SPF|Cloud: Do I only point Matomo to the LB, or also Piwik? [19:24:06] And ^^ is now ready [19:24:09] both [19:24:14] ok [19:24:30] Reception123: or, don't do anything because I'm half way through it [19:24:34] [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZ26 [19:24:36] [02miraheze/dns] 07Reception123 034d93af8 - point Matomo/Piwik to varnish LB [19:24:44] JohnLewis: whoops sorry [19:24:45] Lol [19:24:52] I just pushed literally exactly when I heard the ping sound [19:24:54] so it was too late :( [19:25:04] miraheze/mw-config/MacFan4000-patch-1/4a2d8d2 - MacFan4000 The build was fixed. https://travis-ci.org/miraheze/mw-config/builds/422225827 [19:25:17] Reception123: when someone pings you does it make sounds? [19:25:26] Mine pop ups [19:25:33] I would never enable sound notifications [19:25:36] before we used to have trouble committing and merging stuff now we're all battling for who merges what first and who commits first [19:25:43] paladox: I have both I'm special :P [19:25:50] Lol [19:25:53] miraheze/dns/master/4d93af8 - Reception123 The build was broken. https://travis-ci.org/miraheze/dns/builds/422226124 [19:25:59] and yeah [19:26:04] since when does Travis CI do DNS? [19:26:04] that's a critical failure [19:26:10] and how can it be broken? [19:26:20] geoip!misc2 doesn't exist [19:26:24] paladox: ^ [19:26:44] Change it to mw [19:26:46] Reception123: [19:26:54] [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZ2N [19:26:54] reverting as I really don't want a critical [19:26:55] [02miraheze/dns] 07Reception123 0306925ee - Revert "point Matomo/Piwik to varnish LB" This reverts commit 4d93af88b71f2bc3727bf1af8132540a97acfee0. [19:27:00] I'll do it myself [19:27:03] grr [19:27:08] sorry for the mess [19:27:09] It was a easy fix Reception123 [19:27:10] I'll just do the Puppet PR [19:27:16] more commits I have to put back on top of my work [19:27:17] revi: around? [19:27:21] paladox: yeah but a DNS failure would not be nice [19:27:47] Ok [19:27:53] [02dns] 07Southparkfan created branch 03Southparkfan-patch-1 - 13https://git.io/vbQXl [19:27:55] [02miraheze/dns] 07Southparkfan pushed 031 commit to 03Southparkfan-patch-1 [+0/-0/±1] 13https://git.io/fAZ2j [19:27:56] [02miraheze/dns] 07Southparkfan 030c722c9 - Switch piwik/matomo.mh.o to cache proxies [19:28:16] [02miraheze/dns] 07Southparkfan pushed 031 commit to 03Southparkfan-patch-1 [+0/-0/±1] 13https://git.io/fAZaf [19:28:18] [02miraheze/dns] 07Southparkfan 038f87896 - and update the serial [19:28:33] oh that indenting is terrible [19:28:35] miraheze/dns/master/06925ee - Reception123 The build was fixed. https://travis-ci.org/miraheze/dns/builds/422227187 [19:29:36] [02dns] 07Southparkfan opened pull request 03#96: Switch piwik/matomo.mh.o to cache proxies - 13https://git.io/fAZaO [19:30:04] miraheze/dns/Southparkfan-patch-1/8f87896 - Southparkfan The build passed. https://travis-ci.org/miraheze/dns/builds/422227715 [19:30:45] [02miraheze/dns] 07Southparkfan pushed 031 commit to 03Southparkfan-patch-1 [+0/-0/±1] 13https://git.io/fAZaG [19:30:47] [02miraheze/dns] 07Southparkfan 03c3a7d3b - fix terrible indenting?? [19:30:48] [02dns] 07Southparkfan synchronize pull request 03#96: Switch piwik/matomo.mh.o to cache proxies - 13https://git.io/fAZaO [19:31:12] that didn't fix it.. whatever [19:31:18] about to sleep [19:31:23] I sense someone pinged me [19:31:34] MacFan4000: sup [19:31:53] [02puppet] 07Reception123 created branch 03matomo - 13https://git.io/vbiAS [19:31:55] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03matomo [+5/-5/±5] 13https://git.io/fAZaz [19:31:56] [02miraheze/puppet] 07Reception123 03070ff5d - Piwik -> Matomo [19:32:12] [02puppet] 07Reception123 opened pull request 03#818: Piwik -> Matomo - 13https://git.io/fAZaa [19:32:48] SPF|Cloud: ^ for when it's necessary [19:33:10] revi: MacFan4000 was going to ask about adding an extension to translatewiki.net (since MirahezeMagic has many i18n strings now we want to add it) [19:33:25] but if you're going to sleep perhaps tomorrow would be a better time [19:33:26] your changes include renaming the database, renaming the db user, renaming the directories and renaming the class [19:33:41] does that not also need to be done? Having a db with another name would be confusing [19:33:41] one step at a time! ;) [19:33:47] SPF|Cloud: hint; the quicker you merge your DNS change, the quicker I can get back to doing my DNS stuff :P [19:33:55] SPF|Cloud: ok, we can create separate PRs then :) no problem [19:33:59] [02puppet] 07Reception123 assigned pull request 03#818: Piwik -> Matomo - 13https://git.io/fAZaa [19:34:01] Reception123: phab.miraheze task with me assigned can work [19:34:08] ^ MacFan4000 [19:34:09] JohnLewis: just do your stuff and I'll fix the merge conflicts for my change [19:34:23] SPF|Cloud: well my change involves your work [19:34:40] SPF|Cloud: the thing is, if we're moving to NGINX, wouldn't it be pretty pointless for me to do stuff with apache conf? [19:34:40] just if you merge your change, it removes the time pressure for me to do my stuff [19:34:49] yes [19:35:15] so I guess this should probably be done after the merge [19:35:28] since without db apache is basically the only thing that needs to be changed [19:35:30] Someone is going to have to copy the piwik db to be matomo [19:35:36] Per Reception123 change to grants [19:35:55] yeah, though as SPF said it's better to do one thing at a time :) [19:35:58] so we can do that later then [19:51:32] PROBLEM - cp5 HTTP on cp5 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:51:35] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+2/-1/±2] 13https://git.io/fAZw6 [19:51:37] [02miraheze/puppet] 07Southparkfan 03145e776 - migrate piwik module to nginx [19:51:40] PROBLEM - cp5 Disk Space on cp5 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:51:42] PROBLEM - cp5 HTTP 4xx/5xx ERROR Rate on cp5 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:51:44] PROBLEM - cp5 SSH on cp5 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:51:52] PROBLEM - cp5 HTTPS on cp5 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:52:02] PROBLEM - cp5 Current Load on cp5 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:52:06] let's see what happens [19:52:28] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 2 datacenters are down: 172.104.111.8/cpweb, 2400:8902::f03c:91ff:fe07:444e/cpweb [19:52:32] PROBLEM - cp5 Varnish Backends on cp5 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:52:40] !log migrating piwik to nginx, intermittent service for now [19:52:46] PROBLEM - cp5 Puppet on cp5 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:52:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [19:53:02] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 172.104.111.8/cpweb, 2400:8902::f03c:91ff:fe07:444e/cpweb [19:53:36] PROBLEM - Host cp5 is DOWN: PING CRITICAL - Packet loss = 100% [19:55:08] RECOVERY - misc2 Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 58 seconds ago with 0 failures [19:55:55] [02dns] 07Southparkfan closed pull request 03#96: Switch piwik/matomo.mh.o to cache proxies - 13https://git.io/fAZaO [19:55:56] [02miraheze/dns] 07Southparkfan pushed 034 commits to 03master [+0/-0/±4] 13https://git.io/fAZrf [19:55:58] [02miraheze/dns] 07Southparkfan 03236cf2e - Merge pull request #96 from miraheze/Southparkfan-patch-1 Switch piwik/matomo.mh.o to cache proxies [20:00:39] [02mw-config] 07Southparkfan created branch 03Southparkfan-patch-1 - 13https://git.io/vbvb3 [20:00:40] [02miraheze/mw-config] 07Southparkfan pushed 031 commit to 03Southparkfan-patch-1 [+0/-0/±1] 13https://git.io/fAZr0 [20:00:41] [02miraheze/mw-config] 07Southparkfan 03899eee7 - piwik.miraheze.org -> matomo.miraheze.org [20:00:43] [02mw-config] 07Southparkfan opened pull request 03#2410: piwik.miraheze.org -> matomo.miraheze.org - 13https://git.io/fAZrE [20:00:55] need to wait until 22:05 due to dns cache [20:01:40] [02miraheze/dns] 07JohnFLewis pushed 032 commits to 03master [+0/-0/±59] 13https://git.io/fAZrw [20:01:41] [02miraheze/dns] 07JohnFLewis 031746dd5 - mw->cp [20:01:43] [02miraheze/dns] 07JohnFLewis 03444ea2f - Merge branch 'master' of github.com:miraheze/dns [20:02:36] it's really great to realise I've finally managed to get literally everything in https://git.io/fAZw6 right in just one attempt [20:02:37] Title: [ Comparing 2f595519e577...145e77675f87 · miraheze/puppet · GitHub ] - git.io [20:03:00] I was expecting something to fail but not this time :P [20:03:03] [02miraheze/dns] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZr6 [20:03:05] [02miraheze/dns] 07JohnFLewis 03b4c983b - commit missing config [20:03:05] miraheze/dns/master/444ea2f - John Lewis The build was broken. https://travis-ci.org/miraheze/dns/builds/422241509 [20:03:55] 183 unread messages... [20:03:56] :P [20:04:38] miraheze/dns/master/b4c983b - John Lewis The build was fixed. https://travis-ci.org/miraheze/dns/builds/422242073 [20:04:53] [02miraheze/dns] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZrS [20:04:55] [02miraheze/dns] 07JohnFLewis 035c55c3e - take cp5 down [20:05:06] [02mw-config] 07Southparkfan closed pull request 03#2410: piwik.miraheze.org -> matomo.miraheze.org - 13https://git.io/fAZrE [20:05:08] [02miraheze/mw-config] 07Southparkfan pushed 032 commits to 03master [+0/-0/±2] 13https://git.io/fAZr9 [20:05:09] [02miraheze/mw-config] 07Southparkfan 031f15ae1 - Merge pull request #2410 from miraheze/Southparkfan-patch-1 piwik.miraheze.org -> matomo.miraheze.org [20:08:18] [02miraheze/mw-config] 07Southparkfan pushed 031 commit to 03revert-2410-Southparkfan-patch-1 [+0/-0/±1] 13https://git.io/fAZoq [20:08:19] [02miraheze/mw-config] 07Southparkfan 032d88ca4 - Revert "piwik.miraheze.org -> matomo.miraheze.org" [20:08:21] [02mw-config] 07Southparkfan created branch 03revert-2410-Southparkfan-patch-1 - 13https://git.io/vbvb3 [20:08:22] [02mw-config] 07Southparkfan opened pull request 03#2411: Revert "piwik.miraheze.org -> matomo.miraheze.org" - 13https://git.io/fAZom [20:08:26] JohnLewis: why are we taking cp5 down? [20:08:27] [02mw-config] 07Southparkfan closed pull request 03#2411: Revert "piwik.miraheze.org -> matomo.miraheze.org" - 13https://git.io/fAZom [20:08:28] [02miraheze/mw-config] 07Southparkfan pushed 032 commits to 03master [+0/-0/±2] 13https://git.io/fAZoY [20:08:30] [02miraheze/mw-config] 07Southparkfan 03dfc7085 - Merge pull request #2411 from miraheze/revert-2410-Southparkfan-patch-1 Revert "piwik.miraheze.org -> matomo.miraheze.org" [20:08:34] SPF|Cloud: why revert? [20:08:38] paladox: see email [20:08:45] something's wrong with internal handling [20:09:13] Oh [20:09:18] JohnLewis: I see now [20:09:43] miraheze/mw-config/revert-2410-Southparkfan-patch-1/2d88ca4 - Southparkfan The build passed. https://travis-ci.org/miraheze/mw-config/builds/422244590 [20:10:22] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZol [20:10:24] [02miraheze/puppet] 07Southparkfan 03d91ddea - also forward matomo.mh.o to misc2 [20:14:06] [02miraheze/mw-config] 07Southparkfan pushed 031 commit to 03revert-2411-revert-2410-Southparkfan-patch-1 [+0/-0/±1] 13https://git.io/fAZoH [20:14:07] [02miraheze/mw-config] 07Southparkfan 033f0d422 - Revert "Revert "piwik.miraheze.org -> matomo.miraheze.org"" [20:14:09] [02mw-config] 07Southparkfan created branch 03revert-2411-revert-2410-Southparkfan-patch-1 - 13https://git.io/vbvb3 [20:14:12] [02mw-config] 07Southparkfan opened pull request 03#2412: Revert "Revert "piwik.miraheze.org -> matomo.miraheze.org"" - 13https://git.io/fAZoQ [20:14:21] [02mw-config] 07Southparkfan closed pull request 03#2412: Revert "Revert "piwik.miraheze.org -> matomo.miraheze.org"" - 13https://git.io/fAZoQ [20:14:23] [02miraheze/mw-config] 07Southparkfan pushed 032 commits to 03master [+0/-0/±2] 13https://git.io/fAZo7 [20:14:24] [02miraheze/mw-config] 07Southparkfan 035f370d2 - Merge pull request #2412 from miraheze/revert-2411-revert-2410-Southparkfan-patch-1 Revert "Revert "piwik.miraheze.org -> matomo.miraheze.org"" [20:15:04] JohnLewis: if you want, you can kill piwik.mh.o from dns now [20:15:28] miraheze/mw-config/revert-2411-revert-2410-Southparkfan-patch-1/3f0d422 - Southparkfan The build passed. https://travis-ci.org/miraheze/mw-config/builds/422247129 [20:16:24] [02miraheze/dns] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZKU [20:16:25] [02miraheze/dns] 07JohnFLewis 038716d6e - rm piwik [20:16:32] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/puppet/compare/d91ddea98260...0ed921f748e9 [20:16:33] Title: [ Comparing d91ddea98260...0ed921f748e9 · miraheze/puppet · GitHub ] - github.com [20:16:34] [02miraheze/puppet] 07Southparkfan 030ed921f - Kill piwik.miraheze.org from puppet [20:17:37] [02puppet] 07Southparkfan commented on pull request 03#818: Piwik -> Matomo - 13https://git.io/fAZK3 [20:17:39] [02puppet] 07Southparkfan closed pull request 03#818: Piwik -> Matomo - 13https://git.io/fAZaa [20:19:18] RECOVERY - Host cp5 is UP: PING OK - Packet loss = 0%, RTA = 267.26 ms [20:20:04] RECOVERY - cp5 HTTPS on cp5 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 1112 bytes in 1.962 second response time [20:20:08] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [20:20:16] !log disable puppet on misc2 again [20:20:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [20:20:24] RECOVERY - cp5 Disk Space on cp5 is OK: DISK OK - free space: / 5875 MB (24% inode=95%); [20:20:30] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [20:20:40] RECOVERY - cp5 Puppet on cp5 is OK: OK: Puppet is currently enabled, last run 38 minutes ago with 0 failures [20:20:48] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZKR [20:20:49] [02miraheze/puppet] 07paladox 03fc50e4e - Update config.ini.php.erb [20:21:08] Well, looks like Voidwalker has a twin [20:22:32] RECOVERY - cp5 HTTP on cp5 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 1120 bytes in 1.107 second response time [20:22:54] PROBLEM - misc2 Puppet on misc2 is WARNING: WARNING: Puppet is currently disabled, message: SPF, last run 2 minutes ago with 0 failures [20:22:56] AmandaCatherine twin? [20:23:11] Voidwalker has a twin named Voidwalker_ [20:23:14] paladox ^ [20:23:21] oh [20:23:25] :P [20:23:27] hai [20:23:45] but there can be only one [20:24:46] Voidwalker: can we block https://meta.miraheze.org/wiki/Special:Contributions/Marseillaise [20:24:49] Title: [ User contributions for Marseillaise - Miraheze Meta ] - meta.miraheze.org [20:24:58] They’ve been disruptive much past their final warning [20:25:00] JohnLewis they have resolved the issue so reverting your pr [20:25:03] pr = commit [20:25:07] [02miraheze/puppet] 07Southparkfan pushed 032 commits to 03master [+6/-6/±3] 13https://git.io/fAZK5 [20:25:08] [02miraheze/puppet] 07Southparkfan 03694cbfe - step 1/2 piwik -> matomo [20:25:11] [02miraheze/puppet] 07Southparkfan 03ef48184 - Merge branch 'master' of github.com:miraheze/puppet [20:25:25] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZKA [20:25:27] [02miraheze/dns] 07paladox 03ddbc32b - Revert "take cp5 down" This reverts commit 5c55c3eaba959fe9475a14071a19fbeb0af1ee2b. [20:25:33] paladox: I'm not sure if your piwik ssl_force change has an impact if it's behind a reverse proxy [20:25:48] SPF|Cloud it was just showing a warning in piwik (matomo) [20:25:53] a new check added in 3.6.0 [20:26:07] that still doesn't tell me if we can safely enable it [20:26:25] I'll have to look shortly, I'm not that available rn [20:26:31] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZ6k [20:26:33] [02miraheze/puppet] 07Southparkfan 032f12c84 - Revert "Update config.ini.php.erb" This reverts commit fc50e4e697abe361693040c12d1f895da2ce9670. [20:27:10] SPF|Cloud we *should* be able to as it didn't fail when i added it manually [20:27:12] to the config [20:27:17] when you had puppet disabled [20:27:38] in that case let me finish the cleanup first then re-enable again [20:27:44] and now it's failing again >_< [20:27:53] ok thanks [20:28:00] nvm [20:28:52] RECOVERY - misc2 Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 12 seconds ago with 0 failures [20:29:25] Uhh... why is it not letting me create an empty test filter with no conditions? https://i.imgur.com/3QXue8V.jpg [20:29:30] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+1/-1/±0] 13https://git.io/fAZ60 [20:29:32] [02miraheze/puppet] 07Southparkfan 03326bcea - fix renaming mistake [20:29:34] I’ve been able to do it before [20:29:58] !log puppet enabled on misc2 [20:30:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [20:32:17] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZ6M [20:32:18] [02miraheze/puppet] 07Southparkfan 033a93cdf - Revert "Revert "Update config.ini.php.erb"" This reverts commit 2f12c841026876d13d88b4657b2b95167427e656. [20:35:55] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 2984 [20:38:12] https://phabricator.miraheze.org/T3532 [20:38:13] Title: [ ⚓ T3532 Can’t create an empty abuse filter ] - phabricator.miraheze.org [20:40:20] paladox: if you want: https://github.com/miraheze/puppet/blob/master/modules/matomo/manifests/init.pp#L21-L51 could probably be replaced with include ::php, https://github.com/miraheze/puppet/blob/master/modules/matomo/manifests/init.pp#L53-L64 must be moved to ::php I guess [20:40:21] Title: [ puppet/init.pp at master · miraheze/puppet · GitHub ] - github.com [20:40:22] Title: [ puppet/init.pp at master · miraheze/puppet · GitHub ] - github.com [20:40:36] makes reusing configuration for the coming migrations much easier [20:40:46] yeh [20:41:28] SPF|Cloud i think we would have to move phab to nginx first [20:41:37] to be able to move that fpm stuff to ::php [20:41:42] why? [20:42:18] it doesn't matter if php-fpm is installed on an apache server, just as long as the apache module itself is installed [20:42:44] at most the running php-fpm service will consume a little bit of memory but it won't interfere with apache [20:43:37] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-5 [+0/-0/±1] 13https://git.io/fAZiN [20:43:39] [02miraheze/puppet] 07paladox 03a73b4fe - Redirect piwik.miraheze.org to matomo.miraheze.org [20:43:40] [02puppet] 07paladox created branch 03paladox-patch-5 - 13https://git.io/vbiAS [20:43:42] [02puppet] 07paladox opened pull request 03#819: Redirect piwik.miraheze.org to matomo.miraheze.org - 13https://git.io/fAZiA [20:43:46] SPF|Cloud does that look ok ^^? [20:44:01] SPF|Cloud ah ok [20:44:05] piwik.mh.o has been removed from dns [20:44:15] SPF|Cloud it now defaults to using mw :) [20:44:20] so it still works [20:44:28] (goes through varnish) [20:44:40] SPF|Cloud see https://piwik.miraheze.org/wiki/ [20:44:40] Title: [ Wiki not Found ] - piwik.miraheze.org [20:45:17] https://github.com/miraheze/dns/commit/8716d6e59d88dc57c9de577c8ac376fede485574 [20:45:17] Title: [ rm piwik · miraheze/dns@8716d6e · GitHub ] - github.com [20:45:44] SPF|Cloud yep though now it uses https://github.com/miraheze/dns/blame/master/zones/miraheze.org#L14 [20:45:44] Title: [ dns/zones/miraheze.org at master · miraheze/dns · GitHub ] - github.com [20:45:55] whild card * [20:46:00] oh, that's true [20:46:17] SPF|Cloud can i merge? :) [20:46:33] I prefer you do an actual redirect [20:46:45] SPF|Cloud oh how do i do the actual redirect? [20:46:51] as in https://github.com/miraheze/ssl/blob/065cc88855080b271fe5f6431e135a0641ed4457/redirects.yaml [20:46:52] Title: [ ssl/redirects.yaml at 065cc88855080b271fe5f6431e135a0641ed4457 · miraheze/ssl · GitHub ] - github.com [20:46:57] so people actually become aware of the new domain [20:47:08] https://github.com/miraheze/ssl/blob/065cc88855080b271fe5f6431e135a0641ed4457/redirects.yaml#L73 [20:47:09] Title: [ ssl/redirects.yaml at 065cc88855080b271fe5f6431e135a0641ed4457 · miraheze/ssl · GitHub ] - github.com [20:48:12] SPF|Cloud i wonder how we get it to use miraheze cert for the redirect? [20:48:51] https://www.irccloud.com/pastebin/n3EP5OIS/ [20:48:52] Title: [ Snippet | IRCCloud ] - www.irccloud.com [20:49:02] gotta go now bye [20:50:09] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZPQ [20:50:11] [02miraheze/ssl] 07paladox 0329e655c - redirect piwik.miraheze.org to matomo.miraheze.org [20:53:48] SPF|Cloud thanks [20:54:12] PROBLEM - piwik.miraheze.org - GlobalSign on sslhost is WARNING: WARNING - Certificate '*.miraheze.org' expires in 23 day(s) (Sat 22 Sep 2018 08:12:13 PM GMT +0000). [20:59:03] FYI https://phabricator.miraheze.org/T3533 [20:59:04] Title: [ ⚓ T3533 Add MirahezeMagic to translatewiki.net ] - phabricator.miraheze.org [21:34:15] SPF|Cloud we should put phab behind varnish with reason "performance" [21:41:48] [02puppet] 07paladox created branch 03paladox-patch-6 - 13https://git.io/vbiAS [21:41:49] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/fAZSa [21:41:51] [02miraheze/puppet] 07paladox 037c7d962 - Add php7.2-fpm to the php class [21:41:52] [02puppet] 07paladox opened pull request 03#820: Add php7.2-fpm to the php class - 13https://git.io/fAZSw [21:42:18] [02mw-config] 07Amanda-Catherine opened pull request 03#2413: Add Hub namespace for statisticswiki T3534 - 13https://git.io/fAZS6 [21:44:03] [02puppet] 07paladox synchronize pull request 03#820: Add php7.2-fpm to the php class - 13https://git.io/fAZSw [21:44:05] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/fAZSy [21:44:06] [02miraheze/puppet] 07paladox 03ce542cf - Update init.pp [21:44:26] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+1/-0/±0] 13https://git.io/fAZS9 [21:44:28] [02miraheze/puppet] 07paladox 032f8c4f6 - Create www-7.2.conf [21:44:30] [02puppet] 07paladox synchronize pull request 03#820: Add php7.2-fpm to the php class - 13https://git.io/fAZSw [21:44:43] [02puppet] 07paladox closed pull request 03#820: Add php7.2-fpm to the php class - 13https://git.io/fAZSw [21:44:45] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/fAZS7 [21:44:46] [02miraheze/puppet] 07paladox 034403a06 - Add php7.2-fpm to the php class (#820) * Add php7.2-fpm to the php class * Update init.pp * Create www-7.2.conf [21:44:48] [02puppet] 07paladox deleted branch 03paladox-patch-6 - 13https://git.io/vbiAS [21:44:49] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-6 [21:50:18] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZ9a [21:50:20] [02miraheze/puppet] 07paladox 03d092aa1 - Use built in php class [21:51:35] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZ9D [21:51:36] [02miraheze/puppet] 07paladox 03366c269 - fix [21:51:58] thats a bad commit msg shame on you paladox [21:52:08] SPF|Cloud done ^^ [21:53:32] [02puppet] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbiAS [21:53:33] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-1 [21:53:35] [02puppet] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vbiAS [21:53:36] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-2 [21:53:38] [02puppet] 07paladox deleted branch 03paladox-patch-3 - 13https://git.io/vbiAS [21:53:39] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-3 [21:53:41] [02puppet] 07paladox deleted branch 03JohnFLewis-patch-1 - 13https://git.io/vbiAS [21:53:43] [02miraheze/puppet] 07paladox deleted branch 03JohnFLewis-patch-1 [21:53:54] JohnLewis i should have asked you first [21:54:04] but could i delete 'JohnFLewis-patch-1'? [21:54:13] if there's no open PR it's fine :P [21:54:17] (for my stuff) [21:54:17] ok [21:56:28] [02mw-config] 07Amanda-Catherine synchronize pull request 03#2413: Add Hub namespace for statisticswiki T3534 - 13https://git.io/fAZS6 [21:57:32] [02mw-config] 07paladox closed pull request 03#2413: Add Hub namespace for statisticswiki T3534 - 13https://git.io/fAZS6 [21:57:34] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZH8 [21:57:35] [02miraheze/mw-config] 07Amanda-Catherine 031136733 - Add Hub namespace for statisticswiki T3534 (#2413) * Add Hub namespace for statisticswiki T3534 * Define namespaces [21:57:59] Of course paladoxbot gets to it first :p [21:58:14] lol [22:26:10] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-1/±5] 13https://git.io/fAZ51 [22:26:11] [02miraheze/puppet] 07paladox 03a877ba6 - Move phabricator to use nginx (same for lizardfs webui) also increase phabs upload limit to 5mb [22:26:46] !log switching phabricator to nginx [22:26:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [22:31:04] PROBLEM - misc4 Puppet on misc4 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/nginx/sites-available/lizard-miraheze-org] [22:32:04] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 52 seconds ago with 0 failures [22:33:34] PROBLEM - misc4 HTTPS on misc4 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2280 bytes in 0.064 second response time [22:33:52] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZdO [22:33:54] [02miraheze/puppet] 07paladox 037694c0e - lizardfs: fix nginx conf location [22:33:56] PROBLEM - misc4 HTTP on misc4 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2280 bytes in 0.067 second response time [22:37:20] RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 18 seconds ago with 0 failures [22:37:26] PROBLEM - misc4 HTTPS on misc4 is WARNING: HTTP WARNING: HTTP/1.1 401 Unauthorized - 397 bytes in 0.025 second response time [22:37:50] PROBLEM - misc4 HTTP on misc4 is WARNING: HTTP WARNING: HTTP/1.1 401 Unauthorized - 397 bytes in 0.013 second response time [22:38:46] meh [22:41:02] JohnLewis review https://git.io/fAZ51 please :) [22:41:02] Title: [ Comparing 366c269ce00b...a877ba605df7 · miraheze/puppet · GitHub ] - git.io [22:41:11] since lizard.miraheze.org (when logged in) shows 502 [22:42:35] anything in nginx error logs? [22:43:04] * paladox looks [22:47:11] PROBLEM - misc4 Puppet on misc4 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 6 minutes ago with 0 failures [22:53:37] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZbZ [22:53:39] [02miraheze/puppet] 07paladox 03e1063a2 - Update nginx.conf [23:04:56] PROBLEM - misc4 HTTP on misc4 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2280 bytes in 0.069 second response time [23:06:40] PROBLEM - misc4 HTTPS on misc4 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2280 bytes in 0.065 second response time [23:09:20] meh [23:24:57] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [23:28:17] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZA9 [23:28:18] [02miraheze/puppet] 07paladox 030d3f18a - fix config [23:28:38] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZAH [23:28:40] [02miraheze/puppet] 07paladox 03e006e34 - Update phab.miraheze.wiki.conf [23:30:36] RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 40 seconds ago with 0 failures [23:32:02] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZAj [23:32:04] [02miraheze/puppet] 07paladox 03cab0d48 - Update phab.miraheze.wiki.conf [23:34:33] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZxZ [23:34:34] [02miraheze/puppet] 07paladox 0334824c9 - Update phab.miraheze.wiki.conf [23:36:50] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZxE [23:36:51] [02miraheze/puppet] 07paladox 03b9af12a - Update phab.miraheze.wiki.conf [23:37:44] oh was about to complain about phab.mh.wiki not loading [23:41:12] revi heh im trying to switch it to ngin [23:41:14] *nginx [23:42:24] PROBLEM - misc4 Puppet on misc4 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 4 minutes ago with 0 failures [23:45:00] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZxA [23:45:02] [02miraheze/puppet] 07paladox 03f5078c0 - Fix config [23:45:22] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZxh [23:45:24] [02miraheze/puppet] 07paladox 03334fa4d - fix config [23:46:23] RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:51:58] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZpR [23:51:59] [02miraheze/puppet] 07paladox 03a3b2c1d - Update init.pp [23:53:22] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZpz [23:53:23] [02miraheze/puppet] 07paladox 03ad72df1 - Update site.pp [23:56:43] RECOVERY - misc4 HTTP on misc4 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 421 bytes in 0.004 second response time [23:58:33] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fAZpH [23:58:35] [02miraheze/puppet] 07paladox 03493e800 - Fix puppet config [23:59:13] RECOVERY - misc3 Puppet on misc3 is OK: OK: Puppet is currently enabled, last run 7 seconds ago with 0 failures