[00:07:00] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.08, 6.50, 5.08 [00:09:00] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 3.68, 5.45, 4.88 [00:28:42] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.54, 6.88, 6.15 [00:30:39] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 6.26, 6.55, 6.11 [01:15:50] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 162 failures. Last run 2 minutes ago with 162 failures. Failed resources (up to 3 shown): File[madzebrascience.wiki_private],File[www.modesofdiscourse.com],File[www.modesofdiscourse.com_private],File[wiki.pablojprieto.com.es] [01:16:02] PROBLEM - cp2 Puppet on cp2 is CRITICAL: CRITICAL: Puppet has 72 failures. Last run 2 minutes ago with 72 failures. Failed resources (up to 3 shown) [01:16:53] PROBLEM - mw1 Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 9 failures. Last run 3 minutes ago with 9 failures. Failed resources (up to 3 shown) [01:16:59] paladox ^ [01:17:06] PROBLEM - mw3 Puppet on mw3 is CRITICAL: CRITICAL: Puppet has 7 failures. Last run 4 minutes ago with 7 failures. Failed resources (up to 3 shown) [01:17:19] It’ll recover shortly [01:17:24] alright [01:17:25] It’s puppet [01:17:33] *GC on the puppet [01:17:43] Host [01:17:54] Paladox: Come check out my IRC bot [01:18:03] https://grafana.miraheze.org/d/W9MIkA7iz/miraheze-cluster?orgId=1&var-job=node&var-node=puppet1.miraheze.org&var-port=9100&fullscreen&panelId=78 [01:18:04] [ Grafana ] - grafana.miraheze.org [01:18:39] Apap caught me just as I was heading to bed [01:18:44] ah i love seeing things i dont understand [01:18:47] woops, sorry [01:18:49] :^) [01:19:26] trust me if the world was ending there would be more alerts [01:19:28] Nothing to be sorry about :) [01:20:24] okay, good night :) [01:20:41] Zppix: You should come see my IRC bot I have rewritten it [01:21:20] And you too :) [01:22:07] Examknow: you mean added one of the modules from my bot's code [01:22:23] Zppix: Nope [01:24:17] Zppix: Honestly I think Exambot works better than ZppixBot [01:24:28] hence the quieting on my channel [01:25:05] RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 5 seconds ago with 0 failures [01:25:22] RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 52 seconds ago with 0 failures [01:25:57] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 38 seconds ago with 0 failures [01:27:40] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:31:28] lol [01:35:19] Apap: Whats so funny? [01:35:49] Exambot > ZppixBot [01:36:10] Apap: It is pretty funny [01:54:55] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.37, 6.26, 5.68 [01:58:51] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 6.54, 6.38, 5.87 [02:24:46] RECOVERY - mw3 Disk Space on mw3 is OK: DISK OK - free space: / 12020 MB (15% inode=99%); [04:07:00] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 11.70, 8.45, 5.97 [04:17:00] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 5.19, 7.46, 6.90 [04:19:00] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 5.32, 6.70, 6.67 [04:27:44] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.04, 7.45, 6.98 [05:11:00] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 4.89, 6.75, 7.70 [05:17:00] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 2.72, 4.35, 6.35 [06:20:26] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.33, 6.64, 6.07 [06:24:26] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 6.26, 6.68, 6.24 [06:26:12] RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 2851 MB (11% inode=94%); [07:19:19] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.12, 6.68, 6.07 [07:25:11] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 6.50, 6.51, 6.18 [10:05:10] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.75, 7.08, 5.76 [10:07:04] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 5.89, 6.57, 5.73 [10:28:49] PROBLEM - aman.info.tm - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'aman.info.tm' expires in 15 day(s) (Sun 10 Nov 2019 10:25:07 AM GMT +0000). [10:29:04] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0xZ [10:29:06] [02miraheze/ssl] 07MirahezeSSLBot 039c2c72d - Bot: Update SSL cert for aman.info.tm [10:42:48] RECOVERY - aman.info.tm - LetsEncrypt on sslhost is OK: OK - Certificate 'aman.info.tm' will expire on Thu 23 Jan 2020 09:28:58 AM GMT +0000. [10:43:46] PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2649 MB (10% inode=94%); [10:50:14] PROBLEM - wiki.lbcomms.co.za - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.lbcomms.co.za' expires in 15 day(s) (Sun 10 Nov 2019 10:46:37 AM GMT +0000). [10:50:30] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0x6 [10:50:32] [02miraheze/ssl] 07MirahezeSSLBot 038cd544a - Bot: Update SSL cert for wiki.lbcomms.co.za [10:51:13] PROBLEM - wiki.hackdown.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.hackdown.org' expires in 15 day(s) (Sun 10 Nov 2019 10:48:55 AM GMT +0000). [10:51:27] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0xP [10:51:28] [02miraheze/ssl] 07MirahezeSSLBot 03e5a35fe - Bot: Update SSL cert for wiki.hackdown.org [10:51:58] PROBLEM - wiki.meeusen.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.meeusen.net' expires in 15 day(s) (Sun 10 Nov 2019 10:49:13 AM GMT +0000). [10:52:13] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0x1 [10:52:15] [02miraheze/ssl] 07MirahezeSSLBot 03f864aa5 - Bot: Update SSL cert for wiki.meeusen.net [10:55:05] PROBLEM - wiki.drones4nature.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.drones4nature.info' expires in 15 day(s) (Sun 10 Nov 2019 10:52:36 AM GMT +0000). [10:55:18] PROBLEM - cp4 Puppet on cp4 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[www.apcwiki.org_private] [10:55:22] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0x5 [10:55:24] [02miraheze/ssl] 07MirahezeSSLBot 038737419 - Bot: Update SSL cert for wiki.drones4nature.info [10:55:33] PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[www.apcwiki.org_private] [10:55:57] PROBLEM - cp2 Puppet on cp2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[www.apcwiki.org_private] [10:56:22] PROBLEM - mw3 Puppet on mw3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[www.apcwiki.org_private] [10:56:35] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[www.apcwiki.org_private] [10:56:56] PROBLEM - mw1 Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[www.apcwiki.org_private] [10:57:07] PROBLEM - wikipuk.cl - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wikipuk.cl' expires in 15 day(s) (Sun 10 Nov 2019 10:54:52 AM GMT +0000). [10:57:22] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0xb [10:57:23] [02miraheze/ssl] 07MirahezeSSLBot 03dabbb5b - Bot: Update SSL cert for wikipuk.cl [10:57:45] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[www.apcwiki.org_private] [10:58:09] PROBLEM - wiki.jacksonheights.nyc - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.jacksonheights.nyc' expires in 15 day(s) (Sun 10 Nov 2019 10:55:29 AM GMT +0000). [10:58:22] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0xA [10:58:24] [02miraheze/ssl] 07MirahezeSSLBot 0351edb6b - Bot: Update SSL cert for wiki.jacksonheights.nyc [11:00:06] PROBLEM - wiki.besuccess.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.besuccess.com' expires in 15 day(s) (Sun 10 Nov 2019 10:57:15 AM GMT +0000). [11:00:34] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0pe [11:00:35] [02miraheze/ssl] 07MirahezeSSLBot 03f86c2d7 - Bot: Update SSL cert for wiki.besuccess.com [11:01:09] PROBLEM - speleo.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'speleo.wiki' expires in 15 day(s) (Sun 10 Nov 2019 10:57:27 AM GMT +0000). [11:01:09] PROBLEM - disabled.life - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'disabled.life' expires in 15 day(s) (Sun 10 Nov 2019 10:58:16 AM GMT +0000). [11:01:22] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0pf [11:01:24] [02miraheze/ssl] 07MirahezeSSLBot 03c07b7b2 - Bot: Update SSL cert for speleo.wiki [11:01:35] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0pJ [11:01:37] [02miraheze/ssl] 07MirahezeSSLBot 037518fb5 - Bot: Update SSL cert for disabled.life [11:01:41] PROBLEM - wiki.teessidehackspace.org.uk - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.teessidehackspace.org.uk' expires in 15 day(s) (Sun 10 Nov 2019 10:59:12 AM GMT +0000). [11:01:55] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0pU [11:01:57] [02miraheze/ssl] 07MirahezeSSLBot 034612b12 - Bot: Update SSL cert for wiki.teessidehackspace.org.uk [11:04:40] PROBLEM - wiki.thunis.eu - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.thunis.eu' expires in 15 day(s) (Sun 10 Nov 2019 11:01:08 AM GMT +0000). [11:04:56] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0ps [11:04:57] [02miraheze/ssl] 07MirahezeSSLBot 03dfc1d97 - Bot: Update SSL cert for wiki.thunis.eu [11:05:18] PROBLEM - wiki.tulpa.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.tulpa.info' expires in 15 day(s) (Sun 10 Nov 2019 11:02:07 AM GMT +0000). [11:05:39] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0pn [11:05:40] [02miraheze/ssl] 07MirahezeSSLBot 0311d413e - Bot: Update SSL cert for wiki.tulpa.info [11:05:47] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 10.44, 7.91, 6.46 [11:08:00] PROBLEM - lodge.jsnydr.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'lodge.jsnydr.com' expires in 15 day(s) (Sun 10 Nov 2019 11:04:01 AM GMT +0000). [11:08:18] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0pW [11:08:20] [02miraheze/ssl] 07MirahezeSSLBot 033794632 - Bot: Update SSL cert for lodge.jsnydr.com [11:09:42] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 6.21, 7.36, 6.60 [11:10:19] PROBLEM - wiki.hanaoto.me - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.hanaoto.me' expires in 15 day(s) (Sun 10 Nov 2019 11:08:14 AM GMT +0000). [11:10:37] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0pB [11:10:39] [02miraheze/ssl] 07MirahezeSSLBot 03399ebab - Bot: Update SSL cert for wiki.hanaoto.me [11:12:07] PROBLEM - wiki.worlduniversityandschool.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.worlduniversityandschool.org' expires in 15 day(s) (Sun 10 Nov 2019 11:09:25 AM GMT +0000). [11:12:23] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0pE [11:12:24] [02miraheze/ssl] 07MirahezeSSLBot 03453bbb0 - Bot: Update SSL cert for wiki.worlduniversityandschool.org [11:13:54] PROBLEM - froggy.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'froggy.info' expires in 15 day(s) (Sun 10 Nov 2019 11:11:25 AM GMT +0000). [11:14:04] PROBLEM - wiki.inebriation-confederation.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.inebriation-confederation.com' expires in 15 day(s) (Sun 10 Nov 2019 11:11:05 AM GMT +0000). [11:14:08] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0pu [11:14:10] [02miraheze/ssl] 07MirahezeSSLBot 03fa36e4d - Bot: Update SSL cert for froggy.info [11:14:22] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0pz [11:14:23] [02miraheze/ssl] 07MirahezeSSLBot 03e51af86 - Bot: Update SSL cert for wiki.inebriation-confederation.com [11:14:51] PROBLEM - madgenderscience.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'madgenderscience.wiki' expires in 15 day(s) (Sun 10 Nov 2019 11:12:34 AM GMT +0000). [11:15:07] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0pg [11:15:09] [02miraheze/ssl] 07MirahezeSSLBot 03d16abab - Bot: Update SSL cert for madgenderscience.wiki [11:15:36] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.42, 6.43, 5.92 [11:16:07] PROBLEM - wiki.autocountsoft.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.autocountsoft.com' expires in 15 day(s) (Sun 10 Nov 2019 11:13:25 AM GMT +0000). [11:16:21] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Je0pV [11:16:22] [02miraheze/ssl] 07MirahezeSSLBot 036b82664 - Bot: Update SSL cert for wiki.autocountsoft.com [11:17:19] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 4.98, 6.45, 6.54 [11:17:35] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 6.26, 6.37, 5.96 [11:27:55] [02miraheze/ssl] 07Reception123 pushed 032 commits to 03master [+0/-1/±20] 13https://git.io/Je0pS [11:27:57] [02miraheze/ssl] 07Reception123 03e5bf801 - remove www.apcwiki.org Wiki has been deleted [11:27:58] [02miraheze/ssl] 07Reception123 030115db8 - Merge branch 'master' of https://github.com/miraheze/ssl [11:27:59] [ GitHub - miraheze/ssl: A repository for storing and managing SSL certificates for Miraheze ] - github.com [11:33:01] RECOVERY - wikipuk.cl - LetsEncrypt on sslhost is OK: OK - Certificate 'wikipuk.cl' will expire on Thu 23 Jan 2020 09:57:16 AM GMT +0000. [11:33:05] RECOVERY - wiki.drones4nature.info - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.drones4nature.info' will expire on Thu 23 Jan 2020 09:55:16 AM GMT +0000. [11:33:13] RECOVERY - wiki.tulpa.info - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.tulpa.info' will expire on Thu 23 Jan 2020 10:05:31 AM GMT +0000. [11:33:16] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 9.43, 7.90, 6.81 [11:33:18] RECOVERY - cp4 Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 32 seconds ago with 0 failures [11:33:20] RECOVERY - madgenderscience.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'madgenderscience.wiki' will expire on Thu 23 Jan 2020 10:14:59 AM GMT +0000. [11:33:20] RECOVERY - wiki.hackdown.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.hackdown.org' will expire on Thu 23 Jan 2020 09:51:20 AM GMT +0000. [11:33:21] RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 4 seconds ago with 0 failures [11:33:22] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 8 seconds ago with 0 failures [11:33:24] RECOVERY - disabled.life - LetsEncrypt on sslhost is OK: OK - Certificate 'disabled.life' will expire on Thu 23 Jan 2020 10:01:30 AM GMT +0000. [11:33:24] RECOVERY - wiki.teessidehackspace.org.uk - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.teessidehackspace.org.uk' will expire on Thu 23 Jan 2020 10:01:49 AM GMT +0000. [11:33:43] RECOVERY - speleo.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'speleo.wiki' will expire on Thu 23 Jan 2020 10:01:16 AM GMT +0000. [11:33:54] RECOVERY - froggy.info - LetsEncrypt on sslhost is OK: OK - Certificate 'froggy.info' will expire on Thu 23 Jan 2020 10:14:02 AM GMT +0000. [11:33:57] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 20 seconds ago with 0 failures [11:33:58] RECOVERY - wiki.meeusen.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.meeusen.net' will expire on Thu 23 Jan 2020 09:52:06 AM GMT +0000. [11:34:01] RECOVERY - lodge.jsnydr.com - LetsEncrypt on sslhost is OK: OK - Certificate 'lodge.jsnydr.com' will expire on Thu 23 Jan 2020 10:08:12 AM GMT +0000. [11:34:04] RECOVERY - wiki.besuccess.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.besuccess.com' will expire on Thu 23 Jan 2020 10:00:22 AM GMT +0000. [11:34:05] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [11:34:05] RECOVERY - wiki.inebriation-confederation.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.inebriation-confederation.com' will expire on Thu 23 Jan 2020 10:14:15 AM GMT +0000. [11:34:06] RECOVERY - wiki.lbcomms.co.za - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.lbcomms.co.za' will expire on Thu 23 Jan 2020 09:50:24 AM GMT +0000. [11:34:07] RECOVERY - wiki.autocountsoft.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.autocountsoft.com' will expire on Thu 23 Jan 2020 10:16:15 AM GMT +0000. [11:34:07] RECOVERY - wiki.worlduniversityandschool.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.worlduniversityandschool.org' will expire on Thu 23 Jan 2020 10:12:16 AM GMT +0000. [11:34:09] RECOVERY - wiki.jacksonheights.nyc - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.jacksonheights.nyc' will expire on Thu 23 Jan 2020 09:58:16 AM GMT +0000. [11:34:20] RECOVERY - wiki.hanaoto.me - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.hanaoto.me' will expire on Thu 23 Jan 2020 10:10:31 AM GMT +0000. [11:34:35] RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 43 seconds ago with 0 failures [11:34:42] RECOVERY - wiki.thunis.eu - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.thunis.eu' will expire on Thu 23 Jan 2020 10:04:48 AM GMT +0000. [11:35:19] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.34, 7.43, 6.76 [11:35:44] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [11:37:20] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.85, 7.66, 6.90 [11:39:23] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 5.95, 6.86, 6.70 [11:45:23] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 6.41, 6.55, 6.61 [11:53:20] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.16, 7.35, 6.86 [11:55:20] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 6.14, 6.71, 6.67 [11:59:10] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 6.87, 6.86, 6.43 [12:01:06] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 6.59, 6.68, 6.41 [12:29:21] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.46, 6.69, 6.20 [12:31:21] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 6.34, 6.37, 6.13 [12:48:23] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 9.97, 7.54, 6.43 [12:50:22] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.75, 7.31, 6.47 [12:56:21] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 4.77, 6.20, 6.28 [13:06:22] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.13, 6.78, 6.37 [13:08:25] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 6.17, 6.47, 6.30 [13:18:57] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 6.97, 6.97, 6.52 [13:22:45] PROBLEM - mw3 Current Load on mw3 is CRITICAL: CRITICAL - load average: 9.25, 7.96, 6.98 [13:23:32] PROBLEM - mw2 Current Load on mw2 is CRITICAL: CRITICAL - load average: 10.67, 7.78, 6.00 [13:24:34] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.04, 6.81, 6.23 [13:24:39] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.83, 7.84, 7.05 [13:26:34] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 5.00, 6.24, 6.09 [13:27:31] PROBLEM - mw2 Current Load on mw2 is WARNING: WARNING - load average: 5.88, 7.02, 6.14 [13:31:29] RECOVERY - mw2 Current Load on mw2 is OK: OK - load average: 6.23, 6.62, 6.17 [13:36:03] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 5.57, 6.39, 6.75 [13:59:13] Hello upmime! If you have any questions, feel free to ask and someone should answer soon. [14:18:05] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.92, 7.11, 6.63 [14:20:04] PROBLEM - mw3 Current Load on mw3 is CRITICAL: CRITICAL - load average: 8.30, 7.37, 6.77 [14:22:02] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 6.70, 7.03, 6.71 [14:22:27] Paladox: Hey [14:22:31] hi [14:23:14] Paladox: do you know if RhinosF1 is around? [14:23:29] I think he is [14:24:01] it says he is online but he did not respond to my ping [14:24:50] Maybe he's busy [14:24:57] maybe [14:25:08] he just usually always responds [14:30:57] I'm just looking at everything I've missed while away [14:33:39] PROBLEM - mw3 Current Load on mw3 is CRITICAL: CRITICAL - load average: 8.34, 7.33, 6.96 [14:35:14] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.88, 7.27, 6.33 [14:37:11] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.47, 7.43, 6.51 [14:37:36] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.72, 7.70, 7.22 [14:39:08] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 4.42, 6.29, 6.20 [14:45:31] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 5.23, 6.28, 6.78 [14:58:34] PROBLEM - mw2 Current Load on mw2 is CRITICAL: CRITICAL - load average: 10.06, 6.59, 5.28 [14:59:36] PROBLEM - mw3 Current Load on mw3 is CRITICAL: CRITICAL - load average: 8.36, 7.24, 6.81 [15:00:33] PROBLEM - mw2 Current Load on mw2 is WARNING: WARNING - load average: 7.48, 6.64, 5.44 [15:00:50] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 9.26, 7.50, 6.41 [15:02:49] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.53, 7.40, 6.50 [15:02:57] .mh wikidb test [15:02:57] https://wikidb.miraheze.org/wiki/test [15:03:05] Examknow: ^ [15:03:14] Shortlinks.py? [15:03:31] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.83, 7.48, 7.01 [15:06:27] RECOVERY - mw2 Current Load on mw2 is OK: OK - load average: 5.23, 6.30, 5.71 [15:06:47] PROBLEM - test1 HTTPS on test1 is CRITICAL: connect to address 185.52.2.243 and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [15:06:47] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.08, 8.05, 7.00 [15:07:46] PROBLEM - cp4 Stunnel Http for test1 on cp4 is CRITICAL: HTTP CRITICAL - No data received from host [15:07:55] PROBLEM - cp2 Stunnel Http for test1 on cp2 is CRITICAL: HTTP CRITICAL - No data received from host [15:08:24] PROBLEM - cp3 Stunnel Http for test1 on cp3 is CRITICAL: HTTP CRITICAL - No data received from host [15:08:49] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.82, 7.77, 7.01 [15:10:48] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.11, 7.71, 7.06 [15:11:31] PROBLEM - mw3 Current Load on mw3 is CRITICAL: CRITICAL - load average: 8.13, 7.63, 7.22 [15:12:51] RECOVERY - test1 HTTPS on test1 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 444 bytes in 0.033 second response time [15:13:31] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.12, 7.38, 7.17 [15:13:44] RECOVERY - cp4 Stunnel Http for test1 on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 24638 bytes in 0.021 second response time [15:13:56] RECOVERY - cp2 Stunnel Http for test1 on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 24638 bytes in 0.506 second response time [15:14:32] RECOVERY - cp3 Stunnel Http for test1 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 24638 bytes in 0.870 second response time [15:15:31] PROBLEM - mw3 Current Load on mw3 is CRITICAL: CRITICAL - load average: 10.38, 8.32, 7.52 [15:18:50] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 6.54, 7.88, 7.51 [15:22:57] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 10.93, 8.51, 7.77 [15:30:53] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.00, 7.72, 7.77 [15:30:55] PROBLEM - misc1 Puppet on misc1 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 8 minutes ago with 0 failures [15:32:57] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 34 seconds ago with 0 failures [15:33:30] PROBLEM - mw2 Current Load on mw2 is CRITICAL: CRITICAL - load average: 9.34, 7.11, 6.37 [15:34:51] PROBLEM - cp3 Stunnel Http for test1 on cp3 is CRITICAL: HTTP CRITICAL - No data received from host [15:35:06] PROBLEM - test1 HTTPS on test1 is CRITICAL: connect to address 185.52.2.243 and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [15:35:30] RECOVERY - mw2 Current Load on mw2 is OK: OK - load average: 6.34, 6.64, 6.28 [15:35:38] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 minute ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [15:35:46] PROBLEM - cp4 Stunnel Http for test1 on cp4 is CRITICAL: HTTP CRITICAL - No data received from host [15:36:52] RECOVERY - cp3 Stunnel Http for test1 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 24638 bytes in 0.801 second response time [15:37:06] RECOVERY - test1 HTTPS on test1 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 444 bytes in 0.020 second response time [15:37:34] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:37:44] RECOVERY - cp4 Stunnel Http for test1 on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 24638 bytes in 0.017 second response time [15:42:45] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.24, 7.56, 7.60 [15:43:38] PROBLEM - test1 Puppet on test1 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 7 minutes ago with 0 failures [15:44:24] [02ssl] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vxP9L [15:44:26] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JeEUH [15:44:27] [02miraheze/ssl] 07paladox 03a802049 - Change wildcard.miraheze.org to Comodo certificate [15:44:29] [02ssl] 07paladox opened pull request 03#230: Change wildcard.miraheze.org to Comodo certificate - 13https://git.io/JeEUQ [15:46:42] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.14, 7.79, 7.73 [15:47:11] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JeEUF [15:47:13] [02miraheze/ssl] 07paladox 03bbdae3e - Update redirects.yaml [15:47:14] [02ssl] 07paladox synchronize pull request 03#230: Change wildcard.miraheze.org to Comodo certificate - 13https://git.io/JeEUQ [15:48:40] [02puppet] 07paladox created branch 03paladox-patch-6 - 13https://git.io/vbiAS [15:48:42] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeEUN [15:48:43] [02miraheze/puppet] 07paladox 038f0a061 - Change wildcard.miraheze.org to use Comodo [15:48:45] [02puppet] 07paladox opened pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:49:10] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeEUp [15:49:11] [02miraheze/puppet] 07paladox 03e07addb - Update nginx.conf [15:49:13] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:49:31] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeEUh [15:49:32] [02miraheze/puppet] 07paladox 03f93372a - Update ssl.conf [15:49:34] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:49:49] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeEUj [15:49:51] [02miraheze/puppet] 07paladox 03672e51a - Update parsoid [15:49:52] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:50:11] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeETv [15:50:13] [02miraheze/puppet] 07paladox 03c4c23d0 - Update grafana.conf [15:50:15] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:50:31] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeETf [15:50:32] [02miraheze/puppet] 07paladox 03b031cea - Update icinga2.conf [15:50:34] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:50:59] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeETJ [15:51:00] [02miraheze/puppet] 07paladox 03e3c67f1 - Update mw.cnf.erb [15:51:02] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:51:18] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeETU [15:51:20] [02miraheze/puppet] 07paladox 03cc3d018 - Update nginx.conf [15:51:21] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:52:17] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeETT [15:52:19] [02miraheze/puppet] 07paladox 03f3d0e66 - Update mediawiki.conf [15:52:20] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:52:43] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeETk [15:52:45] [02miraheze/puppet] 07paladox 03edafb74 - Update init.pp [15:52:46] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:53:03] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeETL [15:53:05] [02miraheze/puppet] 07paladox 0357963bc - Update phabricator.miraheze.org.conf [15:53:06] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:53:22] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeETt [15:53:24] [02miraheze/puppet] 07paladox 0360ed833 - Update mail.miraheze.org.conf [15:53:25] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:53:31] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 6.99, 6.80, 7.93 [15:53:41] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeETq [15:53:42] [02miraheze/puppet] 07paladox 03cc577d1 - Update roundcubemail.conf [15:53:44] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:53:59] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeETm [15:54:01] [02miraheze/puppet] 07paladox 03d76d99a - Update mathoid [15:54:02] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:54:17] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeETY [15:54:18] [02miraheze/puppet] 07paladox 0362e7a56 - Update restbase [15:54:20] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:54:42] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeET3 [15:54:43] [02miraheze/puppet] 07paladox 03ac31bf9 - Update mediawiki.conf [15:54:45] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:55:30] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeETs [15:55:31] PROBLEM - mw3 Current Load on mw3 is CRITICAL: CRITICAL - load average: 8.69, 7.31, 7.96 [15:55:32] [02miraheze/puppet] 07paladox 0397b0e63 - Update wildcard.pp [15:55:33] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:56:31] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.91, 7.84, 7.68 [15:56:36] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeET8 [15:56:37] [02miraheze/puppet] 07paladox 037f6a6a3 - Update phab.miraheze.wiki.conf [15:56:39] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [15:57:30] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeET4 [15:57:32] [02miraheze/puppet] 07paladox 036d8e673 - Update config.ini.php.erb [15:58:29] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 6.70, 7.40, 7.54 [15:59:31] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 6.94, 7.19, 7.76 [16:00:27] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 9.40, 7.94, 7.70 [16:02:24] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 6.53, 7.17, 7.43 [16:03:59] PROBLEM - cp2 Stunnel Http for misc2 on cp2 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2344 bytes in 0.458 second response time [16:05:08] PROBLEM - misc2 HTTPS on misc2 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2344 bytes in 0.294 second response time [16:05:12] PROBLEM - cp3 Stunnel Http for misc2 on cp3 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2344 bytes in 0.719 second response time [16:05:21] huh [16:05:26] matomo cannot connect to the db [16:05:38] wasen't me :) [16:05:51] PROBLEM - cp4 Stunnel Http for misc2 on cp4 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2344 bytes in 0.162 second response time [16:10:19] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 6.09, 5.83, 6.70 [16:11:34] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 5.72, 5.98, 6.75 [16:12:37] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeETo [16:12:39] [02miraheze/puppet] 07paladox 03d16d73e - Revert "Update config.ini.php.erb" This reverts commit 6d8e6733b939e30ebc1841a371e00247f3cc3878. [16:13:09] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeETK [16:13:11] [02miraheze/puppet] 07paladox 032a22cd8 - Update config.ini.php.erb [16:13:12] [02puppet] 07paladox synchronize pull request 03#1120: Change wildcard.miraheze.org to use Comodo - 13https://git.io/JeEUA [16:14:15] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.33, 6.63, 6.81 [16:14:59] RECOVERY - cp3 Stunnel Http for misc2 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 41802 bytes in 0.909 second response time [16:15:06] RECOVERY - misc2 HTTPS on misc2 is OK: HTTP OK: HTTP/1.1 200 OK - 41810 bytes in 0.808 second response time [16:15:49] RECOVERY - cp4 Stunnel Http for misc2 on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 41802 bytes in 0.114 second response time [16:15:53] RECOVERY - cp2 Stunnel Http for misc2 on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 41802 bytes in 0.646 second response time [16:16:14] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 5.25, 6.11, 6.60 [16:16:18] turns out it was me [16:16:28] accidentally commited a change that was ment to be a pull [16:20:46] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeETS [16:20:48] [02miraheze/puppet] 07paladox 03f7d3cc1 - mariadb: Use mariadb-server-10.2 over mariadb-server [16:21:35] !log upgrade puppetdb to 6.7.2 on puppet1 [16:21:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [16:23:49] * paladox prays [16:23:54] re-enabed puppet on db4 [16:24:20] RECOVERY - db4 Puppet on db4 is OK: OK: Puppet is currently enabled, last run 9 seconds ago with 0 failures [16:24:29] PROBLEM - lizardfs5 Puppet on lizardfs5 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [16:24:36] puppet ran successfully (didn't try changing mariadb-server-10.2) [16:24:42] PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [16:32:32] RECOVERY - lizardfs5 Puppet on lizardfs5 is OK: OK: Puppet is currently enabled, last run 17 seconds ago with 0 failures [16:34:45] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [17:03:40] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.95, 6.47, 6.03 [17:07:38] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeEkc [17:07:39] [02miraheze/mw-config] 07Reception123 0316b9ebf - shared file repo nbdbwiki and nbdbwiki T4834 [17:07:40] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 5.51, 6.11, 5.98 [17:10:19] Hi JohnLewis [17:13:40] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.98, 7.93, 6.75 [17:15:41] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 6.11, 7.21, 6.63 [17:17:41] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 5.80, 6.65, 6.49 [17:18:32] PROBLEM - wiki.bullshit.systems - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.bullshit.systems' expires in 15 day(s) (Sun 10 Nov 2019 05:15:12 PM GMT +0000). [17:18:46] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeEk2 [17:18:47] [02miraheze/ssl] 07MirahezeSSLBot 03e4944ab - Bot: Update SSL cert for wiki.bullshit.systems [17:18:47] That url ^ xD [17:19:34] PROBLEM - wiki.kemeneth.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.kemeneth.com' expires in 15 day(s) (Sun 10 Nov 2019 05:17:00 PM GMT +0000). [17:19:47] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeEka [17:19:48] [02miraheze/ssl] 07MirahezeSSLBot 035fcf15d - Bot: Update SSL cert for wiki.kemeneth.com [17:20:12] PROBLEM - madzebrascience.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'madzebrascience.wiki' expires in 15 day(s) (Sun 10 Nov 2019 05:16:23 PM GMT +0000). [17:20:28] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeEkw [17:20:30] [02miraheze/ssl] 07MirahezeSSLBot 030567fb5 - Bot: Update SSL cert for madzebrascience.wiki [17:22:14] PROBLEM - paramony.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'paramony.org' expires in 15 day(s) (Sun 10 Nov 2019 05:19:22 PM GMT +0000). [17:22:28] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeEko [17:22:29] PROBLEM - secularknowledge.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'secularknowledge.com' expires in 15 day(s) (Sun 10 Nov 2019 05:19:57 PM GMT +0000). [17:22:30] [02miraheze/ssl] 07MirahezeSSLBot 034ef1d8d - Bot: Update SSL cert for paramony.org [17:22:44] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeEkK [17:22:46] [02miraheze/ssl] 07MirahezeSSLBot 035e5ca09 - Bot: Update SSL cert for secularknowledge.com [17:24:32] RECOVERY - wiki.bullshit.systems - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.bullshit.systems' will expire on Thu 23 Jan 2020 04:18:40 PM GMT +0000. [17:25:40] PROBLEM - wiki.nowchess.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.nowchess.org' expires in 15 day(s) (Sun 10 Nov 2019 05:23:32 PM GMT +0000). [17:25:55] PROBLEM - vise.dayid.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'vise.dayid.org' expires in 15 day(s) (Sun 10 Nov 2019 05:23:10 PM GMT +0000). [17:25:59] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeEk1 [17:26:00] [02miraheze/ssl] 07MirahezeSSLBot 0399e625a - Bot: Update SSL cert for wiki.nowchess.org [17:26:13] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeEkM [17:26:14] [02miraheze/ssl] 07MirahezeSSLBot 03fc46ca5 - Bot: Update SSL cert for vise.dayid.org [17:26:44] PROBLEM - wiki.pablojprieto.com.es - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.pablojprieto.com.es' expires in 15 day(s) (Sun 10 Nov 2019 05:24:31 PM GMT +0000). [17:26:58] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeEkS [17:27:00] [02miraheze/ssl] 07MirahezeSSLBot 03a44c2ef - Bot: Update SSL cert for wiki.pablojprieto.com.es [17:28:11] !log reception@mw2:/srv/mediawiki/w/maintenance$ sudo -u www-data php /srv/mediawiki/w/maintenance/dumpBackup.php --wiki speleowiki --full --output gzip:/home/reception/speleo25102019.xml [17:28:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:30:29] PROBLEM - mw2 Current Load on mw2 is CRITICAL: CRITICAL - load average: 8.68, 6.94, 5.75 [17:32:37] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.04, 6.33, 6.16 [17:32:48] RECOVERY - wiki.pablojprieto.com.es - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.pablojprieto.com.es' will expire on Thu 23 Jan 2020 04:26:52 PM GMT +0000. [17:33:33] RECOVERY - wiki.kemeneth.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.kemeneth.com' will expire on Thu 23 Jan 2020 04:19:41 PM GMT +0000. [17:33:38] RECOVERY - wiki.nowchess.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.nowchess.org' will expire on Thu 23 Jan 2020 04:25:50 PM GMT +0000. [17:33:56] RECOVERY - vise.dayid.org - LetsEncrypt on sslhost is OK: OK - Certificate 'vise.dayid.org' will expire on Thu 23 Jan 2020 04:26:06 PM GMT +0000. [17:34:08] RECOVERY - madzebrascience.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'madzebrascience.wiki' will expire on Thu 23 Jan 2020 04:20:22 PM GMT +0000. [17:34:13] RECOVERY - paramony.org - LetsEncrypt on sslhost is OK: OK - Certificate 'paramony.org' will expire on Thu 23 Jan 2020 04:22:23 PM GMT +0000. [17:34:27] RECOVERY - secularknowledge.com - LetsEncrypt on sslhost is OK: OK - Certificate 'secularknowledge.com' will expire on Thu 23 Jan 2020 04:22:38 PM GMT +0000. [17:36:26] PROBLEM - mw2 Current Load on mw2 is WARNING: WARNING - load average: 6.66, 7.56, 6.48 [17:38:26] RECOVERY - mw2 Current Load on mw2 is OK: OK - load average: 5.45, 6.68, 6.28 [17:38:32] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 5.69, 6.44, 6.33 [17:47:17] PROBLEM - mw3 Disk Space on mw3 is WARNING: DISK WARNING - free space: / 8410 MB (10% inode=99%); [17:55:08] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.21, 6.57, 6.36 [17:55:34] PROBLEM - mw3 Current Load on mw3 is CRITICAL: CRITICAL - load average: 8.40, 7.89, 6.89 [17:57:06]