[00:03:52] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 2 minutes ago with 8 failures. Failed resources (up to 3 shown): Service[prometheus-node-exporter],Service[nagios-nrpe-server],Package[openssh-client],Package[openssh-server] [00:04:49] sleepy-rhino: ok :( [00:06:14] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 3 datacenters are down: 128.199.139.216/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb [00:07:13] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 107.191.126.23/cpweb [00:08:22] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [00:09:11] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [00:11:41] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 6 seconds ago with 0 failures [00:18:32] MirahezeRC bot seems to be having issues in #miraheze-feed. The bot keeps quitting and reconnecting [00:20:19] It's an issue with flooding, the bot isn't designed to throttle messages properly [00:21:03] Ok, as long as it's a known issue [00:23:37] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 2 minutes ago with 10 failures. Failed resources (up to 3 shown): Service[bacula-fd],Service[salt-minion],Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22] [00:33:37] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:37:40] PROBLEM - db4 Current Load on db4 is CRITICAL: CRITICAL - load average: 22.67, 15.78, 7.79 [00:37:58] PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 1 backends are down. mw3 [00:38:00] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb [00:38:04] PROBLEM - mw2 MediaWiki Rendering on mw2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:38:16] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb [00:38:57] PROBLEM - db4 Disk Space on db4 is WARNING: DISK WARNING - free space: / 31379 MB (8% inode=95%); [00:39:55] RECOVERY - cp3 Varnish Backends on cp3 is OK: All 6 backends are healthy [00:39:57] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [00:40:00] RECOVERY - mw2 MediaWiki Rendering on mw2 is OK: HTTP OK: HTTP/1.1 200 OK - 18586 bytes in 0.310 second response time [00:40:15] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [00:43:34] RECOVERY - db4 Current Load on db4 is OK: OK - load average: 1.13, 6.59, 6.32 [00:53:43] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 9 failures. Last run 2 minutes ago with 9 failures. Failed resources (up to 3 shown): Service[salt-minion],Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666] [01:03:37] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:13:37] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 9 failures. Last run 2 minutes ago with 9 failures. Failed resources (up to 3 shown): Service[salt-minion],Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666] [01:41:37] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 2 seconds ago with 0 failures [02:13:40] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 9 failures. Last run 2 minutes ago with 9 failures. Failed resources (up to 3 shown): Service[salt-minion],Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666] [02:33:37] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:43:38] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 2 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [03:03:37] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [03:33:41] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 2 minutes ago with 12 failures. Failed resources (up to 3 shown): Service[bacula-fd],Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666] [03:48:06] PROBLEM - cp3 Disk Space on cp3 is CRITICAL: DISK CRITICAL - free space: / 1442 MB (5% inode=94%); [03:50:08] PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 1465 MB (6% inode=94%); [03:51:38] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 7 seconds ago with 0 failures [04:04:10] PROBLEM - cp3 Disk Space on cp3 is CRITICAL: DISK CRITICAL - free space: / 1445 MB (5% inode=94%); [04:06:10] PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 1446 MB (6% inode=94%); [04:14:13] PROBLEM - cp3 Disk Space on cp3 is CRITICAL: DISK CRITICAL - free space: / 1444 MB (5% inode=94%); [04:33:39] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 11 failures. Last run 2 minutes ago with 11 failures. Failed resources (up to 3 shown): Package[openssh-client],Package[openssh-server],Service[ssh],Package[exim4] [04:41:37] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 4 seconds ago with 0 failures [04:53:37] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 9 failures. Last run 2 minutes ago with 9 failures. Failed resources (up to 3 shown): Service[salt-minion],Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666] [05:03:37] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:13:37] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 2 minutes ago with 10 failures. Failed resources (up to 3 shown): Service[bacula-fd],Service[salt-minion],Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22] [06:21:37] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 3 seconds ago with 0 failures [06:26:39] PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2398 MB (9% inode=94%); [06:53:38] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 2 minutes ago with 10 failures. Failed resources (up to 3 shown): Service[bacula-fd],Service[salt-minion],Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22] [07:01:37] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 4 seconds ago with 0 failures [07:18:56] [02miraheze/ManageWiki] 07JohnFLewis pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/Jebbc [07:18:58] [02miraheze/ManageWiki] 07JohnFLewis 0328ad7a0 - add conversion script [07:21:27] Hello sleepy-rhino! If you have any questions, feel free to ask and someone should answer soon. [07:22:47] [02miraheze/mediawiki] 07JohnFLewis pushed 031 commit to 03REL1_34 [+0/-0/±1] 13https://git.io/Jebbl [07:22:49] [02miraheze/mediawiki] 07JohnFLewis 03fc98d73 - update MW [07:44:31] [02miraheze/ManageWiki] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jebbr [07:44:33] [02miraheze/ManageWiki] 07JohnFLewis 03f43cba9 - adjust how extensions json is built [07:44:55] [02miraheze/mediawiki] 07JohnFLewis pushed 031 commit to 03REL1_34 [+0/-0/±1] 13https://git.io/Jebbo [07:44:56] [02miraheze/mediawiki] 07JohnFLewis 035250c48 - MW [07:45:13] !log create mw_settings on mhglobal [07:45:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [07:53:37] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 11 failures. Last run 2 minutes ago with 11 failures. Failed resources (up to 3 shown): Package[openssh-client],Package[openssh-server],Service[ssh],Package[exim4] [08:03:37] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:04:26] JohnLewis: can you sort of viewing policy for https://phabricator.miraheze.org/T4982 [08:04:27] [ ⚓ T4982 Request Configuration Changes of CategoryTree (private wiki) ] - phabricator.miraheze.org [08:24:40] PROBLEM - cp4 Current Load on cp4 is CRITICAL: CRITICAL - load average: 1.44, 2.01, 1.46 [08:25:25] sleepy-rhino: I don't see a need? [08:26:40] RECOVERY - cp4 Current Load on cp4 is OK: OK - load average: 0.08, 0.95, 1.15 [08:32:42] PROBLEM - cp4 Current Load on cp4 is CRITICAL: CRITICAL - load average: 2.49, 2.97, 2.27 [08:36:42] PROBLEM - cp4 Current Load on cp4 is WARNING: WARNING - load average: 1.23, 1.44, 1.78 [08:42:41] RECOVERY - cp4 Current Load on cp4 is OK: OK - load average: 0.97, 1.33, 1.57 [09:33:37] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 2 minutes ago with 10 failures. Failed resources (up to 3 shown): Service[bacula-fd],Service[salt-minion],Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22] [09:41:39] PROBLEM - cp4 Current Load on cp4 is WARNING: WARNING - load average: 1.24, 1.78, 1.52 [09:41:40] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 7 seconds ago with 0 failures [09:43:42] RECOVERY - cp4 Current Load on cp4 is OK: OK - load average: 0.33, 1.03, 1.28 [10:13:47] PROBLEM - cp4 Current Load on cp4 is CRITICAL: CRITICAL - load average: 6.62, 3.99, 2.15 [10:15:47] PROBLEM - cp4 Current Load on cp4 is WARNING: WARNING - load average: 0.34, 1.93, 1.72 [10:17:47] RECOVERY - cp4 Current Load on cp4 is OK: OK - load average: 0.37, 1.12, 1.43 [10:25:37] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 11 failures. Last run 3 minutes ago with 11 failures. Failed resources (up to 3 shown): Package[openssh-client],Package[openssh-server],Service[ssh],Package[exim4] [10:31:37] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 3 seconds ago with 0 failures [10:42:59] PROBLEM - cp4 Current Load on cp4 is CRITICAL: CRITICAL - load average: 1.20, 3.19, 2.54 [10:49:08] PROBLEM - cp4 Current Load on cp4 is WARNING: WARNING - load average: 0.70, 1.20, 1.76 [10:51:07] RECOVERY - cp4 Current Load on cp4 is OK: OK - load average: 0.71, 1.07, 1.60 [11:11:50] JohnLewis: say that then [11:12:11] You did [12:23:37] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 2 minutes ago with 10 failures. Failed resources (up to 3 shown): Service[bacula-fd],Service[salt-minion],Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22] [12:31:38] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 1 second ago with 0 failures [13:07:11] PROBLEM - cp4 Current Load on cp4 is WARNING: WARNING - load average: 0.50, 1.93, 1.84 [13:11:08] RECOVERY - cp4 Current Load on cp4 is OK: OK - load average: 0.85, 1.24, 1.55 [13:33:37] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[salt-minion] [13:41:37] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 8 seconds ago with 0 failures [13:52:50] Hello quirc! If you have any questions, feel free to ask and someone should answer soon. [14:03:37] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100],Exec[ufw-allow-tcp-from-185.52.1.76-to-any-port-8081] [14:11:37] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 5 seconds ago with 0 failures [14:33:37] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 9 failures. Last run 2 minutes ago with 9 failures. Failed resources (up to 3 shown): Service[salt-minion],Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666] [14:49:54] PROBLEM - test1 Puppet on test1 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 7 minutes ago with 0 failures [14:51:40] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 13 seconds ago with 0 failures [15:25:20] https://meta.miraheze.org/wiki/Stewards%27_noticeboard#The_.22Reception.22_Wikis_need_to_go. paladox Reception123 PuppyKun sleepy-rhino review? [15:25:21] [ Stewards' noticeboard - Miraheze Meta ] - meta.miraheze.org [15:25:46] * sleepy-rhino looks [15:26:17] Okay, I'm just a little confused by the text [15:29:28] hispano76: that's an unprecedented situation but I'd say it's something for general community sanctions to apply. Citing the closest precedent as that against fungster) [15:30:30] huh? [15:30:44] fungster was sanctioned due to his actions on meta. [15:30:55] paladox: we also closed his wiki [15:34:50] Due to his actions on meta. [15:35:04] hispano76: citing https://meta.miraheze.org/wiki/Community_noticeboard/Archive_7#Proposal_2, precedent is that the community could close a wiki if they wanted to and had reason to [15:35:05] [ Community noticeboard/Archive 7 - Miraheze Meta ] - meta.miraheze.org [15:35:33] that is not a precdent. [15:35:44] paladox: no because it deliberately impersonated users without their permission and he originally refused to take content down on request. [15:35:46] As far as i'm aware [15:36:58] paladox: precedent = earlier event to be regarded as a guide. Given it happened then and no formal wiki closure by request policy exists. I'm using it as a guide for what to do now. [15:44:11] sleepy-rhino: i spoke with another staff which says there is no precedents for the community to shut down a wiki [15:45:19] paladox: ok [15:46:56] paladox: does that "staffer" want to say what they would do? [15:47:29] * sleepy-rhino also notes you spoke to a "sysadmin" or board member" not staff. Miraheze doesn't have "staff" [15:47:37] s/b/"b [15:47:37] sleepy-rhino meant to say: also notes you spoke to a "sysadmin" or "board member" not staff. Miraheze doesn't have "staff" [15:48:44] sleepy-rhino: staff@ still exists afaik, please don't pointlessly nitpick. [15:49:18] PuppyKun: it redirects now to tech@ [15:50:29] :) it's still an alias and still refers to a group of people. (although admittedly it's ambiguous, it's still nitpicking, it doesn't really matter the exact role of the person paladox was speaking to [15:50:48] Anyways since you asked and I was the person, [15:50:49] 10:44:55 AM <+PuppyKun> I don't want there to be a precedent for the community just closing wikis, but they did bring something to our attention and I would like there to be a precedent for staff enforcing the content policy if there is an issue. [15:51:12] emphasis on /if there is an issue/ I'm sure we'll review it in due time and that's really all there is ot say [15:51:14] to say* [15:52:20] PuppyKun: not "just closing wikis" - when there's a clear reason. The Znotch case had its reason and the content policy was clarified to explicitly include that issue as an example. Good luck reviewing that paragraph then [15:53:15] and yes it's still an alias and refers to an ambiguous group of people but is slowly being taken off use. [15:54:10] Is it? [15:54:14] (@ slowly) [15:55:33] PuppyKun: yes, Miraheze doesn't have staff at all. Legally, the term is wrong. That's why it was taken off the emails and the page renamed. Slowly because we have 4.5 years of history where people have used it so habit and docs need changing over time. [15:56:06] sleepy-rhino: but, what's the clear reason? if there was a clear reason you don't think any one of numerous volunteers would have done something by now? [15:56:08] They just can't exist, no matter how one would think that certain things are better or worse than others. It has come to the point that users have gotten into too many flame wars and have become biased, strict, belligerent, impolite, cruel, disdainful, unfair and have conflicting opinions towards one another. They were also filled with trolls and vandals. These wikis are becoming the opposite of what they were made to be, [15:56:28] none of that sounds illegal to me and local wiki management is responsible for managing users on it's wiki [15:56:50] note that immediately after the part i quoted it does mention tou violations, but that's currently unproven or they wouldn't still be up [15:57:22] PuppyKun: I'm not sure there is. That's why if we wanted to we could debate whether there is. If you can prove a reason to close it, do it. If not, maybe we should have a conversation about it. [15:57:33] sleepy-rhino: for what it's worth i personally dislike the entire collection, but that doesn't mean i can just go delete them all. [15:57:40] ^ [15:58:12] PuppyKun: I know that. That's why unanimous action isn't taken in these cases except in an emergency. [15:58:23] also re staff time and policing content: yes there have been numerous complaints about some of those wikis and that /could/ be a reason to remove them, but honestly this conversation has taken longer than I've ever looked at an of those wikis so [15:59:18] sleepy-rhino: I'm not opposed to the community pointing out potential issues, in fact I'm grateful for it, however Terms of Use enforcement & co is not really up to the community, and messages and discussions like that should not really be public (IMO) [15:59:20] I only look at them when it spills onto meta and I nosey so likely same [15:59:31] a better option i think would have been to make a list of specific compaints/issues/etc and email us [15:59:42] PuppyKun: not always [15:59:56] but ToU 9/10 time is for private discussions [16:00:01] There's no real "debate" and the community has no real input. [16:00:08] It's either against the terms of use or it's not. [16:00:17] ^ [16:00:35] Now admittedly the ToU is likely open to some interpretation (re: staff time and other clauses for closing wikis) but that's to be debated by the people enforcing the ToU, not the community [16:00:46] true for ToU. CP is a partly community policy. [16:08:17] PROBLEM - cp4 Current Load on cp4 is WARNING: WARNING - load average: 0.43, 1.81, 1.58 [16:10:17] RECOVERY - cp4 Current Load on cp4 is OK: OK - load average: 1.62, 1.31, 1.40 [16:24:47] trying to write an authentication library is a big fat uwu [16:27:58] I feel like this is slowly becoming proof I shouldn't do security reviews. :D good thing I don't. (also it's another language) [19:13:39] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [19:21:37] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 10 seconds ago with 0 failures [19:40:18] PROBLEM - wiki.exnihilolinux.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.exnihilolinux.org' expires in 15 day(s) (Sun 12 Jan 2020 07:36:27 PM GMT +0000). [19:40:33] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeNIz [19:40:34] [02miraheze/ssl] 07MirahezeSSLBot 038b5a98a - Bot: Update SSL cert for wiki.exnihilolinux.org [19:40:49] PROBLEM - www.lab612.at - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.lab612.at' expires in 15 day(s) (Sun 12 Jan 2020 07:38:24 PM GMT +0000). [19:41:03] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeNI2 [19:41:05] [02miraheze/ssl] 07MirahezeSSLBot 038a278b3 - Bot: Update SSL cert for www.lab612.at [19:43:54] PROBLEM - ndg.nenawiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'nenawiki.org' expires in 15 day(s) (Sun 12 Jan 2020 07:41:13 PM GMT +0000). [19:44:07] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeNI6 [19:44:08] [02miraheze/ssl] 07MirahezeSSLBot 03ad160bb - Bot: Update SSL cert for ndg.nenawiki.org [19:44:27] PROBLEM - nenawiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'nenawiki.org' expires in 15 day(s) (Sun 12 Jan 2020 07:41:13 PM GMT +0000). [19:44:41] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeNIP [19:44:42] [02miraheze/ssl] 07MirahezeSSLBot 0307293cb - Bot: Update SSL cert for nenawiki.org [19:47:31] PROBLEM - www.sdiy.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'sdiy.info' expires in 15 day(s) (Sun 12 Jan 2020 07:44:31 PM GMT +0000). [19:47:56] PROBLEM - sdiy.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'sdiy.info' expires in 15 day(s) (Sun 12 Jan 2020 07:44:31 PM GMT +0000). [19:48:10] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeNIM [19:48:11] [02miraheze/ssl] 07MirahezeSSLBot 034d151c2 - Bot: Update SSL cert for sdiy.info [19:52:13] RECOVERY - wiki.exnihilolinux.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.exnihilolinux.org' will expire on Thu 26 Mar 2020 06:40:25 PM GMT +0000. [19:52:24] RECOVERY - nenawiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'nenawiki.org' will expire on Thu 26 Mar 2020 06:44:34 PM GMT +0000. [19:53:31] RECOVERY - www.sdiy.info - LetsEncrypt on sslhost is OK: OK - Certificate 'sdiy.info' will expire on Thu 26 Mar 2020 06:48:03 PM GMT +0000. [19:53:51] RECOVERY - ndg.nenawiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'nenawiki.org' will expire on Thu 26 Mar 2020 06:44:34 PM GMT +0000. [19:53:52] RECOVERY - sdiy.info - LetsEncrypt on sslhost is OK: OK - Certificate 'sdiy.info' will expire on Thu 26 Mar 2020 06:48:03 PM GMT +0000. [19:54:48] RECOVERY - www.lab612.at - LetsEncrypt on sslhost is OK: OK - Certificate 'www.lab612.at' will expire on Thu 26 Mar 2020 06:40:57 PM GMT +0000. [20:13:37] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 2 minutes ago with 10 failures. Failed resources (up to 3 shown): Service[bacula-fd],Service[salt-minion],Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22] [20:21:38] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 7 seconds ago with 0 failures [20:27:25] PROBLEM - db5 Disk Space on db5 is WARNING: DISK WARNING - free space: / 20076 MB (10% inode=99%); [20:33:38] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 9 failures. Last run 2 minutes ago with 9 failures. Failed resources (up to 3 shown): Service[salt-minion],Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666] [20:41:37] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 16 seconds ago with 0 failures [22:03:37] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 2 minutes ago with 10 failures. Failed resources (up to 3 shown): Service[bacula-fd],Service[salt-minion],Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22] [22:11:37] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 14 seconds ago with 0 failures [22:51:31] RECOVERY - db5 Disk Space on db5 is OK: DISK OK - free space: / 20274 MB (11% inode=99%); [23:26:03] PROBLEM - db5 Disk Space on db5 is WARNING: DISK WARNING - free space: / 20078 MB (10% inode=99%); [23:53:38] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[ufw-enable]