[00:03:51] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:40:08] PROBLEM - cp3 Disk Space on cp3 is CRITICAL: DISK CRITICAL - free space: / 1445 MB (5% inode=94%); [00:53:53] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 2 minutes ago with 12 failures. Failed resources (up to 3 shown): Package[openssh-server],Service[ssh],Package[exim4],Service[postfix] [01:03:50] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:05:52] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 3 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [02:11:50] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 6 seconds ago with 0 failures [02:35:51] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 3 minutes ago with 12 failures. Failed resources (up to 3 shown): Package[openssh-server],Service[ssh],Package[exim4],Service[postfix] [02:43:50] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [03:00:50] PROBLEM - db4 Disk Space on db4 is WARNING: DISK WARNING - free space: / 40281 MB (10% inode=96%); [03:03:50] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 2 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [03:13:53] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [03:33:55] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 11 failures. Last run 2 minutes ago with 11 failures. Failed resources (up to 3 shown): Service[ssh],Package[exim4],Service[postfix],Exec[ufw-logging-low] [03:41:50] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 2 seconds ago with 0 failures [04:33:52] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 7 failures. Last run 2 minutes ago with 7 failures. Failed resources (up to 3 shown): Service[ssh],Package[exim4],Service[postfix],Exec[ufw-logging-low] [05:03:53] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:23:51] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 11 failures. Last run 2 minutes ago with 11 failures. Failed resources (up to 3 shown): Service[ssh],Package[exim4],Service[postfix],Exec[ufw-logging-low] [05:41:50] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 5 seconds ago with 0 failures [06:26:08] PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2299 MB (9% inode=94%); [08:13:54] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 2 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [08:22:01] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 26 seconds ago with 0 failures [08:34:06] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 2 minutes ago with 12 failures. Failed resources (up to 3 shown): Package[openssh-server],Service[ssh],Package[exim4],Service[postfix] [08:41:57] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 19 seconds ago with 0 failures [08:45:46] Relay [08:53:50] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 2 minutes ago with 12 failures. Failed resources (up to 3 shown): Package[openssh-server],Service[ssh],Package[exim4],Service[postfix] [09:11:50] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 10 seconds ago with 0 failures [09:43:54] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 11 failures. Last run 2 minutes ago with 11 failures. Failed resources (up to 3 shown): Service[ssh],Package[exim4],Service[postfix],Exec[ufw-logging-low] [09:49:16] Hello luis2019-2! If you have any questions, feel free to ask and someone should answer soon. [09:49:24] Hi [09:51:50] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 17 seconds ago with 0 failures [09:52:49] Hi!! RhinosF1 Please, approve me a TestWiki Request. [09:54:13] luis2019-2: we already have test.miraheze.org [09:54:36] RhinosF1 testwiki.miraheze.org approve me please. [09:57:30] luis2019-2: unfortunately, you need to use the current testwiki I linked above [09:59:31] RhinosF1 Please, grant me a administrator permissions in PublicTestWiki. [10:01:51] luis2019-2: please wait one moment [10:07:32] luis2019-2: given you it for 24 hours, please ask again once it expires and stick to testing policy. [10:15:55] RhinosF1 Lock me request. [10:16:46] RhinosF1 Bye and ban me [10:20:55] Hello He-Man! If you have any questions, feel free to ask and someone should answer soon. [10:21:49] He-Man: look at the last 3 bans made [10:22:07] yes I'm looking [10:22:07] *!*@144.178.134.121 [10:23:08] try /ban *!*@78.30.* [10:23:56] is it JPGumpy? [10:24:35] Not sure as i wasnt an op then [10:25:02] He-Man: 78.30 wouldnt catch latest [10:25:47] well here's the thing [10:26:04] it's Really easy to evade a ban by using multiple IRC clients (web, phone, etc) [10:26:47] however ima PM you a link with tips on how to deal with em [10:27:07] K [10:27:22] Reception123: can I check cvt wiki to see if void did a profile on him? [10:27:57] RhinosF1: see -staff [11:42:48] Hello bs! If you have any questions, feel free to ask and someone should answer soon. [12:03:50] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 2 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [12:11:50] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 6 seconds ago with 0 failures [12:23:54] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 2 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [12:31:50] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 11 seconds ago with 0 failures [12:43:50] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 2 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [12:51:52] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 14 seconds ago with 0 failures [12:53:41] Hello publictestwiki9! If you have any questions, feel free to ask and someone should answer soon. [12:53:45] Register it there and request permissions. [12:55:17] Register it there and request permissions. [12:55:19] Register it there and request permissions. [12:55:20] Register it there and request permissions. [12:56:13] paladox bye [12:56:17] paladox ban me [13:03:52] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 2 minutes ago with 12 failures. Failed resources (up to 3 shown): Package[openssh-server],Service[ssh],Package[exim4],Service[postfix] [13:11:55] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 7 seconds ago with 0 failures [13:30:37] grumble: publictestwiki9 doesnt look to be same person but spammer none the less [13:53:50] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 2 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [13:55:11] grumble: its about 50/50 the same person id guess looking [14:03:10] Sorry discorders, its down again, https://status.discordapp.com/incidents/j4tqwdq910nb [14:03:10] [ Discord Status - Elevated API Errors ] - status.discordapp.com [14:21:50] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 10 seconds ago with 0 failures [16:13:54] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[ops_ensure_members] [16:41:50] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 3 seconds ago with 0 failures [17:13:55] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 11 failures. Last run 2 minutes ago with 11 failures. Failed resources (up to 3 shown): Service[ssh],Package[exim4],Service[postfix],Exec[ufw-logging-low] [17:21:51] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 4 seconds ago with 0 failures [18:11:56] hii [18:12:24] Hi [18:46:41] hi [18:47:15] JohnLewis: can you check the dns repo? [18:48:33] There is a pr that needs an answer [19:23:54] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 11 failures. Last run 2 minutes ago with 11 failures. Failed resources (up to 3 shown): Service[ssh],Package[exim4],Service[postfix],Exec[ufw-logging-low] [19:31:52] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 11 seconds ago with 0 failures [20:34:56] [02dns] 07JohnFLewis reviewed pull request 03#119 commit - 13https://git.io/JeSnh [20:35:07] [02dns] 07JohnFLewis reviewed pull request 03#119 commit - 13https://git.io/JeSce [20:43:14] JohnLewis: https://phabricator.miraheze.org/T4966 and what do you mean by domain delegation? [20:43:15] [ ⚓ T4966 Fix rename of User:IAmEiji to User:NoFace ] - phabricator.miraheze.org [20:44:15] RhinosF1: not miraheze.org as it's the master domain [20:44:16] JohnLewis: 2019-12-07 - 14:08:45UTC ask JohnLewis can you merge https://github.com/miraheze/dns/pull/119 (T4536) [20:44:30] ZppixBot: he’s on it [20:44:42] JohnLewis: what would you prefer and I’ll generate it [20:45:07] SPF|Cloud's is fine [20:45:22] One moment [20:45:38] I think he’s saying do SPF|Cloud [20:45:39] Suggestions [20:46:47] paladox: I got it [20:48:05] JohnLewis: sent you new values, not sure what to do for a separate domain in terms of which zone to add it to [20:48:45] [02dns] 07RhinosF1 commented on pull request 03#119: Add mailerlite TXT and SPF records to sign mail - 13https://git.io/JeSc2 [20:49:09] miraheze.org [20:49:42] JohnLewis: where in the file [20:49:58] under the same SPF and DKIM sections [20:50:11] Let me try [20:53:31] [02dns] 07RhinosF1 synchronize pull request 03#119: Add mailerlite TXT and SPF records to sign mail - 13https://git.io/JeyST [20:53:40] JohnLewis: ^?? [20:54:38] new line between sections [20:54:47] also you're doing ml._domainkey.lists. [20:55:06] JohnLewis: okay and does that need the full domain or? [20:55:21] no dot [20:55:28] Ah [20:56:10] [02dns] 07RhinosF1 synchronize pull request 03#119: Add mailerlite TXT and SPF records to sign mail - 13https://git.io/JeyST [20:56:20] JohnLewis: ^ [20:57:30] Are you using tabs or spaces? [20:58:07] JohnLewis: spaces I’m mobile atm [20:58:50] mkay [20:59:26] JohnLewis: it’s easier to use spaces from a mobile, no tab key [20:59:48] But of course we don't use spaces in that repo [21:00:08] JohnLewis: how many spaces to a tab? 4 or 8 [21:02:19] PROBLEM - munwiki.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'munwiki.info' expires in 15 day(s) (Tue 24 Dec 2019 08:59:07 PM GMT +0000). [21:02:33] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeSCf [21:02:35] [02miraheze/ssl] 07MirahezeSSLBot 03c0fe55e - Bot: Update SSL cert for munwiki.info [21:03:20] Copy the tabs [21:04:24] paladox: k [21:08:02] [02dns] 07RhinosF1 synchronize pull request 03#119: Add mailerlite TXT and SPF records to sign mail - 13https://git.io/JeyST [21:08:21] JohnLewis, paladox: I think they copied right [21:08:28] Paste button was being awkward [21:09:31] ml._domainkey.lists [21:09:32] Dosen’t that make it ml.miraheze.org.list? [21:10:21] paladox: I’m just doing what makes sense or what I’m told [21:14:20] RECOVERY - munwiki.info - LetsEncrypt on sslhost is OK: OK - Certificate 'munwiki.info' will expire on Sat 07 Mar 2020 08:02:27 PM GMT +0000. [21:14:21] JohnLewis: ? [21:14:31] I have no idea where paladox got that idea from [21:14:39] Ok [21:14:54] hence the question :) [21:15:25] JohnLewis: so is it good to merge? [21:19:03] sure [21:19:21] JohnLewis: can you then [21:19:48] And should we have anything sent to lists.miraheze.org back as an email sent back somewhere [21:28:59] Test relay [21:29:21] Is it working [21:29:39] @cosyizumi#5190 looks to be back now [21:29:44] Yep [21:38:02] Marco [21:38:06] Polo [21:40:03] [02dns] 07JohnFLewis closed pull request 03#119: Add mailerlite TXT and SPF records to sign mail - 13https://git.io/JeyST [21:40:05] [02miraheze/dns] 07JohnFLewis pushed 035 commits to 03master [+0/-0/±5] 13https://git.io/JeSWn [21:40:06] [02miraheze/dns] 07RhinosF1 0391f18a8 - Add mailerlite TXT and SPF records to sign mail [21:40:08] [02miraheze/dns] 07RhinosF1 03db54327 - Change to lists. [21:40:09] [02miraheze/dns] 07RhinosF1 03dbcd194 - Per feedback [21:40:11] [02miraheze/dns] ... and 2 more commits. [21:40:13] Yey [21:40:39] Now to wait for it to push to the global servers [21:48:01] m a i l i n g l i s t s ? [21:48:22] apap: coming soon [21:48:36] fun stuff [21:49:04] apap: very [21:49:11] Probably very soon [21:50:53] RhinosF1: how long is "very soon" [21:51:28] apap: whenever we finish testing the last few things [21:51:34] oh okay [21:52:01] So maybe 1-2 week if the outstanding bugs get resolved [21:53:05] oo that is very soon [21:54:09] JohnLewis: mailerlite doesnt like the DKIM [21:54:48] JohnLewis: actually its found it now, must be still pushing to dns servers [21:55:21] question, is there a policy on giving users a username that's inactive? [21:55:34] like User:Andreas is inactive. can it be taken over? [21:56:09] apap: youd need to ask a steward, it was done once before [21:56:29] RhinosF1: okay, i will [21:56:30] .mhca Andreas [21:56:31] https://meta.miraheze.org/wiki/Special:CentralAuth/Andreas [21:57:55] mmm [21:58:18] nvm [21:58:22] i'll go with -andreas [21:59:01] apap: if stewards allow it, theres no issue. Theres just not much of a precedent [22:01:01] i don't wanna take a username from a 2016 user, i would feel guilty if the account is still in use.. [22:01:22] apap: theyve never actually editied though [22:01:57] If we ask them first, they might be nice about it [22:03:32] it's been three years they wont notice ;) [22:03:52] but that's just me being selfish [22:05:11] They’ve used the account up to mid this year [22:05:19] Or it was merged to meta then [22:05:46] Ask [22:05:55] okay i will [22:50:52] Hi! Here is the list of currently open high priority tasks on Phabricator [22:50:58] No updates for 6 days - https://phabricator.miraheze.org/T4877 - loops not working - authored by Figurehigh, assigned to None [22:51:05] No updates for 17 days - https://phabricator.miraheze.org/T4857 - Thumbnails of recently uploaded images not showing up. Actual image opens up when navigated to. - authored by AngelCakes, assigned to None [22:51:11] No updates for 8 days - https://phabricator.miraheze.org/T4850 - Cookie warnings on several wikis not collapsing - authored by NeoMahler, assigned to None [22:52:06] paladox: ? few of them probably you're area [22:56:17] hi RhinosF1 [22:56:28] hi hispano76 [22:56:46] One resolved [22:56:57] :) [22:57:02] The loops extension is more mw-admins than Me [22:57:28] paladox: the other two are you [22:57:41] What other two? [22:57:53] .tasl t4857 [22:57:57] .task t4857 [22:57:59] https://phabricator.miraheze.org/T4857 - Thumbnails of recently uploaded images not showing up. Actual image opens up when navigated to. [Open] authored by AngelCakes, assigned to None [22:58:03] ^ [22:58:06] and [22:58:15] Oh ok [22:58:16] .task t4850 [22:58:19] https://phabricator.miraheze.org/T4850 - Cookie warnings on several wikis not collapsing [Resolved] authored by NeoMahler, assigned to Paladox [22:58:23] which you fixed [22:58:27] That was resolved [22:58:28] Yup [23:04:20] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 3 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 2400:6180:0:d0::403:f001/cpweb [23:04:42] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 81.4.109.133/cpweb [23:04:45] next goal for miraheze: switch to gerrit for PRs [23:04:51] also could tend to https://phabricator.miraheze.org/T4966 (rename stuck on deleted wikis) [23:04:52] [ ⚓ T4966 Fix rename of User:IAmEiji to User:NoFace ] - phabricator.miraheze.org [23:05:54] I wonder why it keeps getting stuck [23:06:15] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [23:06:23] deleted wikis where the database still exists [23:06:31] not sure what should be done about that [23:06:39] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [23:07:26] Voidwalker: the fix script [23:07:38] but no one is around to run it and monitor [23:07:55] Well it’s a CA script? [23:09:08] yeah, but ideally, this shouldn't be an issue [23:09:54] Yup [23:12:47] PROBLEM - cp2 Stunnel Http for mw3 on cp2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:12:47] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 4 datacenters are down: 107.191.126.23/cpweb, 128.199.139.216/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb [23:14:27] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb [23:14:44] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [23:14:45] RECOVERY - cp2 Stunnel Http for mw3 on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 24640 bytes in 0.390 second response time [23:16:25] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [23:34:07] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 2 minutes ago with 12 failures. Failed resources (up to 3 shown): Package[openssh-server],Service[ssh],Package[exim4],Service[postfix] [23:37:47] .task T4953 [23:37:50] https://phabricator.miraheze.org/T4953 - Evaluate sysadmin training course [Stalled] authored by RhinosF1, assigned to RhinosF1 [23:40:16] is it possible to turn to Gerrit (or GerritHub if hosting Gerrit is too much right now) for future work? [23:41:51] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 15 seconds ago with 0 failures [23:53:50] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 2 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100]