[00:05:47] PROBLEM - bacula2 Bacula Private Git on bacula2 is CRITICAL: CRITICAL: Full, 5399 files, 17.10MB, 2021-02-14 00:05:00 (2.3 weeks ago) [03:33:43] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-5 [+0/-0/±1] 13https://git.io/JthgL [03:33:45] [02miraheze/mw-config] 07Universal-Omega 031b5a753 - Add opk to $wgFileExtensions options [03:33:47] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-5 - 13https://git.io/vbvb3 [03:34:00] [02mw-config] 07Universal-Omega opened pull request 03#3752: Add opk to $wgFileExtensions options (T6918) - 13https://git.io/Jthgt [03:35:07] miraheze/mw-config - Universal-Omega the build passed. [03:47:09] PROBLEM - bacula2 APT on bacula2 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [03:49:10] PROBLEM - bacula2 APT on bacula2 is CRITICAL: APT CRITICAL: 4 packages available for upgrade (3 critical updates). [10:45:26] PROBLEM - ping4 on cp3 is WARNING: PING WARNING - Packet loss = 0%, RTA = 335.63 ms [10:49:33] RECOVERY - ping4 on cp3 is OK: PING OK - Packet loss = 0%, RTA = 251.43 ms [11:55:39] PROBLEM - mail2 IMAP on mail2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:57:52] paladox: ^ [11:58:54] !log restart dovecot [11:58:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:59:39] RECOVERY - mail2 IMAP on mail2 is OK: IMAP OK - 0.006 second response time on 51.195.236.253 port 143 [* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ STARTTLS LOGINDISABLED] Dovecot (Debian) ready.] [12:03:12] paladox: more IPs need banning? [12:04:56] No, doesn't look like the blocking of the ip is working. I've written an email to them anyways. [12:10:10] Okay [13:38:51] PROBLEM - ping4 on cp3 is WARNING: PING WARNING - Packet loss = 0%, RTA = 335.58 ms [13:40:55] RECOVERY - ping4 on cp3 is OK: PING OK - Packet loss = 0%, RTA = 252.81 ms [15:00:54] paladox: for the old domains for the bots project, assuming we get our last few blockers done, could we just have them pointed at geoip!cp and use redirects.yaml to set a redirect [15:01:37] i guess you could [15:02:15] paladox: we might end up doing that assuming I fix our last remaining blockers on not officially using them domains [15:08:01] ok [15:12:19] PROBLEM - vise.dayid.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - vise.dayid.org All nameservers failed to answer the query. [15:12:31] paladox: what code does that return? [15:12:42] code? [15:13:25] paladox: http [15:14:16] Like 301 etc [15:15:57] PROBLEM - ping4 on cp3 is WARNING: PING WARNING - Packet loss = 0%, RTA = 335.67 ms [15:26:30] it returns 301 [15:28:40] Good [16:34:17] RECOVERY - vise.dayid.org - reverse DNS on sslhost is OK: rDNS OK - vise.dayid.org reverse DNS resolves to cp11.miraheze.org [16:42:20] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JtjR7 [16:42:22] [02miraheze/mediawiki] 07paladox 0317a1d39 - Update EditNotify [18:13:20] PROBLEM - services3 APT on services3 is CRITICAL: APT CRITICAL: 27 packages available for upgrade (4 critical updates). [18:20:18] PROBLEM - ns2 APT on ns2 is CRITICAL: APT CRITICAL: 25 packages available for upgrade (4 critical updates). [18:21:34] PROBLEM - db11 APT on db11 is CRITICAL: APT CRITICAL: 65 packages available for upgrade (4 critical updates). [18:21:34] PROBLEM - mem2 APT on mem2 is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [18:21:40] PROBLEM - db12 APT on db12 is CRITICAL: APT CRITICAL: 65 packages available for upgrade (4 critical updates). [18:21:50] PROBLEM - gluster3 APT on gluster3 is CRITICAL: APT CRITICAL: 24 packages available for upgrade (4 critical updates). [18:22:59] PROBLEM - mon2 APT on mon2 is CRITICAL: APT CRITICAL: 10 packages available for upgrade (4 critical updates). [18:23:04] PROBLEM - cp12 APT on cp12 is CRITICAL: APT CRITICAL: 24 packages available for upgrade (4 critical updates). [18:23:52] PROBLEM - db13 APT on db13 is CRITICAL: APT CRITICAL: 36 packages available for upgrade (4 critical updates). [18:23:59] PROBLEM - puppet3 APT on puppet3 is CRITICAL: APT CRITICAL: 27 packages available for upgrade (4 critical updates). [18:24:24] PROBLEM - cp10 APT on cp10 is CRITICAL: APT CRITICAL: 25 packages available for upgrade (4 critical updates). [18:25:54] PROBLEM - gluster4 APT on gluster4 is CRITICAL: APT CRITICAL: 24 packages available for upgrade (4 critical updates). [18:31:32] PROBLEM - graylog2 APT on graylog2 is CRITICAL: APT CRITICAL: 25 packages available for upgrade (4 critical updates). [18:31:33] PROBLEM - mail2 APT on mail2 is CRITICAL: APT CRITICAL: 28 packages available for upgrade (4 critical updates). [18:34:16] PROBLEM - mw10 APT on mw10 is CRITICAL: APT CRITICAL: 29 packages available for upgrade (4 critical updates). [18:34:56] PROBLEM - phab2 APT on phab2 is CRITICAL: APT CRITICAL: 24 packages available for upgrade (4 critical updates). [18:35:04] PROBLEM - mw11 APT on mw11 is CRITICAL: APT CRITICAL: 29 packages available for upgrade (4 critical updates). [18:35:12] PROBLEM - mw9 APT on mw9 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [18:35:17] PROBLEM - rdb3 APT on rdb3 is CRITICAL: APT CRITICAL: 24 packages available for upgrade (4 critical updates). [18:35:49] PROBLEM - test3 APT on test3 is CRITICAL: APT CRITICAL: 31 packages available for upgrade (4 critical updates). [18:35:57] PROBLEM - jobrunner3 APT on jobrunner3 is CRITICAL: APT CRITICAL: 32 packages available for upgrade (4 critical updates). [18:36:02] PROBLEM - mw8 APT on mw8 is CRITICAL: APT CRITICAL: 29 packages available for upgrade (4 critical updates). [18:36:16] PROBLEM - jobrunner4 APT on jobrunner4 is CRITICAL: APT CRITICAL: 30 packages available for upgrade (4 critical updates). [18:37:14] PROBLEM - ldap2 APT on ldap2 is CRITICAL: APT CRITICAL: 24 packages available for upgrade (4 critical updates). [18:37:29] PROBLEM - cp11 APT on cp11 is CRITICAL: APT CRITICAL: 25 packages available for upgrade (4 critical updates). [18:41:41] PROBLEM - services4 APT on services4 is CRITICAL: APT CRITICAL: 27 packages available for upgrade (4 critical updates). [18:42:56] PROBLEM - cp3 APT on cp3 is CRITICAL: APT CRITICAL: 26 packages available for upgrade (4 critical updates). [18:45:11] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/services/compare/57035597d4f8...db45c8f06a25 [18:45:11] [ Comparing 57035597d4f8...db45c8f06a25 · miraheze/services · GitHub ] - github.com [18:45:12] [02miraheze/services] 07MirahezeSSLBot 03db45c8f - BOT: Updating services config for wikis [18:47:28] RECOVERY - cp11 APT on cp11 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [18:47:56] RECOVERY - jobrunner3 APT on jobrunner3 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [18:48:02] PROBLEM - mw8 APT on mw8 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [18:49:02] RECOVERY - cp3 APT on cp3 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [18:49:13] RECOVERY - mw9 APT on mw9 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [18:49:45] !log sudo apt-get upgrade on mw*/cp*/phab2/ldap2 [18:49:46] RECOVERY - test3 APT on test3 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [18:49:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:50:02] RECOVERY - mw8 APT on mw8 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [18:50:15] RECOVERY - jobrunner4 APT on jobrunner4 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [18:50:58] RECOVERY - phab2 APT on phab2 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [18:51:14] RECOVERY - ldap2 APT on ldap2 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [19:00:49] [02GlobalNewFiles] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/GlobalNewFiles [19:00:51] [02miraheze/GlobalNewFiles] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+1/-0/±0] 13https://github.com/miraheze/GlobalNewFiles/commit/55470d521122 [19:00:52] [02miraheze/GlobalNewFiles] 07Universal-Omega 0355470d5 - Add LICENSE [19:00:54] [02GlobalNewFiles] 07Universal-Omega opened pull request 03#6: Add LICENSE - 13https://github.com/miraheze/GlobalNewFiles/pull/6 [19:01:58] miraheze/GlobalNewFiles - Universal-Omega the build passed. [19:03:44] [02miraheze/GlobalNewFiles] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/GlobalNewFiles/compare/55470d521122...4a5d3ddf4980 [19:03:45] [ Comparing 55470d521122...4a5d3ddf4980 · miraheze/GlobalNewFiles · GitHub ] - github.com [19:03:46] [02miraheze/GlobalNewFiles] 07Universal-Omega 034a5d3dd - Update extension.json [19:03:47] [02GlobalNewFiles] 07Universal-Omega synchronize pull request 03#6: Add LICENSE - 13https://github.com/miraheze/GlobalNewFiles/pull/6 [19:05:48] miraheze/GlobalNewFiles - Universal-Omega the build passed. [19:06:30] [02GlobalNewFiles] 07Universal-Omega closed pull request 03#6: Add LICENSE - 13https://github.com/miraheze/GlobalNewFiles/pull/6 [19:06:32] [02miraheze/GlobalNewFiles] 07Universal-Omega pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/GlobalNewFiles/compare/ab3785e958ba...027a20add396 [19:06:33] [ Comparing ab3785e958ba...027a20add396 · miraheze/GlobalNewFiles · GitHub ] - github.com [19:06:33] [02miraheze/GlobalNewFiles] 07Universal-Omega 03027a20a - Add LICENSE (#6) [19:06:35] [02miraheze/GlobalNewFiles] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [19:06:36] [02GlobalNewFiles] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/GlobalNewFiles [19:07:34] miraheze/GlobalNewFiles - Universal-Omega the build passed. [19:09:29] [02miraheze/MirahezeMagic] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/3ae56d623c80...bc82cc760fe4 [19:09:30] [ Comparing 3ae56d623c80...bc82cc760fe4 · miraheze/MirahezeMagic · GitHub ] - github.com [19:09:30] [02miraheze/MirahezeMagic] 07Universal-Omega 03bc82cc7 - Add license-name [19:10:35] miraheze/MirahezeMagic - Universal-Omega the build passed. [19:17:36] PROBLEM - ping6 on cp3 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 269.42 ms [19:19:37] RECOVERY - ping6 on cp3 is OK: PING OK - Packet loss = 0%, RTA = 267.86 ms [19:42:46] PROBLEM - thesimswiki.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - thesimswiki.com All nameservers failed to answer the query. [19:43:04] PROBLEM - wiki.thesimswiki.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.thesimswiki.com All nameservers failed to answer the query. [19:49:25] RECOVERY - thesimswiki.com - reverse DNS on sslhost is OK: rDNS OK - thesimswiki.com reverse DNS resolves to cp12.miraheze.org [19:49:39] [02miraheze/MirahezeMagic] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/c20f23158687...b072f1d851a3 [19:49:40] [ Comparing c20f23158687...b072f1d851a3 · miraheze/MirahezeMagic · GitHub ] - github.com [19:49:40] [02miraheze/MirahezeMagic] 07Universal-Omega 03b072f1d - Add localisation support to sitenotices [19:49:42] [02MirahezeMagic] 07Universal-Omega synchronize pull request 03#214: Move SiteNoticeAfter hooks from mw-config to MirahezeMagic - 13https://github.com/miraheze/MirahezeMagic/pull/214 [19:49:58] RECOVERY - wiki.thesimswiki.com - reverse DNS on sslhost is OK: rDNS OK - wiki.thesimswiki.com reverse DNS resolves to cp11.miraheze.org [19:50:45] miraheze/MirahezeMagic - Universal-Omega the build passed. [22:30:06] RECOVERY - ping4 on cp3 is OK: PING OK - Packet loss = 0%, RTA = 251.77 ms