[00:03:40] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:33:53] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 2 minutes ago with 12 failures. Failed resources (up to 3 shown): Package[openssh-server],Service[ssh],Package[exim4],Service[postfix] [00:43:45] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:05:40] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 3 minutes ago with 12 failures. Failed resources (up to 3 shown): Package[openssh-server],Service[ssh],Package[exim4],Service[postfix] [01:13:39] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:23:37] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 2 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [01:43:34] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:15:38] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[ops_ensure_members] [02:33:34] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:37:20] [02mw-config] 07jidanni opened pull request 03#2825: Update ManageWikiSettings.php - 13https://git.io/JeQJ4 [02:45:34] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 3 minutes ago with 12 failures. Failed resources (up to 3 shown): Package[openssh-server],Service[ssh],Package[exim4],Service[postfix] [02:53:34] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [03:44:11] PROBLEM - cp3 Disk Space on cp3 is CRITICAL: DISK CRITICAL - free space: / 1443 MB (5% inode=94%); [04:05:36] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 3 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [04:13:58] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:05:36] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 3 minutes ago with 12 failures. Failed resources (up to 3 shown): Package[openssh-server],Service[ssh],Package[exim4],Service[postfix] [05:13:36] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:23:35] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 2 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [05:33:34] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:45:34] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 4 failures. Last run 3 minutes ago with 4 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [06:03:34] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:13:34] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 2 minutes ago with 12 failures. Failed resources (up to 3 shown): Package[openssh-server],Service[ssh],Package[exim4],Service[postfix] [06:23:34] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:26:11] PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2347 MB (9% inode=94%); [06:33:39] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 2 minutes ago with 12 failures. Failed resources (up to 3 shown): Package[openssh-server],Service[ssh],Package[exim4],Service[postfix] [06:43:47] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [07:13:39] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 1 minute ago with 3 failures. Failed resources (up to 3 shown): Service[ssh],Package[exim4],Service[postfix] [07:21:35] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 3 seconds ago with 0 failures [07:45:34] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 4 failures. Last run 3 minutes ago with 4 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [08:03:34] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:45:07] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeQIX [08:45:08] [02miraheze/services] 07MirahezeSSLBot 037f71db4 - BOT: Updating services config for wikis [08:45:36] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 4 failures. Last run 3 minutes ago with 4 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [08:53:35] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:15:34] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[ops_ensure_members] [09:23:34] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:35:36] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[gdnsd] [09:53:36] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [10:15:34] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 3 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [10:23:37] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [10:45:34] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 4 failures. Last run 3 minutes ago with 4 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [10:53:34] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [11:45:34] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 3 minutes ago with 2 failures. Failed resources (up to 3 shown): Service[ufw],Service[postfix] [11:53:34] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [12:55:36] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[ops_ensure_members] [12:58:05] [02ssl] 07revi opened pull request 03#246: Remove wikibase.revi.wiki - 13https://git.io/JeQqX [13:03:34] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:15:34] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[ops_ensure_members] [13:21:53] [02mw-config] 07paladox reviewed pull request 03#2824 commit - 13https://git.io/JeQmk [13:22:12] [02mw-config] 07paladox closed pull request 03#2825: Update ManageWikiSettings.php - 13https://git.io/JeQJ4 [13:22:13] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeQmI [13:22:15] [02miraheze/mw-config] 07jidanni 039efa491 - Update ManageWikiSettings.php (#2825) Better help the user know what seconds are being talked about in the other section. https://phabricator.miraheze.org/T4970 (Made with perl for (qw/1 3 7 14 30 180/) { printf "'%d (%d seconds)' => %d,\n", $_, $_ * 60 * 60 * 24, $_; }) [13:22:16] [ ⚓ T4970 Recentchanges 15552000 seconds hard to understand ] - phabricator.miraheze.org [13:23:13] [02MatomoAnalytics] 07paladox synchronize pull request 03#11: Adds `$wgMatomoAnalyticsDisableJS` to allow disabling of JS tracking code globally - 13https://git.io/JeHuc [13:23:25] [02MatomoAnalytics] 07paladox synchronize pull request 03#11: Adds `$wgMatomoAnalyticsDisableJS` to allow disabling of JS tracking code globally - 13https://git.io/JeHuc [13:23:35] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:24:52] vedmaka/MatomoAnalytics/master/c911aad - paladox The build is still failing. https://travis-ci.com/vedmaka/MatomoAnalytics/builds/141068235 [13:26:17] vedmaka/MatomoAnalytics/master/2255345 - paladox The build is still failing. https://travis-ci.com/vedmaka/MatomoAnalytics/builds/141068242 [13:27:27] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JeQmY [13:27:28] [02miraheze/ManageWiki] 07paladox 03acbfcb2 - travis: Upgrade dist to bionic [13:27:30] [02ManageWiki] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vpSns [13:27:37] [02ManageWiki] 07paladox opened pull request 03#121: travis: Upgrade dist to bionic - 13https://git.io/JeQmO [13:29:57] miraheze/ManageWiki/paladox-patch-1/acbfcb2 - paladox The build has errored. https://travis-ci.org/miraheze/ManageWiki/builds/624991407 [13:30:24] [02ManageWiki] 07paladox closed pull request 03#121: travis: Upgrade dist to bionic - 13https://git.io/JeQmO [13:30:25] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeQmG [13:30:27] [02miraheze/ManageWiki] 07paladox 0334c60f3 - travis: Upgrade dist to bionic (#121) [13:30:28] [02miraheze/ManageWiki] 07paladox deleted branch 03paladox-patch-1 [13:30:30] [02ManageWiki] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vpSns [13:30:58] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JeQmc [13:31:00] [02miraheze/ManageWiki] 07paladox 03a82c911 - travis: Stop testing php 7.0 (keeping 7.1, 7.2) [13:31:01] [02ManageWiki] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vpSns [13:31:03] [02ManageWiki] 07paladox opened pull request 03#122: travis: Stop testing php 7.0 (keeping 7.1, 7.2) - 13https://git.io/JeQmC [13:32:42] miraheze/ManageWiki/master/34c60f3 - paladox The build has errored. https://travis-ci.org/miraheze/ManageWiki/builds/624992131 [13:33:27] miraheze/ManageWiki/paladox-patch-1/a82c911 - paladox The build passed. https://travis-ci.org/miraheze/ManageWiki/builds/624992295 [13:34:37] [02ManageWiki] 07paladox closed pull request 03#122: travis: Stop testing php 7.0 (keeping 7.1, 7.2) - 13https://git.io/JeQmC [13:34:38] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeQmB [13:34:40] [02miraheze/ManageWiki] 07paladox 0300e7ddb - travis: Stop testing php 7.0 (keeping 7.1, 7.2) (#122) [13:34:41] [02miraheze/ManageWiki] 07paladox deleted branch 03paladox-patch-1 [13:34:43] [02ManageWiki] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vpSns [13:34:56] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JeQmR [13:34:57] [02miraheze/ManageWiki] 07paladox 037155d85 - travis: Test php 7.3 [13:34:59] [02ManageWiki] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vpSns [13:35:00] [02ManageWiki] 07paladox opened pull request 03#123: travis: Test php 7.3 - 13https://git.io/JeQm0 [13:35:46] [02MatomoAnalytics] 07paladox closed pull request 03#11: Adds `$wgMatomoAnalyticsDisableJS` to allow disabling of JS tracking code globally - 13https://git.io/JeHuc [13:35:48] [02miraheze/MatomoAnalytics] 07paladox pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/JeQmu [13:35:49] [02miraheze/MatomoAnalytics] 07vedmaka 03e285143 - Adds `$wgMatomoAnalyticsDisableJS` to allow disabling of JS tracking code globally (#11) * Adds to allow disabling of JS tracking code globally * Update CHANGELOG * Update extension.json [13:36:43] [02miraheze/MatomoAnalytics] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JeQmz [13:36:44] [02miraheze/MatomoAnalytics] 07paladox 03f4ac883 - travis: Upgrade dist to bionic and remove php 7.0 (keeping 7.1, 7.2) [13:36:46] [02MatomoAnalytics] 07paladox created branch 03paladox-patch-2 - 13https://git.io/fN4LT [13:36:50] [02MatomoAnalytics] 07paladox opened pull request 03#12: travis: Upgrade dist to bionic and remove php 7.0 (keeping 7.1, 7.2) - 13https://git.io/JeQmg [13:37:02] miraheze/ManageWiki/master/00e7ddb - paladox The build passed. https://travis-ci.org/miraheze/ManageWiki/builds/624993054 [13:38:40] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/JeQmV [13:38:42] [02miraheze/puppet] 07Reception123 032351631 - remove rhinosf1 as mw-admin T4930 [13:39:18] [02ManageWiki] 07paladox closed pull request 03#123: travis: Test php 7.3 - 13https://git.io/JeQm0 [13:39:19] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeQmr [13:39:21] [02miraheze/ManageWiki] 07paladox 039f813a8 - travis: Test php 7.3 (#123) [13:39:22] [02ManageWiki] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vpSns [13:39:24] [02miraheze/ManageWiki] 07paladox deleted branch 03paladox-patch-1 [13:40:03] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeQmK [13:40:04] [02miraheze/mw-config] 07Reception123 03a7ba661 - remove rhinosf1 from travis alerts [13:40:28] !log removed RhinosF1 from @GitHub project [13:40:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [13:40:52] miraheze/MatomoAnalytics/paladox-patch-2/f4ac883 - paladox The build passed. https://travis-ci.org/miraheze/MatomoAnalytics/builds/624993668 [13:41:55] !log root@misc1:/home/paladox# deluser rhinosf1 [13:42:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [13:43:04] [02MatomoAnalytics] 07paladox closed pull request 03#12: travis: Upgrade dist to bionic and remove php 7.0 (keeping 7.1, 7.2) - 13https://git.io/JeQmg [13:43:05] [02miraheze/MatomoAnalytics] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeQmP [13:43:07] [02miraheze/MatomoAnalytics] 07paladox 03ef327a2 - travis: Upgrade dist to bionic and remove php 7.0 (keeping 7.1, 7.2) (#12) [13:43:08] [02MatomoAnalytics] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/fN4LT [13:43:10] [02miraheze/MatomoAnalytics] 07paladox deleted branch 03paladox-patch-2 [13:43:25] [02miraheze/MatomoAnalytics] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JeQmX [13:43:27] [02miraheze/MatomoAnalytics] 07paladox 0394c057c - travis: Test php 7.3 [13:43:28] [02MatomoAnalytics] 07paladox created branch 03paladox-patch-2 - 13https://git.io/fN4LT [13:43:30] [02MatomoAnalytics] 07paladox opened pull request 03#13: travis: Test php 7.3 - 13https://git.io/JeQm1 [13:44:16] !log remove rhinosf1 from status.miraheze.wiki [13:44:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [13:45:42] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[gdnsd] [13:45:43] PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[mediawiki-admins_ensure_members] [13:45:56] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[mediawiki-admins_ensure_members] [13:46:46] PROBLEM - mw1 Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[mediawiki-admins_ensure_members] [13:47:15] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeQmS [13:47:17] [02miraheze/puppet] 07paladox 033c1ddbd - Remove rhinos from mw-admin group [13:47:18] Reception123 ^ [13:47:28] PROBLEM - mw3 Puppet on mw3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 4 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[mediawiki-admins_ensure_members] [13:48:04] paladox: my bad, must've forgotten to save the file the second time [13:48:12] :) [13:53:23] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:53:28] RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 3 seconds ago with 0 failures [13:53:39] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 51 seconds ago with 0 failures [13:53:48] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 24 seconds ago with 0 failures [13:54:48] RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:04:50] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 2 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [14:22:43] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 52 seconds ago with 0 failures [14:29:42] Reception123: https://phabricator.miraheze.org/T4930#94849 [14:29:43] [ ⚓ T4930 Off board RhinosF1 ] - phabricator.miraheze.org [14:30:22] Reception123: discord just vanished [14:30:47] But no former sysadmin role [14:31:05] paladox: ^ [14:34:43] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 2 minutes ago with 12 failures. Failed resources (up to 3 shown): Package[openssh-server],Service[ssh],Package[exim4],Service[postfix] [14:35:14] RhinosF1: ah sorry didn't realize [14:35:16] I'll add it [14:35:48] RhinosF1: to answer your Phab reply, test1 has those schema issues so I can't give myself the permission to remove crat = internal error [14:35:57] Reception123: cool, github i’ve had an email about, mail has litterally vanished [14:35:59] Ok [14:36:32] RhinosF1: and when I said staffwiki removed I didn't include the global role [14:36:44] Reception123: ok [14:36:51] RhinosF1: well I'm not sure what the procedure is for mail, paladox took care of that [14:37:16] But afaik yeah you don't have access anymore, though now that I think of it it would make sense that you still keep the content [14:37:43] Reception123: It removed itself from my email client [14:38:22] well you'll have to ask paladox or John since I have no idea about anything related to mail except how to add an account [14:38:54] Reception123: you need to remove me from the -staff invex list [14:39:10] RhinosF1: how do you do that? :P [14:39:30] I knew something was missing, because I tried -I and only -i worked [14:39:54] Reception123: let me look [14:40:11] oh, I thought you knew in that case I can just look [14:41:18] Reception123: via /mode #CHANNEL_NAME +I *!*@your/cloak [14:46:11] Reception123: what about slack as well? [14:52:42] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:58:23] RhinosF1: thanks [14:58:23] Reception123 what issue? [14:58:28] RhinosF1: I don't think I have access there [14:58:35] or I don't know how to use it anyway so paladox ^ [14:58:57] RhinosF1: Internal Error when trying to change MW settings on test1 but that's normal because of the 1.34 thing [14:59:18] Done [14:59:26] deactivated RhinosF1 account on slack [14:59:32] Reception123 that's not normal [15:00:58] paladox: well it is because we have 1.34 running on test1 with the 1.33 tables, no? [15:01:49] nope [15:02:13] paladox: thx [15:02:44] paladox: i very much have slack access [15:03:13] I can see evetything but not send new stuff [15:03:52] Reception123: I still have -staff IRC [15:04:07] well I did what it said, I guess it didn't work [15:04:26] Does someone mind testing something on wikimedia phab? [15:05:26] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03DBConnRef [+0/-0/±1] 13https://git.io/JeQOD [15:05:27] [02miraheze/ManageWiki] 07paladox 03a65845d - Update ManageWikiFormFactory.php [15:05:59] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_34 [+0/-0/±1] 13https://git.io/JeQOS [15:06:00] [02miraheze/mediawiki] 07paladox 033a14115 - Update ManageWiki [15:06:54] RhinosF1: sure [15:09:51] Reception123: paladox just did it [15:12:54] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JeQOF [15:12:55] [02miraheze/ManageWiki] 07paladox 03a582b52 - DBConnRef -> MaintainableDBConnRef [15:12:57] [02ManageWiki] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vpSns [15:12:58] [02ManageWiki] 07paladox opened pull request 03#124: DBConnRef -> MaintainableDBConnRef - 13https://git.io/JeQOb [15:13:59] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JeQON [15:14:01] [02miraheze/ManageWiki] 07paladox 03d5bc9ef - Update ManageWikiFormFactoryBuilder.php [15:14:02] [02ManageWiki] 07paladox synchronize pull request 03#124: DBConnRef -> MaintainableDBConnRef - 13https://git.io/JeQOb [15:14:17] [02ManageWiki] 07paladox closed pull request 03#124: DBConnRef -> MaintainableDBConnRef - 13https://git.io/JeQOb [15:14:19] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03DBConnRef [+0/-0/±2] 13https://git.io/JeQOA [15:14:20] [02miraheze/ManageWiki] 07paladox 03f5a45cc - DBConnRef -> MaintainableDBConnRef (#124) * DBConnRef -> MaintainableDBConnRef * Update ManageWikiFormFactoryBuilder.php [15:14:43] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_34 [+0/-0/±1] 13https://git.io/JeQOx [15:14:45] [02miraheze/mediawiki] 07paladox 036532494 - Update ManageWiki [15:21:31] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeQ3U [15:21:32] [02miraheze/puppet] 07paladox 0326571d6 - Remove elastica_composer [15:22:02] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeQ3k [15:22:04] [02miraheze/puppet] 07paladox 030ef3225 - Add oathauth_composer [15:24:45] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 3 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [15:29:26] [02miraheze/mediawiki] 07paladox pushed 032 commits to 03REL1_34 [+0/-0/±4] 13https://git.io/JeQ3Z [15:29:27] [02miraheze/mediawiki] 07sbassett29 03f0492a1 - rdbms: Log debug message traces as 'exception.trace' instead of 'trace' Code cleanup and hardening (see also: T234014) of Database-related lib code in MediaWiki core. Bug: T233342 Change-Id: I3c968f4f5300374253dc80d99596cac50fbeb59e (cherry picked from commit 2e11b14455b44b8fcfd528da5efcc214902c2ffb) [15:29:29] [02miraheze/mediawiki] 07paladox 03a56185b - Merge branch 'REL1_34' of https://github.com/wikimedia/mediawiki into REL1_34 [15:29:30] [ GitHub - wikimedia/mediawiki: 🌻 The collaborative editing software that runs Wikipedia. This is a mirror from gerrit.wikimedia.org. See https://www.mediawiki.org/wiki/Developer_access for contributing ] - github.com [15:32:02] [02ssl] 07paladox closed pull request 03#246: Remove wikibase.revi.wiki - 13https://github.com/miraheze/ssl/pull/246 [15:32:03] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±1] 13https://git.io/JeQ34 [15:32:05] [02miraheze/ssl] 07revi 03816ca6c - Remove wikibase.revi.wiki (#246) No immediate plan to reuse the same cert, remove from the SSL renew pool to stop Certificate Transparency Warning Signed-off-by: Yongmin Hong [15:34:59] [02ManageWiki] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vpSns [15:35:00] [02miraheze/ManageWiki] 07paladox deleted branch 03paladox-patch-1 [15:35:54] [02miraheze/CreateWiki] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JeQ3E [15:35:55] [02miraheze/CreateWiki] 07paladox 03a91aff5 - travis: Upgrade dist to bionic and remove php 7.0 and add php 7.3 [15:35:57] [02CreateWiki] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vpJTL [15:35:58] [02CreateWiki] 07paladox opened pull request 03#124: travis: Upgrade dist to bionic and remove php 7.0 and add php 7.3 - 13https://git.io/JeQ3u [15:40:54] [02CreateWiki] 07paladox closed pull request 03#124: travis: Upgrade dist to bionic and remove php 7.0 and add php 7.3 - 13https://git.io/JeQ3u [15:40:55] [02miraheze/CreateWiki] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeQ36 [15:40:57] [02miraheze/CreateWiki] 07paladox 0389946f0 - travis: Upgrade dist to bionic and remove php 7.0 and add php 7.3 (#124) [15:40:58] [02CreateWiki] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vpJTL [15:41:00] [02miraheze/CreateWiki] 07paladox deleted branch 03paladox-patch-1 [15:41:35] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeQ3P [15:41:36] [02miraheze/mw-config] 07paladox 039e31be4 - travis: Upgrade dist to bionic [15:41:58] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 23 seconds ago with 0 failures [15:43:23] PROBLEM - wiki.starship.digital - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.starship.digital' expires in 15 day(s) (Mon 30 Dec 2019 03:40:38 PM GMT +0000). [15:43:36] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeQ3y [15:43:38] [02miraheze/ssl] 07MirahezeSSLBot 03b2ba318 - Bot: Update SSL cert for wiki.starship.digital [15:53:11] RECOVERY - wiki.starship.digital - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.starship.digital' will expire on Fri 13 Mar 2020 02:43:31 PM GMT +0000. [16:01:25] !log sudo -u www-data php purgeParserCache.php --wiki=unmusicfestwiki --age=1 [16:01:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [16:03:34] k [16:03:36] whoops [16:03:38] wrongchannel [16:06:28] [02MatomoAnalytics] 07paladox closed pull request 03#13: travis: Test php 7.3 - 13https://git.io/JeQm1 [16:06:29] [02miraheze/MatomoAnalytics] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeQsm [16:06:31] [02miraheze/MatomoAnalytics] 07paladox 03019dafd - travis: Test php 7.3 (#13) [16:06:32] [02MatomoAnalytics] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/fN4LT [16:06:34] [02miraheze/MatomoAnalytics] 07paladox deleted branch 03paladox-patch-2 [16:06:54] [02MatomoAnalytics] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/fN4LT [16:06:55] [02miraheze/MatomoAnalytics] 07paladox deleted branch 03paladox-patch-1 [16:15:25] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[ops_ensure_members] [16:23:25] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [16:57:21] @Stewards see my comment on https://meta.miraheze.org/wiki/Requests_for_global_rights#Se.27s_Request_for_CTV [16:57:25] [ Requests for global rights - Miraheze Meta ] - meta.miraheze.org [16:58:34] SPF|Cloud: ^ [17:25:25] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 3 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [17:33:25] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [17:45:25] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 3 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [17:53:25] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:15:25] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[ops_ensure_members] [18:16:17] JohnLewis: can you remove my test1 and global sysadmin right [18:16:27] Ref the offboard task [18:20:41] Will do in a bit [18:23:27] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:24:19] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 16 failures. Last run 2 minutes ago with 16 failures. Failed resources (up to 3 shown) [18:25:02] PROBLEM - lizardfs6 Puppet on lizardfs6 is CRITICAL: CRITICAL: Puppet has 320 failures. Last run 2 minutes ago with 320 failures. Failed resources (up to 3 shown) [18:25:07] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeQZg [18:25:08] [02miraheze/services] 07MirahezeSSLBot 0323720ad - BOT: Updating services config for wikis [18:25:26] PROBLEM - cp4 Puppet on cp4 is CRITICAL: CRITICAL: Puppet has 276 failures. Last run 3 minutes ago with 276 failures. Failed resources (up to 3 shown) [18:25:37] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Puppet has 290 failures. Last run 3 minutes ago with 290 failures. Failed resources (up to 3 shown) [18:25:51] PROBLEM - mw3 Puppet on mw3 is CRITICAL: CRITICAL: Puppet has 304 failures. Last run 3 minutes ago with 304 failures. Failed resources (up to 3 shown): File[wiki.ciptamedia.org_private],File[athenapedia.org],File[athenapedia.org_private],File[podpedia.org] [18:25:57] PROBLEM - mw1 Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 293 failures. Last run 3 minutes ago with 293 failures. Failed resources (up to 3 shown) [18:26:06] PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Puppet has 309 failures. Last run 3 minutes ago with 309 failures. Failed resources (up to 3 shown): File[sdiy.info],File[sdiy.info_private],File[wiki.gesamtschule-nordkirchen.de],File[wiki.gesamtschule-nordkirchen.de_private] [18:26:59] PROBLEM - cp2 Puppet on cp2 is CRITICAL: CRITICAL: Puppet has 238 failures. Last run 3 minutes ago with 238 failures. Failed resources (up to 3 shown): File[nenawiki.org_private],File[reviwiki.info],File[reviwiki.info_private],File[wiki.tulpa.info] [18:27:05] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 164 failures. Last run 3 minutes ago with 164 failures. Failed resources (up to 3 shown): File[wikipuk.cl_private],File[wikiescola.com.br],File[wikiescola.com.br_private],File[wiki.jacksonheights.nyc] [18:32:55] PROBLEM - cp3 Stunnel Http for mw1 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [18:34:09] RECOVERY - cp4 Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 26 seconds ago with 0 failures [18:34:18] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 32 seconds ago with 0 failures [18:34:47] RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 27 seconds ago with 0 failures [18:34:48] RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 29 seconds ago with 0 failures [18:34:58] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 40 seconds ago with 0 failures [18:34:59] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:35:20] RECOVERY - cp3 Stunnel Http for mw1 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 24661 bytes in 3.862 second response time [18:35:39] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 48 seconds ago with 0 failures [18:35:42] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 24 seconds ago with 0 failures [18:35:53] RECOVERY - lizardfs6 Puppet on lizardfs6 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:36:19] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[ops_ensure_members] [18:42:02] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 11 seconds ago with 0 failures [18:50:07] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeQnU [18:50:09] [02miraheze/services] 07MirahezeSSLBot 034b572d2 - BOT: Updating services config for wikis [19:13:44] * hispano76 greeting RhinosF1 [19:14:02] Hi hispano76 [20:15:25] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 3 minutes ago with 12 failures. Failed resources (up to 3 shown): Package[openssh-server],Service[ssh],Package[exim4],Service[postfix] [20:23:27] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [20:44:44] hum, it would be nice to have a bot that adds the signature of users who forget to sign. There are several in Wikimedia that do this. Any bot that could do this? (ping Zppix Voidwalker ) [20:45:05] ping revi [20:45:25] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[ufw-allow-tcp-from-any-to-any-port-22] [20:46:49] hispano76: it could be done, would need a host [20:51:12] RhinosF1: host? [20:52:47] hispano76: hosting a bot to do that would cost money [20:53:29] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [20:54:55] Um, it doesn't look like according to eswiki (operators of bots*). Unless.. [20:55:04] RhinosF1: :) [20:59:25] hispano76: I mean, WMF host ZppixBot [20:59:36] Void-bot: I don’t know about [20:59:52] Revibot is a cron job on a raspberry pi [21:00:20] And MH-Discord is free host which is why it can go down [21:01:03] By cost money I mean the budget for it doesn’t exist with most of our users so the more resources it needs the more unlikely [21:03:22] RhinosF1: ok :) yup, although I was referring to the current bot operators (according to Special:ListUsers) [21:05:07] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeQCI [21:05:09] [02miraheze/services] 07MirahezeSSLBot 03ff98e4f - BOT: Updating services config for wikis [21:05:34] hispano76: toolforge is available for WMF wiki bots :) [21:05:57] Free, decent hosting that’s pretty limitless [21:15:25] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 3 minutes ago with 8 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22],Exec[ufw-allow-tcp-from-any-to-any-port-5666],Exec[ufw-allow-tcp-from-185.52.3.121-to-any-port-9100] [21:15:27] Well, we'll see which bot operator does it. RhinosF1 [21:33:18] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb [21:33:32] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [21:33:58] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 128.199.139.216/cpweb [21:35:53] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [21:37:13] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [21:43:25] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[ufw-allow-tcp-from-any-to-any-port-53],Exec[ufw-allow-tcp-from-185.52.1.76-to-any-port-3506] [21:53:25] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:04:02] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 3 datacenters are down: 107.191.126.23/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb [23:05:57] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [23:13:52] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 2400:6180:0:d0::403:f001/cpweb [23:15:51] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online