[00:11:17] PROBLEM - test1 Current Load on test1 is WARNING: WARNING - load average: 0.06, 0.34, 1.88 [00:12:39] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.30, 6.62, 6.05 [00:13:17] RECOVERY - test1 Current Load on test1 is OK: OK - load average: 0.28, 0.30, 1.67 [00:18:39] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 6.24, 6.68, 6.33 [00:22:39] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.98, 7.10, 6.55 [00:28:39] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 6.09, 6.63, 6.53 [05:04:07] PROBLEM - cp4 Current Load on cp4 is CRITICAL: CRITICAL - load average: 2.71, 2.17, 1.20 [05:08:07] RECOVERY - cp4 Current Load on cp4 is OK: OK - load average: 0.73, 1.52, 1.16 [05:08:41] PROBLEM - mw3 Current Load on mw3 is CRITICAL: CRITICAL - load average: 8.36, 6.85, 5.70 [05:10:39] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.14, 6.85, 5.83 [05:11:39] !log purge binary logs before '2019-07-16 03:00:00'; [05:11:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [05:12:39] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 6.58, 6.70, 5.89 [07:12:07] PROBLEM - cp4 Current Load on cp4 is CRITICAL: CRITICAL - load average: 1.60, 2.07, 1.26 [07:14:07] RECOVERY - cp4 Current Load on cp4 is OK: OK - load average: 0.55, 1.56, 1.17 [07:35:11] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.24, 6.53, 6.05 [07:37:07] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 4.89, 5.88, 5.86 [08:32:37] PROBLEM - cp4 Current Load on cp4 is WARNING: WARNING - load average: 1.99, 1.33, 0.94 [08:34:33] PROBLEM - cp4 Current Load on cp4 is CRITICAL: CRITICAL - load average: 2.82, 2.40, 1.41 [08:36:27] PROBLEM - cp4 Current Load on cp4 is WARNING: WARNING - load average: 0.63, 1.74, 1.28 [08:38:22] RECOVERY - cp4 Current Load on cp4 is OK: OK - load average: 0.47, 1.35, 1.19 [09:09:08] !log reception@misc2:/srv/matomo$ sudo /usr/bin/php /srv/matomo/console core:archive --url=http://matmo.miraheze.org/matomo/ > /home/reception/matomo-archive.log [09:09:09] [ 404 Not Found ] - matmo.miraheze.org [09:09:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [09:14:39] PROBLEM - misc2 Current Load on misc2 is WARNING: WARNING - load average: 3.82, 3.33, 2.64 [09:16:39] RECOVERY - misc2 Current Load on misc2 is OK: OK - load average: 3.30, 3.24, 2.68 [09:22:39] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 6.92, 6.57, 6.08 [09:28:39] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 6.24, 6.68, 6.31 [10:34:39] PROBLEM - mw3 Current Load on mw3 is CRITICAL: CRITICAL - load average: 8.06, 6.68, 5.93 [10:34:52] PROBLEM - db4 Current Load on db4 is WARNING: WARNING - load average: 7.04, 5.52, 3.45 [10:36:52] RECOVERY - db4 Current Load on db4 is OK: OK - load average: 4.48, 4.99, 3.51 [10:38:39] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 6.00, 6.81, 6.19 [10:40:39] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 6.02, 6.59, 6.18 [11:02:39] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.50, 6.58, 6.07 [11:04:39] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 5.69, 6.20, 5.99 [11:07:45] !log root@misc2:/var/spool/cron/crontabs# /usr/local/bin/fileLockScript.sh /tmp/matomo_file_locks "/usr/bin/nice -19 /usr/bin/php /srv/matomo/console core:purge-old-archive-data all" [11:07:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [11:21:12] !log MariaDB [piwik]> OPTIMIZE table piwik_archive_blob_2019_06; [11:21:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [12:10:12] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjXjY [12:10:13] [02miraheze/services] 07MirahezeSSLBot 033d66876 - BOT: Updating services config for wikis [12:35:11] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjXjo [12:35:13] [02miraheze/services] 07MirahezeSSLBot 030bea275 - BOT: Updating services config for wikis [13:06:07] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 6.86, 6.63, 6.19 [13:08:04] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 5.57, 6.26, 6.11 [13:42:50] Good morning [13:43:05] morning [13:43:12] hi AmandaCath [13:43:26] Hi Reception123 JohnLewis [14:06:25] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj1vO [14:06:27] [02miraheze/puppet] 07paladox 03789360b - job runner: Make cirrusSearchElasticaWrite use "SEARCH" group [14:08:20] Can someone take a look at T4546 please? [14:08:34] I'm not sure what the process is for stuff like that [14:09:01] i doin't think that should be enabled by default. [14:09:09] Rather wikis should choose whether to enable it. [14:09:53] paladox: is the feature currently in ManageWiki? [14:10:14] nope [14:10:14] https://github.com/miraheze/mw-config/blob/master/ManageWikiExtensions.php [14:10:16] [ mw-config/ManageWikiExtensions.php at master · miraheze/mw-config · GitHub ] - github.com [14:10:57] I don't think it's an extension? [14:11:07] Looks like a config variable to me [14:11:21] ohh they want wgRCWatchCategoryMembership [14:11:38] the setting can be added to https://github.com/miraheze/mw-config/blob/master/ManageWikiSettings.php [14:11:38] [ mw-config/ManageWikiSettings.php at master · miraheze/mw-config · GitHub ] - github.com [14:13:30] Maybe I'm dumb but I still haven't figured out how to work the backend of ManageWiki :/ [14:14:19] AmandaCath you can follow https://github.com/miraheze/mw-config/commit/94bbba4bfed07671cc708be9d2e28fa2bd67a094#diff-009aefb0948b5fb897d79b50c031c9ca [14:14:19] [ add WikiForum settings (#2687) · miraheze/mw-config@94bbba4 · GitHub ] - github.com [14:17:30] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj1vE [14:17:32] [02miraheze/puppet] 07paladox 038292628 - jobrunner: Remove "Search" group [14:26:01] [02mw-config] 07Amanda-Catherine opened pull request 03#2698: Add $wgRCWatchCategoryMembership to ManageWiki T4546 - 13https://git.io/fj1vr [14:27:06] I have no idea if I did that right ^ [14:28:16] paladox JohnLewis please review [14:28:21] ok [14:28:28] Amanda-Catherine/mw-config/patch-1/dea3375 - Amanda The build failed. https://travis-ci.com/Amanda-Catherine/mw-config/builds/119325155 [14:28:38] [02mw-config] 07paladox reviewed pull request 03#2698 commit - 13https://git.io/fj1vi [14:28:58] [02mw-config] 07paladox commented on pull request 03#2698: Add $wgRCWatchCategoryMembership to ManageWiki T4546 - 13https://git.io/fj1vX [14:29:00] Uh oh [14:29:04] Travis failed [14:29:39] [02mw-config] 07paladox reviewed pull request 03#2698 commit - 13https://git.io/fj1vD [14:31:56] [02mw-config] 07Amanda-Catherine synchronize pull request 03#2698: Add $wgRCWatchCategoryMembership to ManageWiki T4546 - 13https://git.io/fj1vr [14:33:27] Dumb network crashes [14:37:15] [02mw-config] 07Amanda-Catherine opened pull request 03#2699: Add $wgRCWatchCategoryMembership T4546 - 13https://git.io/fj1vh [14:40:38] Yay Travis passed that time [14:40:50] Amanda-Catherine/mw-config/patch-2/d332cfb - Amanda The build passed. https://travis-ci.com/Amanda-Catherine/mw-config/builds/119327294 [14:42:32] Paladox [14:42:36] Please merge [14:42:38] * paladox looks [14:42:59] [02mw-config] 07paladox closed pull request 03#2698: Add $wgRCWatchCategoryMembership to ManageWiki T4546 - 13https://git.io/fj1vr [14:43:01] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj1fI [14:43:02] [02miraheze/mw-config] 07Amanda-Catherine 031d6265e - Add $wgRCWatchCategoryMembership to ManageWiki T4546 (#2698) * Add $wgRCWatchCategoryMembership to ManageWiki T4546 * Fix syntax (maybe?) [14:43:11] [02mw-config] 07paladox closed pull request 03#2699: Add $wgRCWatchCategoryMembership T4546 - 13https://git.io/fj1vh [14:43:12] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj1fL [14:43:14] [02miraheze/mw-config] 07Amanda-Catherine 03e11baa0 - Add $wgRCWatchCategoryMembership T4546 (#2699) [14:43:30] paladox: I feel like something is missing from that [14:43:55] JohnLewis nope? [14:43:58] yes [14:44:12] What's missing? [14:44:17] LS.php [14:44:35] JohnLewis she did it in a seperate change : [14:44:38] *:) [14:44:43] https://github.com/miraheze/mw-config/pull/2699/files [14:44:44] [ Add $wgRCWatchCategoryMembership T4546 by Amanda-Catherine · Pull Request #2699 · miraheze/mw-config · GitHub ] - github.com [14:44:55] okay [15:32:09] [02mw-config] 07The-Voidwalker opened pull request 03#2700: typo - 13https://git.io/fj1Jl [15:33:22] [02mw-config] 07paladox closed pull request 03#2700: typo - 13https://git.io/fj1Jl [15:33:24] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj1JR [15:33:25] [02miraheze/mw-config] 07The-Voidwalker 032cb4214 - typo (#2700) [16:26:55] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+4/-0/±1] 13https://git.io/fj1UV [16:26:56] [02miraheze/puppet] 07paladox 03915b4a1 - Add proton as a service Bug: T4445 [16:27:56] PROBLEM - cyberlaw.ccdcoe.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'cyberlaw.ccdcoe.org' expires in 15 day(s) (Thu 01 Aug 2019 04:24:39 PM GMT +0000). [16:28:11] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj1Uw [16:28:13] [02miraheze/ssl] 07MirahezeSSLBot 03feddac5 - Bot: Update SSL cert for cyberlaw.ccdcoe.org [16:30:57] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+4/-0/±1] 13https://git.io/fj1U6 [16:30:58] [02miraheze/puppet] 07paladox 03b641337 - Add proton as a service Bug: T4445 [16:33:56] RECOVERY - cyberlaw.ccdcoe.org - LetsEncrypt on sslhost is OK: OK - Certificate 'cyberlaw.ccdcoe.org' will expire on Mon 14 Oct 2019 03:28:04 PM GMT +0000. [16:36:09] PROBLEM - misc4 Puppet on misc4 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): User[proton] [16:42:04] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+4/-0/±1] 13https://git.io/fj1U9 [16:42:05] [02miraheze/puppet] 07paladox 03428fc5f - Add proton as a service Bug: T4445 [16:44:16] PROBLEM - misc4 proton on misc4 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:46:48] PROBLEM - misc3 Puppet on misc3 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 2 minutes ago with 0 failures [16:48:51] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+4/-0/±1] 13https://git.io/fj1U7 [16:48:53] [02miraheze/puppet] 07paladox 037c140e2 - Add proton as a service Bug: T4445 [16:50:40] RECOVERY - misc3 Puppet on misc3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [16:54:35] PROBLEM - misc3 Puppet on misc3 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 3 minutes ago with 0 failures [17:10:24] !log root@misc3:/srv/restbase# sudo -u restbase git pull [17:10:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:10:48] !log upgrade from ab6b368 to bdc4029 (restbase on misc3) [17:10:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:14:54] PROBLEM - misc3 restbase on misc3 is CRITICAL: connect to address 185.52.1.144 and port 7231: Connection refused [17:18:06] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj1TW [17:18:07] [02miraheze/puppet] 07paladox 03a6f24c0 - Update miraheze_project.yaml [17:19:27] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj1Tl [17:19:28] [02miraheze/puppet] 07paladox 03ce7414f - Update miraheze_project.yaml [17:19:47] RECOVERY - misc3 Puppet on misc3 is OK: OK: Puppet is currently enabled, last run 46 seconds ago with 0 failures [17:48:26] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj1T7 [17:48:27] [02miraheze/puppet] 07paladox 0397d0c0c - Update config.yaml.erb [17:50:54] RECOVERY - misc3 restbase on misc3 is OK: TCP OK - 0.001 second response time on 185.52.1.144 port 7231 [18:02:44] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [18:02:46] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fj1kt [18:02:47] [02miraheze/puppet] 07paladox 0319deb99 - Update config.yaml.erb [18:02:49] [02puppet] 07paladox opened pull request 03#1030: Update config.yaml.erb - 13https://git.io/fj1kq [18:03:41] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fj1km [18:03:43] [02miraheze/puppet] 07paladox 03745a139 - Update restbase.pp [18:03:44] [02puppet] 07paladox synchronize pull request 03#1030: Update config.yaml.erb - 13https://git.io/fj1kq [18:04:24] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+1/-0/±0] 13https://git.io/fj1k3 [18:04:25] [02miraheze/puppet] 07paladox 03c493be3 - Create miraheze_project_sys.yaml [18:04:27] [02puppet] 07paladox synchronize pull request 03#1030: Update config.yaml.erb - 13https://git.io/fj1kq [18:06:58] [02puppet] 07paladox synchronize pull request 03#1030: Update config.yaml.erb - 13https://git.io/fj1kq [18:07:00] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fj1kC [18:07:01] [02miraheze/puppet] 07paladox 035903763 - Update config.yaml.erb [18:08:25] [02puppet] 07paladox synchronize pull request 03#1030: Update config.yaml.erb - 13https://git.io/fj1kq [18:08:26] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fj1kl [18:08:28] [02miraheze/puppet] 07paladox 032d13060 - Update config.yaml.erb [18:08:32] [02puppet] 07paladox closed pull request 03#1030: Update config.yaml.erb - 13https://git.io/fj1kq [18:08:33] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±2] 13https://git.io/fj1k8 [18:08:35] [02miraheze/puppet] 07paladox 030353c36 - Update config.yaml.erb (#1030) * Update config.yaml.erb * Update restbase.pp * Create miraheze_project_sys.yaml * Update config.yaml.erb * Update config.yaml.erb [18:12:54] PROBLEM - misc3 restbase on misc3 is CRITICAL: connect to address 185.52.1.144 and port 7231: Connection refused [18:16:24] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj1kz [18:16:25] [02miraheze/puppet] 07paladox 036b40068 - Update config.yaml.erb [18:18:14] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj1kg [18:18:15] [02miraheze/puppet] 07paladox 036564ce3 - Update config.yaml.erb [18:20:54] RECOVERY - misc3 restbase on misc3 is OK: TCP OK - 0.001 second response time on 185.52.1.144 port 7231 [19:25:19] !log purge binary logs before '2019-07-16 16:00:00'; [19:25:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [19:32:08] [02miraheze/puppet] 07paladox pushed 036 commits to 03master [+0/-1/±7] 13https://git.io/fj1Iw [19:32:09] [02miraheze/puppet] 07paladox 032fd393c - Revert "Update config.yaml.erb" This reverts commit 6564ce369f5c4e0c2eb2d3641c6335b87d69dc77. [19:32:11] [02miraheze/puppet] 07paladox 03fc18bd4 - Revert "Update config.yaml.erb" This reverts commit 6b400689f193fbba4117a52e684b804efe62b227. [19:32:12] [02miraheze/puppet] 07paladox 0357c1338 - Revert "Update config.yaml.erb (#1030)" This reverts commit 0353c36e7bb0dddfb006d904171280a8dc07efd4. [19:32:14] [02miraheze/puppet] ... and 3 more commits. [19:32:36] !log downgrade restbase back to ab6b368 [19:32:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [19:35:29] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj1Ir [19:35:31] [02miraheze/puppet] 07paladox 038a5a183 - Downgrade restbase to ab6b368 [19:38:37] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj1Io [19:38:39] [02miraheze/puppet] 07paladox 03cef4f81 - Update config.yaml.erb [20:12:57] PROBLEM - cp4 Current Load on cp4 is CRITICAL: CRITICAL - load average: 1.59, 2.49, 1.82 [20:14:57] PROBLEM - cp4 Current Load on cp4 is WARNING: WARNING - load average: 0.91, 1.88, 1.67 [20:16:26] Hello [20:16:42] AmandaCath: hi [20:17:51] I was just thinking about how to vote on your Wiki Creator Application [20:18:57] RECOVERY - cp4 Current Load on cp4 is OK: OK - load average: 1.04, 1.44, 1.54 [20:20:04] AmandaCath: What languages do you speak? (Just so I can decide what to out in my reasons) [20:20:56] And 502 on meta, Paladox: any clue why? [20:21:38] RhinosF1: English (Canada, United States, and a tad of British), and intermediate Spanish [20:22:09] (Not sure what that has to do with Wiki Creators, and I also have Babel on my user page) [20:22:48] British :D [20:22:57] PROBLEM - cp4 Current Load on cp4 is WARNING: WARNING - load average: 1.73, 1.94, 1.74 [20:24:09] AmandaCath: more giving me more reason to support (eg. Can help with XX language requests which we have few creators able to do) [20:24:19] Ah [20:24:22] Paladox: meta is back but slow [20:24:57] RECOVERY - cp4 Current Load on cp4 is OK: OK - load average: 1.01, 1.61, 1.64 [20:26:08] RhinosF1: I think it should be working now per the recovery above. [20:26:09] Also did you wait a long period to be getting 502’s? [20:28:56] paladox: yes [20:31:10] Ok, thanks. [20:31:11] I’m not sure what that was that would cause it, but should be resolved now. I’m currently mobile. [20:33:44] PROBLEM - cp4 Current Load on cp4 is WARNING: WARNING - load average: 1.00, 1.75, 1.74 [20:37:38] RECOVERY - cp4 Current Load on cp4 is OK: OK - load average: 0.45, 1.24, 1.55 [20:38:01] AmandaCath: voted support for you, good luck! [20:39:20] 😀 [21:02:51] PROBLEM - misc2 Current Load on misc2 is CRITICAL: CRITICAL - load average: 5.17, 3.30, 2.44 [21:04:51] RECOVERY - misc2 Current Load on misc2 is OK: OK - load average: 2.96, 3.13, 2.48