[00:45:23] Hello wm-bot563! If you have any questions feel free to ask and someone should answer soon. [03:45:27] PROBLEM - misc2 Current Load on misc2 is WARNING: WARNING - load average: 3.61, 3.58, 3.17 [03:47:27] RECOVERY - misc2 Current Load on misc2 is OK: OK - load average: 2.79, 3.34, 3.13 [04:02:19] PROBLEM - misc2 Current Load on misc2 is WARNING: WARNING - load average: 3.93, 3.37, 3.18 [04:04:15] RECOVERY - misc2 Current Load on misc2 is OK: OK - load average: 3.29, 3.35, 3.19 [04:33:27] PROBLEM - misc2 Current Load on misc2 is CRITICAL: CRITICAL - load average: 4.44, 3.56, 3.09 [04:35:27] PROBLEM - misc2 Current Load on misc2 is WARNING: WARNING - load average: 3.67, 3.52, 3.13 [04:43:27] PROBLEM - misc2 Current Load on misc2 is CRITICAL: CRITICAL - load average: 4.85, 4.11, 3.51 [04:47:27] PROBLEM - misc2 Current Load on misc2 is WARNING: WARNING - load average: 3.31, 3.71, 3.47 [04:51:27] PROBLEM - misc2 Current Load on misc2 is CRITICAL: CRITICAL - load average: 4.02, 3.84, 3.57 [04:53:27] PROBLEM - misc2 Current Load on misc2 is WARNING: WARNING - load average: 3.84, 3.83, 3.59 [05:00:04] RECOVERY - misc2 Current Load on misc2 is OK: OK - load average: 1.20, 2.77, 3.25 [05:40:13] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjoe1 [05:40:14] [02miraheze/services] 07MirahezeSSLBot 03c4f3103 - BOT: Updating services config for wikis [05:42:40] PROBLEM - cp2 Puppet on cp2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:42:53] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:42:53] PROBLEM - db4 Puppet on db4 is CRITICAL: CRITICAL: Puppet has 18 failures. Last run 2 minutes ago with 18 failures. Failed resources (up to 3 shown) [05:43:02] PROBLEM - lizardfs1 Puppet on lizardfs1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:43:05] PROBLEM - cp4 Puppet on cp4 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:43:07] PROBLEM - elasticsearch1 Puppet on elasticsearch1 is CRITICAL: CRITICAL: Puppet has 17 failures. Last run 2 minutes ago with 17 failures. Failed resources (up to 3 shown) [05:43:12] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:43:21] PROBLEM - lizardfs3 Puppet on lizardfs3 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:43:23] PROBLEM - mw3 Puppet on mw3 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:43:50] PROBLEM - misc4 Puppet on misc4 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:43:55] PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:44:02] PROBLEM - lizardfs2 Puppet on lizardfs2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:44:17] PROBLEM - misc3 Puppet on misc3 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:44:19] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 201 failures. Last run 2 minutes ago with 201 failures. Failed resources (up to 3 shown) [05:44:22] PROBLEM - bacula1 Puppet on bacula1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:44:23] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:44:27] PROBLEM - misc2 Puppet on misc2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:44:28] PROBLEM - mw1 Puppet on mw1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:53:12] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 5 seconds ago with 0 failures [05:53:21] RECOVERY - lizardfs3 Puppet on lizardfs3 is OK: OK: Puppet is currently enabled, last run 12 seconds ago with 0 failures [05:53:50] RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 27 seconds ago with 0 failures [05:54:02] RECOVERY - lizardfs2 Puppet on lizardfs2 is OK: OK: Puppet is currently enabled, last run 55 seconds ago with 0 failures [05:54:17] RECOVERY - misc3 Puppet on misc3 is OK: OK: Puppet is currently enabled, last run 54 seconds ago with 0 failures [05:54:22] RECOVERY - bacula1 Puppet on bacula1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:54:23] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 54 seconds ago with 0 failures [05:54:27] RECOVERY - misc2 Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:54:28] RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 20 seconds ago with 0 failures [05:54:40] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 4 seconds ago with 0 failures [05:54:53] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:54:53] RECOVERY - db4 Puppet on db4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:55:03] RECOVERY - lizardfs1 Puppet on lizardfs1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:55:06] RECOVERY - cp4 Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:55:23] RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:55:55] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:56:19] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:13:06] RECOVERY - elasticsearch1 Puppet on elasticsearch1 is OK: OK: Puppet is currently enabled, last run 36 seconds ago with 0 failures [10:32:35] PROBLEM - misc2 Current Load on misc2 is CRITICAL: CRITICAL - load average: 5.00, 3.75, 3.21 [10:36:27] PROBLEM - misc2 Current Load on misc2 is WARNING: WARNING - load average: 3.67, 3.79, 3.35 [10:42:17] PROBLEM - misc2 Current Load on misc2 is CRITICAL: CRITICAL - load average: 4.16, 3.94, 3.54 [10:44:13] PROBLEM - misc2 Current Load on misc2 is WARNING: WARNING - load average: 3.98, 3.95, 3.59 [10:46:10] PROBLEM - misc2 Current Load on misc2 is CRITICAL: CRITICAL - load average: 4.64, 4.17, 3.71 [10:48:06] PROBLEM - misc2 Current Load on misc2 is WARNING: WARNING - load average: 3.81, 3.99, 3.70 [10:52:00] RECOVERY - misc2 Current Load on misc2 is OK: OK - load average: 1.50, 2.71, 3.24 [11:45:09] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 335 [11:51:10] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [12:40:12] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjoIL [12:40:14] [02miraheze/services] 07MirahezeSSLBot 030b4120c - BOT: Updating services config for wikis [13:49:23] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjoL7 [13:49:24] [02miraheze/mw-config] 07paladox 030eab227 - Set search type to cirrus for buswiki Change-Id: I113591b9551339c4f2b96bfad8df65b70aef67f2 [13:51:30] !log drop searchindex table for buswiki [13:51:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [13:53:33] !log TRUNCATE TABLE objectcache; for buswiki [13:53:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [13:54:47] !log TRUNCATE TABLE objectcache; for allthetropeswiki [13:54:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [16:34:26] [02miraheze/WikiDiscover] 07translatewiki pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/fjoOC [16:34:27] [02miraheze/WikiDiscover] 07translatewiki 03ab0c389 - Localisation updates from https://translatewiki.net. [16:34:29] [02miraheze/MirahezeMagic] 07translatewiki pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/fjoOW [16:34:30] [02miraheze/MirahezeMagic] 07translatewiki 0375d671c - Localisation updates from https://translatewiki.net. [16:37:46] miraheze/MirahezeMagic/master/75d671c - translatewiki.net The build passed. https://travis-ci.org/miraheze/MirahezeMagic/builds/551405895 [16:40:39] miraheze/WikiDiscover/master/ab0c389 - translatewiki.net The build failed. https://travis-ci.org/miraheze/WikiDiscover/builds/551405892 [17:03:08] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.33, 6.75, 4.67 [17:05:06] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 6.51, 6.76, 4.94 [17:15:05] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 6.98, 6.81, 5.80 [17:17:05] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 5.33, 6.47, 5.81 [17:17:19] !log depool mw1 [17:17:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:17:25] !log restart php7.2-fpm on mw1 [17:17:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:19:16] !log repool mw1 [17:19:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:21:48] !log depool mw1 && restart nginx && repool [17:21:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:32:28] PROBLEM - mw1 Puppet on mw1 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 9 minutes ago with 0 failures [17:40:24] !log depool mw1 [17:40:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:40:46] !log rebooting mw1 (due to some process causing high load) [17:40:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:45:28] PROBLEM - mw2 Current Load on mw2 is WARNING: WARNING - load average: 6.97, 5.76, 4.12 [17:45:49] PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 1 backends are down. mw1 [17:46:05] PROBLEM - cp2 Varnish Backends on cp2 is CRITICAL: 1 backends are down. mw1 [17:46:37] PROBLEM - cp4 Varnish Backends on cp4 is CRITICAL: 1 backends are down. mw1 [17:47:29] RECOVERY - mw2 Current Load on mw2 is OK: OK - load average: 4.79, 5.35, 4.18 [17:48:16] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 6.11, 6.99, 5.82 [17:50:05] !log repool mw1 [17:50:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:50:16] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 5.47, 6.62, 5.83 [17:50:37] RECOVERY - cp4 Varnish Backends on cp4 is OK: All 5 backends are healthy [17:51:36] RECOVERY - cp3 Varnish Backends on cp3 is OK: All 5 backends are healthy [17:51:52] RECOVERY - cp2 Varnish Backends on cp2 is OK: All 5 backends are healthy [19:08:37] PROBLEM - cp4 Varnish Backends on cp4 is CRITICAL: 1 backends are down. mw1 [19:10:38] RECOVERY - cp4 Varnish Backends on cp4 is OK: All 5 backends are healthy [20:33:06] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.58, 7.32, 6.30 [20:37:07] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 6.72, 7.18, 6.49 [20:43:05] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 6.12, 6.59, 6.45 [20:49:05] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 6.68, 6.97, 6.67 [21:01:05] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.67, 7.28, 6.88 [21:03:05] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.12, 7.26, 6.93 [21:03:29] paladox: that doesn't look nice. ^^ isn't that the mediawiki server? [21:05:05] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.20, 7.66, 7.11 [21:07:05] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 6.46, 7.57, 7.17 [21:10:12] RhinosF1 yeh [21:10:14] looking [21:10:31] thx [21:13:05] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.26, 7.66, 7.28 [21:13:18] what is causing load to go to 8... [21:14:18] paladox: If you look its varying from 6 then going up to 8 and back down [21:14:55] yeh, seems it's due to php-fpm (if i had to guess based on what im seeing). [21:15:38] !log depool mw1 [21:15:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [21:15:50] !log testing setting php-fpm max childs to 6 [21:15:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [21:15:58] !log that ^^ is for mw1 [21:16:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [21:17:05] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 1.57, 5.72, 6.70 [21:18:28] PROBLEM - mw1 Puppet on mw1 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 4 minutes ago with 0 failures [21:18:54] !log repooling mw1 temporarily to see if max childs 6 for php-fpm improves stability [21:18:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [21:24:28] RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [21:25:55] PROBLEM - mw2 Puppet on mw2 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 2 minutes ago with 0 failures [21:26:26] paladox: just a note, although it seems to be fine now based on icinga - you've already restarted mw1 once today due to issues - or not puppet seems to be cycling through servers with issues [21:26:58] Yeh, i thought that rebooting would help after restarting nginx && php-fpm before hand with no luck. [21:27:05] But appears rebooting did not help either :( [21:27:23] PROBLEM - mw3 Puppet on mw3 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 3 minutes ago with 0 failures [21:27:38] paladox: I can see. Apart from puppet though icinga isn't reporting the load errors [21:46:16] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.44, 6.50, 5.29 [21:48:16] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 4.89, 5.93, 5.23 [22:09:13] !log sudo apt-get install php7.2-dev liblua5.1-0-dev on mw1 [22:09:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [22:14:12] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/fjonB [22:14:13] [02miraheze/puppet] 07paladox 0373ff39d - php: Install luasandbox on php 7.2 [22:19:55] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:20:33] well that's a good sign [22:21:36] it's not, i re enabled puppet. [22:21:46] mw1 still has extreamly high load. [22:21:51] oh [22:23:06] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 13.54, 7.06, 4.43 [22:25:05] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 8.00, 6.77, 4.62 [22:25:14] that is bad [22:25:23] RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 12 seconds ago with 0 failures [22:27:55] PROBLEM - mw2 Puppet on mw2 is WARNING: WARNING: Puppet is currently disabled, message: paladpx, last run 5 minutes ago with 0 failures [22:29:05] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 5.83, 6.29, 4.94 [22:29:23] PROBLEM - mw3 Puppet on mw3 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 4 minutes ago with 0 failures [22:30:49] paladox: your name is spelt wrong on the mw2 warning [22:31:01] yeh, accidentially misplet it. [22:31:16] I've noticed it was working before [22:36:03] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 6.83, 6.79, 5.65 [22:39:59] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.29, 7.68, 6.26 [22:45:54] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.69, 7.72, 6.74 [22:49:51] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.78, 7.94, 7.03 [22:51:49] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.90, 7.74, 7.06 [22:53:47] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 9.43, 8.36, 7.36 [22:57:45] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.22, 7.86, 7.39 [22:59:44] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.27, 7.92, 7.46 [23:05:39] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.47, 7.98, 7.70 [23:13:33] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.91, 7.95, 7.68 [23:17:29] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.50, 7.73, 7.65 [23:17:48] !log banning an ip on cp4 firewall [23:17:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [23:20:20] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://git.io/vbiAS [23:20:22] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://git.io/fjocn [23:20:23] [02miraheze/puppet] 07paladox 030dcfe6c - Temporarily blacklist MwClient user agent [23:20:25] [02puppet] 07paladox opened pull request 03#1021: Temporarily blacklist MwClient user agent - 13https://git.io/fjocC [23:21:10] paladox: why mwclient blacklisted? [23:21:29] Someone is using too much resources with it, though only temporarily. [23:22:04] paladox: ah, well none of my bots are even connected. Is that what was messing mw1 up? [23:22:41] I think so. [23:22:56] that's only a user agent [23:23:00] which you can change though [23:23:25] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 8.04, 7.18, 7.38 [23:23:48] [02puppet] 07paladox created branch 03paladox-patch-5 - 13https://git.io/vbiAS [23:23:49] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-5 [+0/-0/±1] 13https://git.io/fjocl [23:23:51] [02miraheze/puppet] 07paladox 0399d9965 - Temporarily blacklist MwClient user agent [23:23:52] [02puppet] 07paladox opened pull request 03#1022: Temporarily blacklist MwClient user agent - 13https://git.io/fjoc8 [23:24:04] ah, btw, once you've fixed that. someone in discord was asking about https://phabricator.miraheze.org/T4498 as they think the old site might go down tommorow if you get a chance to do the import [23:24:05] [ ⚓ T4498 Requesting to import C-Wars fandom dump ] - phabricator.miraheze.org [23:25:18] [02puppet] 07paladox closed pull request 03#1022: Temporarily blacklist MwClient user agent - 13https://git.io/fjoc8 [23:25:19] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjoc4 [23:25:21] [02miraheze/puppet] 07paladox 0346af295 - Temporarily blacklist MwClient user agent (#1022) [23:27:21] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 8.00, 7.15, 7.29 [23:28:53] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [23:31:18] PROBLEM - mw1 Current Load on mw1 is CRITICAL: CRITICAL - load average: 9.34, 7.52, 7.34 [23:32:53] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 50 seconds ago with 0 failures [23:33:16] PROBLEM - mw1 Current Load on mw1 is WARNING: WARNING - load average: 7.98, 7.50, 7.34 [23:37:25] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjoc0 [23:37:27] [02miraheze/puppet] 07paladox 03701df16 - Fix user agent blacklist for MwClient/0.9.3 [23:39:11] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 2.26, 5.19, 6.47 [23:42:32] paladox: u did it ^ [23:42:39] yup! [23:42:57] though in all fairness, JohnLewis got me looking in the correct place [23:43:14] i was looking in another place until john said check nginx log. [23:43:34] well thanks all. and see my post a few messages above. [23:43:47] See you tommorow. Im off to sleep now [23:45:00] ok [23:47:24] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjoc1 [23:47:26] [02miraheze/mw-config] 07paladox 0391892de - Compress cwarswiki Change-Id: I1272528b4904edd8622fafbc8139e2e40009584f [23:49:34] !log starting import for cwarswiki - T4498 [23:49:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master