[00:05:50] [02mw-config] 07paladox synchronize pull request 03#2658: Initial configuration for DataDump - 13https://git.io/fjkuM [00:05:51] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjkuj [00:05:53] [02miraheze/mw-config] 07paladox 03a5de339 - Update LocalExtensions.php [00:06:04] [02mw-config] 07paladox synchronize pull request 03#2658: Initial configuration for DataDump - 13https://git.io/fjkuM [00:06:06] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjkze [00:06:07] [02miraheze/mw-config] 07paladox 0311da281 - Update LocalExtensions.php [00:07:13] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjkzf [00:07:14] [02miraheze/mw-config] 07paladox 038da06e1 - Update LocalSettings.php [00:07:16] [02mw-config] 07paladox synchronize pull request 03#2658: Initial configuration for DataDump - 13https://git.io/fjkuM [00:07:26] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjkzJ [00:07:27] [02miraheze/mw-config] 07paladox 0357fa0cf - Update GlobalExtensions.php [00:07:29] [02mw-config] 07paladox synchronize pull request 03#2658: Initial configuration for DataDump - 13https://git.io/fjkuM [00:11:03] [02mw-config] 07paladox synchronize pull request 03#2658: Initial configuration for DataDump - 13https://git.io/fjkuM [00:11:05] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjkzk [00:11:06] [02miraheze/mw-config] 07paladox 038a4f6a4 - Update LocalExtensions.php [00:12:47] [02mw-config] 07paladox synchronize pull request 03#2658: Initial configuration for DataDump - 13https://git.io/fjkuM [00:12:48] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjkzI [00:12:50] [02miraheze/mw-config] 07paladox 03bdffdf0 - Update LocalExtensions.php [00:32:33] [02mw-config] 07paladox synchronize pull request 03#2658: Initial configuration for DataDump - 13https://git.io/fjkuM [00:32:35] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjkzW [00:32:36] [02miraheze/mw-config] 07paladox 0301116f2 - Update LocalExtensions.php [00:32:49] [02mw-config] 07paladox synchronize pull request 03#2658: Initial configuration for DataDump - 13https://git.io/fjkuM [00:32:50] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjkzl [00:32:52] [02miraheze/mw-config] 07paladox 038d4a2f8 - Update LocalExtensions.php [00:33:05] PROBLEM - db4 Disk Space on db4 is CRITICAL: DISK CRITICAL - free space: / 40297 MB (10% inode=96%); [00:34:51] [02mw-config] 07paladox synchronize pull request 03#2658: Initial configuration for DataDump - 13https://git.io/fjkuM [00:34:53] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjkz4 [00:34:54] [02miraheze/mw-config] 07paladox 03c27c5e5 - Update LocalExtensions.php [00:49:25] [02mw-config] 07paladox synchronize pull request 03#2658: Initial configuration for DataDump - 13https://git.io/fjkuM [00:49:26] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjkz2 [00:49:28] [02miraheze/mw-config] 07paladox 038a9494b - Update LocalExtensions.php [01:11:20] [02mw-config] 07paladox synchronize pull request 03#2658: Initial configuration for DataDump - 13https://git.io/fjkuM [01:11:22] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjkzD [01:11:23] [02miraheze/mw-config] 07paladox 03a111266 - Update LocalExtensions.php [01:12:46] [02mw-config] 07paladox synchronize pull request 03#2658: Initial configuration for DataDump - 13https://git.io/fjkuM [01:12:48] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjkzy [01:12:49] [02miraheze/mw-config] 07paladox 03a7657f8 - Update LocalExtensions.php [01:13:27] [02mw-config] 07paladox synchronize pull request 03#2658: Initial configuration for DataDump - 13https://git.io/fjkuM [01:13:28] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjkzS [01:13:30] [02miraheze/mw-config] 07paladox 03034215b - Update LocalExtensions.php [01:41:09] ? [07:49:03] Hello Zamujasa! If you have any questions feel free to ask and someone should answer soon. [07:49:37] howdy. i'm trying to upload an image to a private wiki @ sol.miraheze.org but getting errors. [07:49:40] Could not store file "/tmp/phpMdWrrK" at "mwstore://local-backend/local-public/6/63/Sign_arrow-dir1.png". [07:49:59] this occurred after i enabled the "msupload" and "inputbar" extensions but i do not know if that is related. [07:50:17] i've tried uploading different files and different filenames and they all give the same sort of error. [08:20:21] Zamujasa: hi [08:20:34] The issue could be due to space issues [08:20:35] ^ paladox [08:23:11] is there anything i can do about it, or is it a waiting game? [08:34:46] Zamujasa: Try again, it could work [08:34:53] when paladox is around he can confirm [08:41:07] Nope, same problem. [08:50:07] Zamujasa: Sorry for the inconvenience :( [08:50:08] we' [08:50:12] *we'll get back to you [08:51:39] Hello CurePopuri! If you have any questions feel free to ask and someone should answer soon. [13:35:59] Reception123 yup, ran out of storage [13:36:05] we only have 120 MiB left [14:39:07] :( [14:53:28] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjkKU [14:53:29] [02miraheze/puppet] 07paladox 03061d4ae - Lower HDD_LEAVE_SPACE_DEFAULT to 5GB [14:56:04] !log upgrade puppet agent and salt on lizardfs* and misc4 [14:56:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [14:56:18] uh, that's ment to be misc3 [14:57:59] PROBLEM - cp4 HTTP 4xx/5xx ERROR Rate on cp4 is WARNING: WARNING - NGINX Error Rate is 53% [15:01:54] PROBLEM - cp4 HTTP 4xx/5xx ERROR Rate on cp4 is CRITICAL: CRITICAL - NGINX Error Rate is 64% [15:03:52] RECOVERY - cp4 HTTP 4xx/5xx ERROR Rate on cp4 is OK: OK - NGINX Error Rate is 9% [15:14:51] [02mw-config] 07paladox synchronize pull request 03#2658: Initial configuration for DataDump - 13https://git.io/fjkuM [15:14:52] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjkKu [15:14:54] [02miraheze/mw-config] 07paladox 035bd3449 - Update LocalExtensions.php [15:15:21] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjkKz [15:15:22] [02miraheze/mw-config] 07paladox 03eff7839 - Update extension-list [15:15:24] [02mw-config] 07paladox synchronize pull request 03#2658: Initial configuration for DataDump - 13https://git.io/fjkuM [15:35:18] PROBLEM - db4 Current Load on db4 is WARNING: WARNING - load average: 7.48, 6.64, 5.51 [15:39:18] RECOVERY - db4 Current Load on db4 is OK: OK - load average: 6.74, 6.78, 5.83 [16:06:32] [02miraheze/DataDump] 07paladox pushed 032 commits to 03master [+7/-0/±1] 13https://git.io/fjk6c [16:06:34] [02miraheze/DataDump] 07paladox 03e1d0907 - Initial Import * This create's a extension for MediaWiki where users can request there own dumps (ie backups). Change-Id: Idc15143bb9a6904db9be28f5ca7e4ee9495d87c0 [16:06:35] [02miraheze/DataDump] 07paladox 03a1ad5c0 - Merge branch 'master' of https://github.com/miraheze/DataDump [16:06:36] Title: [ GitHub - miraheze/DataDump: An extension that provides dumps of wikis ] - github.com [16:10:06] PROBLEM - db4 Current Load on db4 is WARNING: WARNING - load average: 7.09, 6.86, 6.21 [16:12:00] PROBLEM - db4 Current Load on db4 is CRITICAL: CRITICAL - load average: 8.47, 7.19, 6.40 [16:13:54] RECOVERY - db4 Current Load on db4 is OK: OK - load average: 6.39, 6.65, 6.28 [16:17:42] PROBLEM - db4 Current Load on db4 is WARNING: WARNING - load average: 7.15, 7.06, 6.53 [16:19:37] RECOVERY - db4 Current Load on db4 is OK: OK - load average: 6.36, 6.78, 6.49 [16:34:07] PROBLEM - db4 Current Load on db4 is CRITICAL: CRITICAL - load average: 8.43, 7.27, 6.80 [16:36:01] PROBLEM - db4 Current Load on db4 is WARNING: WARNING - load average: 7.64, 7.25, 6.83 [16:37:55] PROBLEM - db4 Current Load on db4 is CRITICAL: CRITICAL - load average: 8.72, 7.75, 7.06 [16:39:49] PROBLEM - db4 Current Load on db4 is WARNING: WARNING - load average: 6.81, 7.20, 6.93 [16:45:32] RECOVERY - db4 Current Load on db4 is OK: OK - load average: 4.58, 5.97, 6.53 [18:09:18] PROBLEM - db4 Current Load on db4 is WARNING: WARNING - load average: 7.84, 6.30, 4.64 [18:13:18] PROBLEM - db4 Current Load on db4 is CRITICAL: CRITICAL - load average: 8.05, 7.07, 5.31 [18:15:18] RECOVERY - db4 Current Load on db4 is OK: OK - load average: 4.71, 6.44, 5.31 [18:49:25] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+1/-0/±1] 13https://git.io/fjkPU [18:49:26] [02miraheze/mediawiki] 07paladox 0388d9993 - Add DataDump (New Extension) [18:52:16] [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjkPY [18:52:18] [02miraheze/DataDump] 07paladox 03e30a33a - Add missing i18n (datadump) Change-Id: I0feb26cbabaccf8adfe1d7ed2fde753207591091 [18:52:41] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+0/-0/±1] 13https://git.io/fjkPO [18:52:43] [02miraheze/mediawiki] 07paladox 033bc641a - Update DataDump [18:53:21] PROBLEM - misc4 Current Load on misc4 is WARNING: WARNING - load average: 3.94, 3.63, 2.29 [18:55:18] PROBLEM - misc4 Current Load on misc4 is CRITICAL: CRITICAL - load average: 5.58, 4.21, 2.65 [18:58:24] PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [18:59:07] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjkPC [18:59:09] [02miraheze/puppet] 07paladox 034047478 - Update php_fpm.pp [19:01:17] !log upgraded phabricator on misc4 [19:01:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [19:04:23] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [19:06:15] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjkP0 [19:06:17] [02miraheze/puppet] 07paladox 03f70307a - php: Install liblua5.1-0, which is required by luasandbox [19:08:02] !log reboot misc4 due to high load [19:08:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [19:13:34] PROBLEM - misc4 Puppet on misc4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:13:39] PROBLEM - misc4 phab.miraheze.wiki HTTPS on misc4 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:13:40] PROBLEM - misc4 phabricator.miraheze.org HTTPS on misc4 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:13:45] PROBLEM - misc4 SSH on misc4 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:13:47] PROBLEM - misc4 phd on misc4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:14:05] PROBLEM - misc4 Disk Space on misc4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:15:34] RECOVERY - misc4 phab.miraheze.wiki HTTPS on misc4 is OK: HTTP OK: Status line output matched "HTTP/1.1 200" - 17648 bytes in 0.069 second response time [19:15:38] RECOVERY - misc4 phabricator.miraheze.org HTTPS on misc4 is OK: HTTP OK: HTTP/1.1 200 OK - 18598 bytes in 0.158 second response time [19:15:42] RECOVERY - misc4 phd on misc4 is OK: PROCS OK: 1 process with args 'phd' [19:15:43] RECOVERY - misc4 SSH on misc4 is OK: SSH OK - OpenSSH_7.4p1 Debian-10+deb9u6 (protocol 2.0) [19:15:58] RECOVERY - misc4 Disk Space on misc4 is OK: DISK OK - free space: / 49533 MB (80% inode=99%); [19:17:06] RECOVERY - misc4 Current Load on misc4 is OK: OK - load average: 3.33, 2.35, 0.97 [19:21:17] !log apply DataDump sql to all wiki [19:21:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [19:21:53] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjkPM [19:21:55] [02miraheze/mw-config] 07paladox 03626dc51 - Update LocalSettings.php [19:21:56] [02mw-config] 07paladox synchronize pull request 03#2658: Initial configuration for DataDump - 13https://git.io/fjkuM [19:22:26] [02mw-config] 07paladox closed pull request 03#2658: Initial configuration for DataDump - 13https://git.io/fjkuM [19:22:28] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/fjkPD [19:22:29] [02miraheze/mw-config] 07paladox 0310b92de - Initial configuration for DataDump (#2658) [19:22:31] [02mw-config] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbvb3 [19:22:33] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-1 [19:23:10] RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 44 seconds ago with 0 failures [19:23:16] miraheze/mw-config/paladox-patch-1/626dc51 - paladox The build was broken. https://travis-ci.org/miraheze/mw-config/builds/513834793 [19:25:29] !log running lc on mw* [19:25:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [19:26:11] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki config] [19:29:55] !log granting view-dump, generate-dump and delete-dump rights to sysops on metawiki [19:29:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [19:30:39] oh meh, im not in that group [19:30:59] !log reverting prevous action [19:31:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [19:31:25] that and, you probably don't need to log it if doing the action through managewiki :P [19:32:32] heh ok [20:01:58] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjkXt [20:01:59] [02miraheze/mw-config] 07paladox 03d2bdf07 - Increase wgMaxShellWallClockTime to 21600 [20:25:10] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjkXR [20:25:11] [02miraheze/services] 07MirahezeSSLBot 0321fb940 - BOT: Updating services config for wikis [20:27:22] !log authorising sonarcloud to read ManageWiki in github. (for finding vulns/bugs) [20:27:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [20:36:59] !log granting the rest of our mw extensions access to sonarcloud [20:37:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [21:04:11] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [21:15:58] [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/fjkXA [21:16:00] [02miraheze/DataDump] 07paladox 03f4cb2e7 - Make Shell commands limits configurable Due to the nature of the extension, the default limits applied to the shell methoid's cannot be used. So we make the filesize, memory and timeout unlimited by default, allowing these to be configured through a config. Change-Id: If862c427c9849ce9a9703d6e02452c1f2391f25e [21:19:41] [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjkXh [21:19:42] [02miraheze/DataDump] 07paladox 03574ec31 - Use random_bytes( 10 ) instead of rand() Change-Id: I67caba1d107fd042b9b7eaf803864ade667e2b3e [21:19:59] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+0/-0/±1] 13https://git.io/fjkXj [21:20:00] [02miraheze/mediawiki] 07paladox 034c0715c - Update DD [21:20:20] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+0/-0/±1] 13https://git.io/fjk1e [21:20:22] [02miraheze/mediawiki] 07paladox 03b05318e - Update DD [21:25:41] [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjk1I [21:25:42] [02miraheze/DataDump] 07paladox 0357d9fac - Fix using random_bytes Change-Id: I33c7f8c495cab04ef6ce8e9cf5501c19f5d3bf6a [21:29:48] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+0/-0/±1] 13https://git.io/fjk1q [21:29:49] [02miraheze/mediawiki] 07paladox 039a0b614 - Update DataDump [21:42:32] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjk18 [21:42:34] [02miraheze/mw-config] 07paladox 03469c47e - Revert "Increase wgMaxShellWallClockTime to 21600" This reverts commit d2bdf07492df35fcc93c78429de84a3720ab4ad5. [22:02:34] [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjk1r [22:02:36] [02miraheze/DataDump] 07paladox 0382baf42 - Fix DataDumpGenerateJob Change-Id: I3f5e522a33cd78eb1ef23ea4467c95a7a2a1e294 [22:03:42] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+0/-0/±1] 13https://git.io/fjk1o [22:03:43] [02miraheze/mediawiki] 07paladox 030078836 - Update DataDump [22:08:58] [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjk1K [22:08:59] [02miraheze/DataDump] 07paladox 030f52a71 - Do not show the delete button if there is no files to delete Change-Id: Ie0b908bc3ed9557183a40c88c553699e107810a6 [22:09:15] [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjk16 [22:09:16] [02miraheze/DataDump] 07paladox 0308643a4 - Fix showing "no-results" if no dumps have been found Change-Id: I34150d56cdf0433e77d73be76fc8d9caad2b8b69 [22:09:48] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+0/-0/±1] 13https://git.io/fjk1i [22:09:50] [02miraheze/mediawiki] 07paladox 03aa10b33 - Update DD [22:12:41] [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjk1X [22:12:42] [02miraheze/DataDump] 07paladox 03e106866 - Fix no-results showing part 2 Change-Id: I73dce932aa391b0033e3f0563680423c53b76f51 [22:13:06] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+0/-0/±1] 13https://git.io/fjk11 [22:13:08] [02miraheze/mediawiki] 07paladox 0392814db - Update DD [22:20:13] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjk1H [22:20:15] [02miraheze/services] 07MirahezeSSLBot 03778f315 - BOT: Updating services config for wikis [22:20:38] [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjk1Q [22:20:40] [02miraheze/DataDump] 07paladox 03f514bbe - Fix no-results showing part 3 Change-Id: Id491fdbf7373caf2041c6ef649703789c6d035e3 [22:28:54] !log depool mw3 [22:28:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [22:29:38] !log reboot mw3 to reduce ram consumption (almost OOM) [22:29:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [22:31:47] PROBLEM - mw3 Current Load on mw3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:31:55] PROBLEM - mw3 HTTPS on mw3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:32:04] PROBLEM - mw3 JobRunner Service on mw3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:32:05] PROBLEM - mw3 JobChron Service on mw3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:32:35] PROBLEM - cp2 Varnish Backends on cp2 is CRITICAL: 1 backends are down. mw3 [22:32:41] PROBLEM - mw3 SSH on mw3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:32:46] PROBLEM - cp5 Varnish Backends on cp5 is CRITICAL: 1 backends are down. mw3 [22:32:50] PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 1 backends are down. mw3 [22:33:10] PROBLEM - mw3 Puppet on mw3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:33:11] PROBLEM - cp4 Varnish Backends on cp4 is CRITICAL: 1 backends are down. mw3 [22:33:14] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:33:18] PROBLEM - mw3 php-fpm on mw3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:33:46] PROBLEM - mw3 Disk Space on mw3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:34:34] RECOVERY - mw3 SSH on mw3 is OK: SSH OK - OpenSSH_7.4p1 Debian-10+deb9u6 (protocol 2.0) [22:35:09] RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 12 minutes ago with 0 failures [22:35:13] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [22:35:14] RECOVERY - mw3 php-fpm on mw3 is OK: PROCS OK: 2 processes with command name 'php-fpm7.3' [22:35:41] RECOVERY - mw3 Disk Space on mw3 is OK: DISK OK - free space: / 52385 MB (68% inode=99%); [22:35:44] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 0.90, 0.28, 0.09 [22:35:48] RECOVERY - mw3 HTTPS on mw3 is OK: HTTP OK: HTTP/1.1 200 OK - 22921 bytes in 0.012 second response time [22:35:55] RECOVERY - mw3 JobRunner Service on mw3 is OK: PROCS OK: 1 process with args 'redisJobRunnerService' [22:35:56] RECOVERY - mw3 JobChron Service on mw3 is OK: PROCS OK: 1 process with args 'redisJobChronService' [22:36:28] !log repool mw3 [22:36:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [22:36:33] RECOVERY - cp2 Varnish Backends on cp2 is OK: All 5 backends are healthy [22:36:43] RECOVERY - cp5 Varnish Backends on cp5 is OK: All 5 backends are healthy [22:36:50] RECOVERY - cp3 Varnish Backends on cp3 is OK: All 5 backends are healthy [22:37:11] RECOVERY - cp4 Varnish Backends on cp4 is OK: All 5 backends are healthy [22:37:59] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjkMJ [22:38:01] [02miraheze/puppet] 07paladox 037447765 - Update jobrunner.json.erb [22:39:02] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+0/-0/±1] 13https://git.io/fjkMT [22:39:04] [02miraheze/mediawiki] 07paladox 03708ef6d - Update DD