[00:04:54] RECOVERY - Host test1 is UP: PING OK - Packet loss = 0%, RTA = 0.49 ms [00:04:58] PROBLEM - test1 php-fpm on test1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [00:05:00] RECOVERY - test1 Disk Space on test1 is OK: DISK OK - free space: / 9717 MB (23% inode=97%); [00:05:07] RECOVERY - test1 Current Load on test1 is OK: OK - load average: 0.14, 0.07, 0.03 [00:05:14] RECOVERY - test1 php-fpm on test1 is OK: PROCS OK: 5 processes with command name 'php-fpm7.3' [00:05:15] RECOVERY - test1 SSH on test1 is OK: SSH OK - OpenSSH_7.4p1 Debian-10+deb9u6 (protocol 2.0) [00:14:06] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:15:19] !log installing openvpn on puppet1 [00:15:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [00:18:05] PROBLEM - bacula1 Bacula Private Git on bacula1 is CRITICAL: CRITICAL: Timeout or unknown client: puppet1-fd [00:20:12] PROBLEM - bacula1 Bacula Private Git on bacula1 is WARNING: WARNING: Full, 3745 files, 7.213MB, 2019-05-26 00:05:00 (1.6 weeks ago) [01:25:39] PROBLEM - vmcodex.net - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'vmcodex.net' expired on Wed 05 Jun 2019 04:07:43 PM GMT +0000. [01:27:39] RECOVERY - vmcodex.net - LetsEncrypt on sslhost is OK: OK - Certificate 'sni.cloudflaressl.com' will expire on Fri 27 Dec 2019 12:00:00 PM GMT +0000. [02:01:39] PROBLEM - vmcodex.net - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'vmcodex.net' expired on Wed 05 Jun 2019 04:07:43 PM GMT +0000. [02:03:39] RECOVERY - vmcodex.net - LetsEncrypt on sslhost is OK: OK - Certificate 'sni.cloudflaressl.com' will expire on Fri 27 Dec 2019 12:00:00 PM GMT +0000. [05:27:47] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 2502 [06:39:44] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [08:35:20] PROBLEM - mw1 Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[/mnt/mediawiki-static] [10:34:48] PROBLEM - db4 Disk Space on db4 is CRITICAL: DISK CRITICAL - free space: / 21961 MB (5% inode=96%); [11:01:44] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 377 [11:13:45] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [12:01:45] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 450 [12:19:44] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [12:48:20] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjzU4 [12:48:21] [02miraheze/ssl] 07paladox 03e6ae837 - Update vmcodex.net.crt [12:50:14] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjzUR [12:50:16] [02miraheze/services] 07MirahezeSSLBot 038970510 - BOT: Updating services config for wikis [12:50:27] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjzU0 [12:50:29] [02miraheze/ssl] 07paladox 03ab9d556 - Update bh.gyaanipedia.co.in.crt [12:51:07] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjzUu [12:51:09] [02miraheze/ssl] 07paladox 03998881a - Switch bh.gyaanipedia.co.in over to it's own cert [12:52:11] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fjzUg [12:52:13] [02miraheze/ssl] 07paladox 036d40368 - Delete commons.gyaanipedia.co.in.crt Cert has been merged into en.gyaanipedia.co.in [12:52:44] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fjzU2 [12:52:45] [02miraheze/ssl] 07paladox 0398a82ec - Delete fa.gyaanipedia.co.in.crt Cert has been merged into en.gyaanipedia.co.in [12:54:47] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fjzUw [12:54:48] [02miraheze/ssl] 07paladox 0349e6fea - Delete gu.gyaanipedia.co.in.crt This certificate has been merged into en.gyaanipedia.co.in [12:55:06] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fjzUr [12:55:07] [02miraheze/ssl] 07paladox 031533872 - Delete hif.gyaanipedia.co.in.crt This certificate has been merged into en.gyaanipedia.co.in [12:56:37] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fjzUo [12:56:38] [02miraheze/ssl] 07paladox 034aa33f2 - Delete kn.gyaanipedia.co.in.crt This certificate has been merged into en.gyaanipedia.co.in [12:57:17] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fjzU6 [12:57:18] [02miraheze/ssl] 07paladox 0333e1c65 - Delete la.gyaanipedia.co.in.crt This certificate has been merged into en.gyaanipedia.co.in [12:57:54] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fjzUP [12:57:55] [02miraheze/ssl] 07paladox 03ffd91f1 - Delete mai.gyaanipedia.co.in.crt This certificate has been merged into en.gyaanipedia.co.in [12:58:22] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fjzUX [12:58:23] [02miraheze/ssl] 07paladox 03481e0c0 - Delete meta.gyaanipedia.co.in.crt This certificate has been merged into en.gyaanipedia.co.in [12:58:48] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fjzUM [12:58:49] [02miraheze/ssl] 07paladox 03e0fe1a3 - Delete mg.gyaanipedia.co.in.crt This certificate has been merged into en.gyaanipedia.co.in [12:59:35] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fjzUS [12:59:37] [02miraheze/ssl] 07paladox 03a00f9c6 - Delete pa.gyaanipedia.co.in.crt This certificate has been merged into en.gyaanipedia.co.in [12:59:50] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fjzU9 [12:59:52] [02miraheze/ssl] 07paladox 039c3e3d5 - Delete or.gyaanipedia.co.in.crt This certificate has been merged into en.gyaanipedia.co.in [13:00:20] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fjzUH [13:00:22] [02miraheze/ssl] 07paladox 03510a046 - Delete mr.gyaanipedia.co.in.crt This certificate has been merged into mr.gyaanipedia.co.in [13:00:52] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fjzUQ [13:00:53] [02miraheze/ssl] 07paladox 03ce63954 - Delete sa.gyaanipedia.co.in.crt This certificate has been merged into en.gyaanipedia.co.in [13:02:34] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fjzUA [13:02:36] [02miraheze/ssl] 07paladox 037c9f3b8 - Delete ta.gyaanipedia.co.in.crt This certificate has been merged into en.gyaanipedia.co.in [13:03:38] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjzUp [13:03:40] [02miraheze/ssl] 07paladox 0311f06da - Remove www.mywiki.icu ssl certificate Domain no longer pointing at us. [13:03:52] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fjzUj [13:03:54] [02miraheze/ssl] 07paladox 03502ba08 - Delete www.mywiki.icu.crt [13:04:32] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fjzTv [13:04:33] [02miraheze/ssl] 07paladox 034267958 - Delete ml.gyaanipedia.co.in.crt This certificate has been merged into en.gyaanipedia.co.in [13:14:00] [02ssl] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vxP9L [13:14:01] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjzT3 [13:14:03] [02miraheze/ssl] 07paladox 03593dcd3 - Disable event for *.gyaanipedia.co.in apart from where it's en.gyaanipedia.co.in Also it does not disable the event if the domain uses it's own ssl certificate. [13:14:04] [02ssl] 07paladox opened pull request 03#191: Disable event for *.gyaanipedia.co.in apart from where it's en.gyaani… - 13https://git.io/fjzTG [13:14:23] [02ssl] 07paladox edited pull request 03#191: Disable event for *.gyaanipedia.co.in apart from where the domain is en.gyaanipedia.co.in - 13https://git.io/fjzTG [13:14:28] [02ssl] 07paladox edited pull request 03#191: Disable event for *.gyaanipedia.co.in apart from where the domain is en.gyaanipedia.co.in - 13https://git.io/fjzTG [13:14:37] [02ssl] 07paladox closed pull request 03#191: Disable event for *.gyaanipedia.co.in apart from where the domain is en.gyaanipedia.co.in - 13https://git.io/fjzTG [13:14:38] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjzTn [13:14:40] [02miraheze/ssl] 07paladox 03e17e8d0 - Disable event for *.gyaanipedia.co.in apart from where it's en.gyaanipedia.co.in (#191) Also it does not disable the event if the domain uses it's own ssl certificate. [13:14:41] [02ssl] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vxP9L [13:14:43] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-1 [13:18:33] !log depool mw1 [13:18:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [13:22:16] hey paladox, some users have been noticing issues with the file storage, files are disappearing and reappearing, and earlier I encountered a 500 internal error when directly viewing a file on static.miraheze.org (since resolved) [13:22:26] Voidwalker hi, yup. Investigating [13:22:30] (noticed it too) [13:22:36] it's mw1 (i've depooled it now) [13:23:23] RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 43 seconds ago with 0 failures [13:24:13] !log rebooting mw1 [13:24:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [13:26:45] PROBLEM - mw1 Current Load on mw1 is CRITICAL: connect to address 185.52.1.75 port 5666: Connection refusedconnect to host 185.52.1.75 port 5666: Connection refused [13:26:54] PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 1 backends are down. mw1 [13:28:42] RECOVERY - mw1 Current Load on mw1 is OK: OK - load average: 0.26, 0.13, 0.05 [13:28:49] RECOVERY - cp3 Varnish Backends on cp3 is OK: All 5 backends are healthy [13:29:11] !log upgrade php-redis to 4.3 on mw* [13:29:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [13:29:21] !log repool mw1 [13:29:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [13:30:55] Voidwalker|away fixed now! Do let me know if it still happens please :) [14:00:41] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjzk2 [14:00:42] [02miraheze/puppet] 07paladox 0389f8c5b - icingaweb2: Tweek opcache [14:03:08] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjzkK [14:03:10] [02miraheze/puppet] 07paladox 03904b7a3 - icingaweb2: Upgrade php to 7.3 [14:06:14] PROBLEM - misc1 icinga.miraheze.org HTTPS on misc1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 311 bytes in 0.009 second response time [14:06:22] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:06:23] known [14:07:04] PROBLEM - misc1 webmail.miraheze.org HTTPS on misc1 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 502 Bad Gateway [14:11:53] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjzky [14:11:54] [02miraheze/puppet] 07paladox 03d40d27e - Update init.pp [14:13:31] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 522 [14:21:31] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [14:25:31] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 315 [14:27:31] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [14:28:00] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 33 seconds ago with 0 failures [14:31:04] RECOVERY - misc1 webmail.miraheze.org HTTPS on misc1 is OK: HTTP OK: Status line output matched "HTTP/1.1 401 Unauthorized" - 5805 bytes in 0.171 second response time [14:32:14] RECOVERY - misc1 icinga.miraheze.org HTTPS on misc1 is OK: HTTP OK: HTTP/1.1 302 Found - 347 bytes in 0.115 second response time [14:36:02] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjzIs [14:36:03] [02miraheze/puppet] 07paladox 03082af72 - Update misc1.yaml [14:37:49] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:47:56] I'm getting invalid consumer accessing the api - keys look fine [14:49:04] https://www.irccloud.com/pastebin/jr7T8E5O/ [14:49:04] [ Snippet | IRCCloud ] - www.irccloud.com [14:53:34] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 43 seconds ago with 0 failures [14:57:25] RF1dle: what’s the full error? (the actual return) [14:57:58] JohnLewis: That's all I see in python [14:58:15] Really? Hm [14:58:21] https://www.irccloud.com/pastebin/X2rBrYEB/ [14:58:21] [ Snippet | IRCCloud ] - www.irccloud.com [14:58:30] JohnLewis: That's the code^ [15:02:30] I’m assuming consumerToken outputs an expected value? [15:03:49] JohnLewis: Tokens are correct yes - just blanked them when I pasted it in here [15:04:28] JohnLewis: Found it - space at front of key [15:06:44] Awesome that it’s fixed [15:07:21] Now I just need to find the response query value [15:07:34] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [15:18:28]

@Zppix ^ Your bot not like a ' [15:19:24] JohnLewis: You on an apple device? Are they messing up MH-Discord with strange keys again? [15:22:27] MH-Discord doesn't like encoding :P [15:23:21] Voidwalker: Neither does my bot - Apple keyboards caused a right mess [15:32:05] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjzLH [15:32:07] [02miraheze/puppet] 07paladox 03b537619 - Revert "icingaweb2: Upgrade php to 7.3" This reverts commit 904b7a350f4d6823e9b5927d80b358d2e42fe240. [15:36:39] RF1dle: yeah I switch between an iPhone and a laptop depending where/what I'm doing currently [15:37:04] PROBLEM - misc1 webmail.miraheze.org HTTPS on misc1 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 502 Bad Gateway [15:38:14] PROBLEM - misc1 icinga.miraheze.org HTTPS on misc1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 311 bytes in 0.009 second response time [15:42:20] JohnLewis: Lovely apple keyboards then. [15:42:49] yep [15:43:14] * paladox likes his apple keyboard! [15:43:34] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 3 seconds ago with 0 failures [15:44:14] RECOVERY - misc1 icinga.miraheze.org HTTPS on misc1 is OK: HTTP OK: HTTP/1.1 302 Found - 348 bytes in 0.018 second response time [15:44:20] !log apt-get install nginx mariadb-client mariadb-client-10.1 mariadb-client-core-10.1 mariadb-common - misc1 [15:44:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [15:45:04] RECOVERY - misc1 webmail.miraheze.org HTTPS on misc1 is OK: HTTP OK: Status line output matched "HTTP/1.1 401 Unauthorized" - 5805 bytes in 0.321 second response time [15:45:14] !log apt-get install dns-root-data grafana python3-cryptography rsync tzdata unzip postfix postfix-pcre postfix-sqlite publicsuffix python-cryptography libjs-jquery - misc1 [15:45:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [15:48:40] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjztt [15:48:41] [02miraheze/puppet] 07paladox 03a5c66c1 - Lower check_disk to 4% left of disk [15:48:51] PROBLEM - db4 Disk Space on db4 is WARNING: DISK WARNING - free space: / 30662 MB (8% inode=96%); [15:51:06] paladox: lowering to make checks be happy is bad practise [15:51:20] revert and keep it at 5% please, round sane numbers [15:51:24] JohnLewis no, i want to lower it so i can catch db4 (when it's near to very low) [15:51:33] 5% is low [15:51:43] 5% for db4 is around 20-19gb [15:51:54] plus 5% for db4 might not be low, but 5% for another server might be near 0 [15:52:01] oh [15:52:22] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjztO [15:52:24] [02miraheze/puppet] 07paladox 03fa7f80e - Revert "Lower check_disk to 4% left of disk" This reverts commit a5c66c1b86fb7a60633479fa05496f141866c29e. [15:55:22] [02miraheze/DataDump] 07paladox pushed 032 commits to 03master [+0/-0/±2] 13https://git.io/fjztc [15:55:23] [02miraheze/DataDump] 07paladox 0314fd7c1 - Fix undefined "filename" config in SpecialDataDump Change-Id: I5254a0c45e4a1ee3e40e4d9c52c94fb50857aacf [15:55:25] [02miraheze/DataDump] 07paladox 035114bd5 - Merge https://github.com/miraheze/DataDump [15:55:25] [ GitHub - miraheze/DataDump: An extension that provides dumps of wikis ] - github.com [15:56:18] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+0/-0/±1] 13https://git.io/fjztC [15:56:19] [02miraheze/mediawiki] 07paladox 034cd494e - Update DD [16:09:36] [02miraheze/WikiDiscover] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjztS [16:09:38] [02miraheze/WikiDiscover] 07translatewiki 0326e106b - Localisation updates from https://translatewiki.net. [16:09:39] [02miraheze/ManageWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±22] 13https://git.io/fjzt9 [16:09:41] [02miraheze/ManageWiki] 07translatewiki 03b0d39ce - Localisation updates from https://translatewiki.net. [16:34:00] [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjzql [16:34:01] [02miraheze/DataDump] 07paladox 035ef6c62 - Fix "Call to a member function getNamespace() on null" Change-Id: Ib4aa0307101a86d99c36f86df1da9622f4da5f34 [16:34:36] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+0/-0/±1] 13https://git.io/fjzqB [16:34:38] [02miraheze/mediawiki] 07paladox 03adf9c46 - Update DD [16:43:49] [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjzqo [16:43:50] [02miraheze/DataDump] 07paladox 0350aa227 - Fix i18n Change-Id: I6cf82184c3f8b464112b33e6fe36930833c4b8b6 [16:45:00] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+0/-0/±1] 13https://git.io/fjzq6 [16:45:01] [02miraheze/mediawiki] 07paladox 03caa75bb - Update DD [16:47:23] !log running lc on test1, mw* [16:47:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [16:50:12] JohnLewis ok [16:51:27] [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjzqQ [16:51:28] [02miraheze/DataDump] 07paladox 03b838a9a - datadump-description -> datadump-desc Change-Id: I944426be5a4ab8d4673053c1d0ed45dcbf953776 [16:52:14] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+0/-0/±1] 13https://git.io/fjzq5 [16:52:15] [02miraheze/mediawiki] 07paladox 033ac0b64 - Update DD [17:00:57] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+0/-0/±2] 13https://git.io/fjzmv [17:00:59] [02miraheze/mediawiki] 07paladox 03b6b9d02 - Update Cargo and PF [19:05:12] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjzON [19:05:14] [02miraheze/services] 07MirahezeSSLBot 032d212fb - BOT: Updating services config for wikis [19:31:31] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 984 [19:51:31] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [20:02:51] Paladox: You planning on a 1.32.2 upgrade - see wikitech list from Sam Reed [20:02:58] yup [20:03:09] im subscribed to that list [20:03:42] Paladox: Do we have a Est deploy date? [20:03:50] nope. [20:03:59] just that i'll do it when ever 1.32.2 is released. [20:05:56] Paladox: Showing on https://www.mediawiki.org/wiki/Download for me [20:05:56] [ Download - MediaWiki ] - www.mediawiki.org [20:06:22] it's not in the git repo.. [20:06:28] those are prepared tars. [20:07:23] Paladox: ah [20:07:31] lol "Download: https://releases.wikimedia.org/mediawiki/1.33/mediawiki-1.33.2.tar.gz" [20:07:31] [ 404 Not Found ] - releases.wikimedia.org [20:07:45] lol [20:07:54] huh [20:07:59] how did you get that Voidwalker ? [20:08:07] shows https://releases.wikimedia.org/mediawiki/1.32/mediawiki-1.32.2.tar.gz for me [20:08:36] it's in the bottom of the email to mediawiki-announce [20:08:42] oh [20:10:05] also the release notes for 1.32.2 are linked to the REL1_29 branch where they don't exist :P [20:20:26] 10/10 for execution [20:25:12] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjzsl [20:25:13] [02miraheze/services] 07MirahezeSSLBot 037a3cf24 - BOT: Updating services config for wikis [20:30:45] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+0/-0/±1] 13https://git.io/fjzsg [20:30:47] [02miraheze/mediawiki] 07paladox 03727298d - Update TemplateStyles [20:31:32] PROBLEM - mw3 JobQueue on mw3 is UNKNOWN: NRPE: Unable to read output [20:33:32] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 1500 [20:40:25] Can one of you approve my revised oauth app [20:52:32] RF1dle: done [21:02:04] @RhinosF1-QuIRC apple keyboards destroy bots [21:02:23] PROBLEM - bacula1 Bacula Phabricator Static on bacula1 is WARNING: WARNING: Diff, 4265 files, 86.67MB, 2019-05-22 21:00:00 (2.1 weeks ago) [21:02:49] @RhinosF1-QuIRC also encoding issues with @Miraheze-IRC are known and are a WONTFIX unless i get really bored [21:03:25]

@Zppix Same with mine - Vermont kept crashing it - took ages to fully clean up but it just replaces dodgy things with ? [21:04:16] @RhinosF1-QuIRC i think iirc thats its attempt at converting it [21:04:38]

@Zppix Probs [21:05:06] I dont remember the exact way apple encoding works with UTF so i cant remember why it does that [21:19:31] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [21:55:06] Hello unfairest! If you have any questions feel free to ask and someone should answer soon. [21:56:33] Is there a way to alter the files / layout of a skin from the source or to make a totally custom skin on Miraheze? I'm a web designer and want to have more customization than just selecting a skin and using the common CSS. [21:57:17] unfairest, if you have the skin set up then a sysadmin might set it up [21:58:09] https://www.mediawiki.org/wiki/Manual:Skins [21:58:10] [ Manual:Skins - MediaWiki ] - www.mediawiki.org [21:59:55] unfairest hi, you can make a mw skin [21:59:58] and we can deploy it. [22:21:38] Okay, thanks! [23:26:34] [02miraheze/mediawiki] 07paladox pushed 0340 commits to 03REL1_32 [+4/-0/±114] 13https://git.io/fjzZR [23:26:35] [02miraheze/mediawiki] 07AaronSchulz 0333634e4 - Backport WikiMap/JobQueueGroup logic to handle hyphenated DB names Although the documentation in DefaultSettings.php states that such cases should be avoided, some common cases and code paths can be made to work easily enough. Partially cherry-picked from dcd0a3d53, 51945dbca3594, and 5196ac32c6. Bug: T204423 Change-Id: Ia3c5855b18b98d9fc5bc02fe68358cfa52ccbce1 [23:26:37] [02miraheze/mediawiki] 07setian 03177a29d - Return the page_id in list=iwbacklinks as an int rather than string Bug: T216968 Change-Id: I6645c5f1c6e76be3187c24053ed430e99c03bff4 [23:26:38] [02miraheze/mediawiki] 07reedy 0320a3fea - Return the page_id in list=langbacklinks as an int Bug: T216968 Change-Id: I5b16779be7b24b1e46d4787a82a8daa3611f67b1 [23:26:40] [02miraheze/mediawiki] ... and 37 more commits. [23:27:39] PROBLEM - mw3 Puppet on mw3 is WARNING: WARNING: Puppet is currently disabled, message: mw upgrade - paladox, last run 4 minutes ago with 0 failures [23:27:58] !log depool mw1 or a mw upgrade [23:28:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [23:28:55] PROBLEM - mw2 Puppet on mw2 is WARNING: WARNING: Puppet is currently disabled, message: mw upgrade - paladox, last run 6 minutes ago with 0 failures [23:29:08] PROBLEM - mw1 Puppet on mw1 is WARNING: WARNING: Puppet is currently disabled, message: mw upgrade - paladox, last run 6 minutes ago with 0 failures [23:30:57] !log repool mw1 [23:31:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [23:31:02] !log depool mw2 or a mw upgrade [23:31:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [23:33:08] RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 59 seconds ago with 0 failures [23:34:22] !log repool mw2 [23:34:25] !log depool mw3 or a mw upgrade [23:34:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [23:34:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [23:34:47] !log also running lc on mw* [23:34:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [23:34:55] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 40 seconds ago with 0 failures [23:37:39] RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 19 seconds ago with 0 failures [23:38:49] !log repool mw3 [23:38:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [23:39:34] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [23:43:34] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures