[00:54:09] PROBLEM - db5 Puppet on db5 is CRITICAL: CRITICAL: Puppet has 6 failures. Last run 2 minutes ago with 6 failures. Failed resources (up to 3 shown) [00:54:18] PROBLEM - bacula1 Puppet on bacula1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown) [00:54:22] PROBLEM - lizardfs3 Puppet on lizardfs3 is CRITICAL: CRITICAL: Puppet has 4 failures. Last run 2 minutes ago with 4 failures. Failed resources (up to 3 shown) [00:54:28] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown) [00:54:31] PROBLEM - db4 Puppet on db4 is CRITICAL: CRITICAL: Puppet has 5 failures. Last run 2 minutes ago with 5 failures. Failed resources (up to 3 shown) [00:54:46] PROBLEM - misc3 Puppet on misc3 is CRITICAL: CRITICAL: Puppet has 7 failures. Last run 2 minutes ago with 7 failures. Failed resources (up to 3 shown) [00:54:56] PROBLEM - lizardfs2 Puppet on lizardfs2 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 2 minutes ago with 3 failures. Failed resources (up to 3 shown) [00:55:19] PROBLEM - misc4 Puppet on misc4 is CRITICAL: CRITICAL: Puppet has 14 failures. Last run 3 minutes ago with 14 failures. Failed resources (up to 3 shown) [00:55:26] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Puppet has 361 failures. Last run 2 minutes ago with 361 failures. Failed resources (up to 3 shown) [00:55:30] PROBLEM - misc2 Puppet on misc2 is CRITICAL: CRITICAL: Puppet has 9 failures. Last run 3 minutes ago with 9 failures. Failed resources (up to 3 shown) [00:55:31] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 3 minutes ago with 10 failures. Failed resources (up to 3 shown) [00:55:32] PROBLEM - cp4 Puppet on cp4 is CRITICAL: CRITICAL: Puppet has 360 failures. Last run 3 minutes ago with 360 failures. Failed resources (up to 3 shown) [00:55:32] PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Puppet has 359 failures. Last run 3 minutes ago with 359 failures. Failed resources (up to 3 shown) [00:55:40] PROBLEM - lizardfs1 Puppet on lizardfs1 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 3 minutes ago with 3 failures. Failed resources (up to 3 shown) [00:55:41] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 220 failures. Last run 2 minutes ago with 220 failures. Failed resources (up to 3 shown): File[antiguabarbudacalypso.com],File[antiguabarbudacalypso.com_private],File[spiral.wiki],File[spiral.wiki_private] [00:56:14] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 40 failures. Last run 3 minutes ago with 40 failures. Failed resources (up to 3 shown) [00:56:16] PROBLEM - mw3 Puppet on mw3 is CRITICAL: CRITICAL: Puppet has 358 failures. Last run 3 minutes ago with 358 failures. Failed resources (up to 3 shown) [00:56:17] PROBLEM - cp2 Puppet on cp2 is CRITICAL: CRITICAL: Puppet has 347 failures. Last run 3 minutes ago with 347 failures. Failed resources (up to 3 shown): File[spiral.wiki],File[spiral.wiki_private],File[permanentfuturelab.wiki],File[permanentfuturelab.wiki_private] [00:56:21] PROBLEM - mw1 Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 364 failures. Last run 3 minutes ago with 364 failures. Failed resources (up to 3 shown) [01:01:58] [02mw-config] 07Hispano76 opened pull request 03#2733: Temporary withdrawal of configuration per request of Hispano76 - 13https://git.io/fjFFM [01:03:31] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 4 seconds ago with 0 failures [01:03:40] RECOVERY - lizardfs1 Puppet on lizardfs1 is OK: OK: Puppet is currently enabled, last run 18 seconds ago with 0 failures [01:04:08] RECOVERY - db5 Puppet on db5 is OK: OK: Puppet is currently enabled, last run 42 seconds ago with 0 failures [01:04:13] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 27 seconds ago with 0 failures [01:04:18] RECOVERY - bacula1 Puppet on bacula1 is OK: OK: Puppet is currently enabled, last run 59 seconds ago with 0 failures [01:04:21] RECOVERY - lizardfs3 Puppet on lizardfs3 is OK: OK: Puppet is currently enabled, last run 59 seconds ago with 0 failures [01:04:28] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:04:30] RECOVERY - db4 Puppet on db4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:04:45] RECOVERY - misc3 Puppet on misc3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:04:56] RECOVERY - lizardfs2 Puppet on lizardfs2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:05:19] RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:05:30] RECOVERY - misc2 Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:05:32] RECOVERY - cp4 Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:05:40] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 14 seconds ago with 0 failures [01:05:41] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 59 seconds ago with 0 failures [01:05:44] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:06:00] [02mw-config] 07JohnFLewis closed pull request 03#2733: Temporary withdrawal of configuration per request of Hispano76 - 13https://git.io/fjFFM [01:06:01] [02miraheze/mw-config] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjFF5 [01:06:03] [02miraheze/mw-config] 07Hispano76 03624aebb - Temporary withdrawal of configuration per request of Hispano76 (#2733) [01:06:13] RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:06:15] RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [01:06:16] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:15:10] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjFAe [04:15:11] [02miraheze/services] 07MirahezeSSLBot 038c26300 - BOT: Updating services config for wikis [04:55:34] PROBLEM - wiki.s23.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:57:33] PROBLEM - wiki.s23.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.s23.org' expires in 11 day(s) (Sat 31 Aug 2019 03:19:24 PM GMT +0000). [05:55:09] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjFAh [05:55:10] [02miraheze/services] 07MirahezeSSLBot 032066e7f - BOT: Updating services config for wikis [06:26:35] RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 3161 MB (13% inode=94%); [06:51:29] [02miraheze/ssl] 07Reception123 deleted branch 03Reception123-patch-1 [06:51:31] [02ssl] 07Reception123 deleted branch 03Reception123-patch-1 - 13https://git.io/vxP9L [06:54:48] [02ssl] 07Reception123 created branch 03Reception123-patch-1 - 13https://git.io/vxP9L [06:54:49] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03Reception123-patch-1 [+0/-1/±1] 13https://git.io/fjFxS [06:54:51] [02miraheze/ssl] 07Reception123 03f3c8f63 - remove espiral.org cert Expiring in 3 days from now CRITICAL - Certificate 'espiral.org' expires in 3 day(s) (Fri 23 Aug 2019 11:59:59 PM GMT +0000). To be merged before that time if no new cert is provided [06:55:23] [02ssl] 07Reception123 opened pull request 03#210: remove espiral.org cert - 13https://git.io/fjFx9 [06:58:33] PROBLEM - cp3 Stunnel Http for test1 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [06:59:07] PROBLEM - cp4 SSH on cp4 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:59:12] PROBLEM - cp4 Varnish Backends on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [06:59:14] PROBLEM - meta.gyaanipedia.co.in - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:59:14] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 2 datacenters are down: 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb [06:59:14] PROBLEM - cp4 HTTPS on cp4 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:59:16] PROBLEM - misc4 phabricator.miraheze.org HTTPS on misc4 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:59:18] PROBLEM - cp4 Stunnel Http for mw2 on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [06:59:20] PROBLEM - wiki.aspirantsadda.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:59:24] PROBLEM - test1 SSH on test1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:59:30] PROBLEM - misc4 Disk Space on misc4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [06:59:39] PROBLEM - test1 Puppet on test1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [06:59:42] PROBLEM - cp4 Stunnel Http for misc2 on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [06:59:44] PROBLEM - netazar.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:59:46] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb [06:59:47] PROBLEM - cp4 Current Load on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [06:59:56] PROBLEM - cp2 Stunnel Http for test1 on cp2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [06:59:58] PROBLEM - cp4 Puppet on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [07:00:18] PROBLEM - guiasdobrasil.com.br - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:00:19] PROBLEM - test1 HTTPS on test1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [07:00:24] PROBLEM - cp4 Disk Space on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [07:00:38] PROBLEM - misc4 Puppet on misc4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [07:00:39] PROBLEM - cp4 Stunnel Http for misc3 on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [07:00:43] PROBLEM - cp4 Stunnel Http for test1 on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [07:00:44] PROBLEM - cp4 HTTP 4xx/5xx ERROR Rate on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [07:00:48] PROBLEM - cp4 Stunnel Http for mw3 on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [07:01:00] PROBLEM - cp4 Stunnel Http for mw1 on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [07:01:08] RECOVERY - meta.gyaanipedia.co.in - LetsEncrypt on sslhost is OK: OK - Certificate 'en.gyaanipedia.co.in' will expire on Thu 31 Oct 2019 03:14:12 PM GMT +0000. [07:01:14] RECOVERY - misc4 phabricator.miraheze.org HTTPS on misc4 is OK: HTTP OK: HTTP/1.1 200 OK - 19058 bytes in 1.379 second response time [07:01:19] RECOVERY - wiki.aspirantsadda.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.aspirantsadda.com' will expire on Mon 11 Nov 2019 10:12:41 AM GMT +0000. [07:01:20] RECOVERY - test1 SSH on test1 is OK: SSH OK - OpenSSH_7.4p1 Debian-10+deb9u6 (protocol 2.0) [07:01:28] RECOVERY - misc4 Disk Space on misc4 is OK: DISK OK - free space: / 46873 MB (76% inode=99%); [07:01:38] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 8 minutes ago with 0 failures [07:01:39] RECOVERY - cp4 Stunnel Http for misc2 on cp4 is OK: HTTP OK: Status line output matched "502" - 309 bytes in 0.008 second response time [07:01:41] RECOVERY - cp4 Current Load on cp4 is OK: OK - load average: 0.54, 0.22, 0.16 [07:01:41] RECOVERY - netazar.org - LetsEncrypt on sslhost is OK: OK - Certificate 'www.netazar.org' will expire on Mon 07 Oct 2019 06:46:45 PM GMT +0000. [07:01:46] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [07:01:53] RECOVERY - cp2 Stunnel Http for test1 on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 23688 bytes in 0.396 second response time [07:01:54] RECOVERY - cp4 Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 9 minutes ago with 0 failures [07:02:14] RECOVERY - guiasdobrasil.com.br - LetsEncrypt on sslhost is OK: OK - Certificate 'guiasdobrasil.com.br' will expire on Sat 16 Nov 2019 01:37:57 PM GMT +0000. [07:02:18] RECOVERY - test1 HTTPS on test1 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 444 bytes in 0.033 second response time [07:02:19] RECOVERY - cp4 Disk Space on cp4 is OK: DISK OK - free space: / 20704 MB (50% inode=99%); [07:02:34] RECOVERY - cp3 Stunnel Http for test1 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 23688 bytes in 0.690 second response time [07:02:34] RECOVERY - cp4 Stunnel Http for misc3 on cp4 is OK: HTTP OK: Status line output matched "502" - 309 bytes in 0.034 second response time [07:02:38] RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 10 minutes ago with 0 failures [07:02:41] RECOVERY - cp4 HTTP 4xx/5xx ERROR Rate on cp4 is OK: OK - NGINX Error Rate is 2% [07:02:41] RECOVERY - cp4 Stunnel Http for test1 on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 23688 bytes in 0.007 second response time [07:02:46] RECOVERY - cp4 Stunnel Http for mw3 on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 23711 bytes in 0.012 second response time [07:02:56] RECOVERY - cp4 Stunnel Http for mw1 on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 23689 bytes in 0.012 second response time [07:03:06] RECOVERY - cp4 SSH on cp4 is OK: SSH OK - OpenSSH_7.4p1 Debian-10+deb9u6 (protocol 2.0) [07:03:14] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [07:03:18] RECOVERY - cp4 HTTPS on cp4 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 1519 bytes in 0.721 second response time [07:03:19] RECOVERY - cp4 Varnish Backends on cp4 is OK: All 5 backends are healthy [07:03:20] RECOVERY - cp4 Stunnel Http for mw2 on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 23705 bytes in 0.082 second response time [09:23:21] [02mw-config] 07RhinosF1 opened pull request 03#2734: Remove config no longer needed per request - 13https://git.io/fjbJa [09:46:20] [02ssl] 07Reception123 commented on pull request 03#210: remove espiral.org cert - 13https://git.io/fjbUx [10:37:42] [02mw-config] 07Reception123 closed pull request 03#2734: Remove config no longer needed per request - 13https://git.io/fjbJa [10:37:44] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjbkZ [10:37:45] [02miraheze/mw-config] 07RhinosF1 03e579be1 - Remove config no longer needed per request (#2734) T4659 [10:42:31] Thanks Reception123, Hispano76 ^ done your task [10:43:18] no problem [10:43:34] :) [12:35:21] [02ssl] 07JohnFLewis commented on pull request 03#210: remove espiral.org cert - 13https://git.io/fjbtn [12:37:18] Reception123: ^ [12:38:29] [02ssl] 07Reception123 commented on pull request 03#210: remove espiral.org cert - 13https://git.io/fjbtR [12:38:34] JohnLewis: ^ [12:38:50] [02ssl] 07Reception123 edited a comment on pull request 03#210: remove espiral.org cert - 13https://git.io/fjbtR [12:39:26] [02ssl] 07JohnFLewis commented on pull request 03#210: remove espiral.org cert - 13https://git.io/fjbt0 [12:40:22] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.02, 6.20, 4.72 [12:42:22] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 4.12, 5.42, 4.62 [13:22:19] [02ssl] 07Reception123 commented on pull request 03#210: remove espiral.org cert - 13https://git.io/fjbYG [13:22:21] [02ssl] 07Reception123 closed pull request 03#210: remove espiral.org cert - 13https://git.io/fjFx9 [13:22:23] [02ssl] 07Reception123 deleted branch 03Reception123-patch-1 - 13https://git.io/vxP9L [13:22:24] [02miraheze/ssl] 07Reception123 deleted branch 03Reception123-patch-1 [13:35:09] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjbOB [13:35:11] [02miraheze/services] 07MirahezeSSLBot 0385b0b35 - BOT: Updating services config for wikis [13:45:01] [02mw-config] 07RhinosF1 opened pull request 03#2735: add file extensions to MW/s per T4658 - 13https://git.io/fjb3v [13:46:28] * RhinosF1 pokes Reception123 and JohnLewis ^^^ [13:47:50] JohnLewis: ^ all those safe? I don't recognize some tbh [13:47:55] anyway it's restricted but still want to be sure [13:48:56] I don't recognise some either :P [13:49:00] Google! [13:49:08] ok then I'll have to check all [13:49:40] Reception123: I did ask you to approve first :) [13:49:59] he can approve first and last [13:50:21] they should be fine, but I'll double check [13:50:31] Reception123: perfect [13:52:50] RhinosF1: they all seem fine [13:52:56] [02mw-config] 07Reception123 closed pull request 03#2735: add file extensions to MW/s per T4658 - 13https://git.io/fjb3v [13:52:58] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjb3z [13:52:59] [02miraheze/mw-config] 07RhinosF1 036d50de8 - add file extensions to MW/s per T4658 (#2735) [13:53:59] !log UPDATE cw_wikis SET wiki_dbcluster = 'c1'; (c2 wikis are old from db3, therefore pointless and confusing) [13:54:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [13:56:52] Reception123: you know what your next job is, don't u? [13:56:58] enable them all! [14:04:56] yup [14:05:23] Reception123: enjoy! [14:05:56] lol I was considering doing it in db but 1) that's bad 2) I might make a mistake and give myself more work to fix it [14:06:30] Reception123: nah, stick to ManageWiki - tedious but safe [14:06:46] yeah [14:13:52] done [14:25:45] thx [14:27:41] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[wiki.gesamtschule-nordkirchen.de] [14:35:40] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 48 seconds ago with 0 failures [16:10:10] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjbc7 [16:10:12] [02miraheze/services] 07MirahezeSSLBot 03c692e3a - BOT: Updating services config for wikis [18:30:35] PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2650 MB (10% inode=94%); [18:37:25] [02landing] 07The-Voidwalker opened pull request 03#11: fix donation button, standardize indentation - 13https://git.io/fjb8k [18:53:13] [02landing] 07paladox closed pull request 03#11: fix donation button, standardize indentation - 13https://git.io/fjb8k [18:53:15] [02miraheze/landing] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjb82 [18:53:16] [02miraheze/landing] 07The-Voidwalker 03bc9ddc4 - fix donation button, standardize indentation (#11) indentation has been set to 4 spaces [19:57:18] Voidwalker: thanks! :) as always doing the sysadmin's dev work :P [20:04:07] !log added horizonwiki to PrivateSettings Discord webhook [20:04:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [20:42:23] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.76, 6.15, 4.59 [20:44:22] PROBLEM - mw3 Current Load on mw3 is CRITICAL: CRITICAL - load average: 11.43, 8.10, 5.49 [20:44:41] Reception123: ^ [20:45:57] Hmm nothing unusual going on on that server no scripts or anything so no idea [20:46:22] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 5.07, 6.83, 5.33 [20:46:23] Reception123: https://phabricator.miraheze.org/T4655 - pls look [20:46:24] [ ⚓ T4655 Flow doesn't work after importing xml dump ] - phabricator.miraheze.org [20:46:33] And it's calming down [20:48:22] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 3.64, 5.71, 5.10 [20:51:20] Thanks R [20:55:55] Reception123, JohnLewis: https://phabricator.miraheze.org/T4445 seen as Paladox isnt here to answer [20:55:56] [ ⚓ T4445 Update PDF Rendering for nenawiki.org and ndg.nenawiki.org ] - phabricator.miraheze.org [20:56:11] RhinosF1: he is :) [20:57:37] JohnLewis: he's supposed to enjoying a holiday! [20:58:23] but if it helps, it's a paladox issue anyway. We don't know, we couldn't even guess :) [20:59:48] I tried, but couldn’t get it working [21:02:59] paladox: can you tell the user that in plain English? [21:03:23] And JohnLewis neither do I [21:06:57] Done [21:06:58] https://phabricator.miraheze.org/T4445#88658 [21:06:59] [ ⚓ T4445 Update PDF Rendering for nenawiki.org and ndg.nenawiki.org ] - phabricator.miraheze.org [21:13:55] :) [23:33:17] PROBLEM - cp2 Puppet on cp2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[wiki.grottocenter.org_private] [23:35:14] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures