[00:25:11] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5ep [00:25:12] [02miraheze/services] 07MirahezeSSLBot 030ab2e93 - BOT: Updating services config for wikis [02:09:05] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.96, 6.69, 5.21 [02:13:02] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 6.02, 6.60, 5.55 [02:52:40] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_33 [+0/-0/±1] 13https://git.io/fj5fv [02:52:41] [02miraheze/mediawiki] 07paladox 0386f4aea - Update MF [04:50:13] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5J3 [04:50:14] [02miraheze/services] 07MirahezeSSLBot 03aee7525 - BOT: Updating services config for wikis [04:51:30] Hello crustycris! If you have any questions feel free to ask and someone should answer soon. [04:52:35] Hello, I'm trying to upload images but keep getting this error: [04:52:43] Could not store file "/tmp/phpMKMNMX" at "mwstore://local-backend/local-public/8/81/Main_Menu-sharedassets0.assets-16.png". [04:53:35] I see that user Zamujasa had this same issue 03/31/2019 and it was due to storage issues - is this the case again? [04:54:10] Any help would be appreciated! I don't have a crazy amount of images to upload but they are necessary [04:59:41] Crustycris: You can report the issue in Phabricator https://phabricator.miraheze.org/maniphest/task/edit/form/1/ [04:59:42] [ Login ] - phabricator.miraheze.org [04:59:57] @CrustyCris [05:00:52] crustycris: Hi, yes unfortunately it is storage issues again with the fileserver. We are waiting for our provider to provide us with a new package. I will try to fix the issue now. [05:04:10] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5JG [05:04:11] [02miraheze/puppet] 07Reception123 0362041fd - HDD_LEAVE_SPACE_DEFAULT to 8 GB (lack of disk space) should probably be reverted later, but this is an emergency action due to a lack of disk space [05:07:27] That worked, thanks so much for the quick reply [05:10:43] crustycris: no problem :) [05:10:51] !log sudo service lizardfs-chunkserver restart [05:10:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [05:16:14] !log reception@mw2:/mnt/mediawiki-static$ sudo php /srv/mediawiki/w/maintenance/dumpBackup.php --wiki nltramswiki --full --output gzip:/home/reception/nltramswiki12082019.xml --report 1 (part of wikibackups) [05:16:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [05:22:57] !log reception@mw2:/$ sudo php /srv/mediawiki/w/maintenance/dumpBackup.php --wiki zhdelwiki --full --output gzip:/home/reception/zhdelwiki12082019.xml --report 1 (part of wikibackups) [05:23:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [06:26:42] RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 2910 MB (12% inode=94%); [07:05:44] @Stewards anyome for a CU [08:02:58] Heh I just thought of something. We kept the Tweeki skin on wikis, but since it causes only MobileFrontend to work on wikis, then it's pointless for us to allow users to enable it, since it won't work anyway :P [08:03:29] Reception123: good point [08:06:05] [02mw-config] 07RhinosF1 opened pull request 03#2723: remove tweeki - 13https://git.io/fj5TH [08:06:23] ^ give me a sec to make sure i've taken it off everywhere [08:06:30] RhinosF1: there's no need [08:06:41] RhinosF1: please leave it in other places, we just don't want it in ManageWiki [08:06:46] so then when it is fixed we can add it back [08:07:12] Reception123: okay, erm, that should work then [08:07:13] RhinosF1: and rather than removing, you can comment it out by using // and writing a comment above [08:07:31] RhinosF1: like // Removed due to it making MobileFrontend enabled on all wikis -RhinosF1 [08:07:44] well a more clear message, what I jut wrote is very unclear :P [08:07:45] Reception123: okay, redoing [08:07:50] [02mw-config] 07RhinosF1 closed pull request 03#2723: remove tweeki - 13https://git.io/fj5TH [08:09:59] [02mw-config] 07RhinosF1 opened pull request 03#2724: // out tweeki - 13https://git.io/fj5Td [08:10:09] Reception123: ^ [08:11:03] [02mw-config] 07Reception123 commented on pull request 03#2724: // out tweeki - 13https://git.io/fj5TF [08:11:07] [02mw-config] 07Reception123 closed pull request 03#2724: // out tweeki - 13https://git.io/fj5Td [08:11:09] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5Tb [08:11:10] [02miraheze/mw-config] 07RhinosF1 03ee290ae - // out tweeki (#2724) as doesn't work as will always be in mobile view [08:12:13] Reception123: :) - did someone speed up meta as well, earlier was the fastest a wiki has ever created [08:12:27] RhinosF1: Oh no, I was very confused I just realized I confused two issues... [08:12:33] [02mw-config] 07Reception123 created branch 03revert-2724-patch-14 - 13https://git.io/vbvb3 [08:12:34] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03revert-2724-patch-14 [+0/-0/±1] 13https://git.io/fj5Tx [08:12:36] [02miraheze/mw-config] 07Reception123 030e16f5c - Revert "// out tweeki (#2724)" This reverts commit ee290aede1f083224f0e633b0e7d27b97ac756ee. [08:12:38] [02mw-config] 07Reception123 opened pull request 03#2725: Revert "// out tweeki" - 13https://git.io/fj5Tp [08:12:43] [02mw-config] 07Reception123 closed pull request 03#2725: Revert "// out tweeki" - 13https://git.io/fj5Tp [08:12:45] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5Th [08:12:46] [02miraheze/mw-config] 07Reception123 0319d5cbd - Revert "// out tweeki (#2724)" (#2725) This reverts commit ee290aede1f083224f0e633b0e7d27b97ac756ee. [08:13:06] Reception123: eh [08:14:20] RhinosF1: Tweeki just makes some sections not work on MobileFrontend, but another issue makes only MobileFrontend appear even on PC [08:14:24] I confused the two [08:14:31] [02miraheze/mw-config] 07Reception123 deleted branch 03revert-2724-patch-14 [08:14:33] [02mw-config] 07Reception123 deleted branch 03revert-2724-patch-14 - 13https://git.io/vbvb3 [08:14:44] Reception123: OH [08:14:52] * RhinosF1 wonders why caps was on [08:16:57] but even so, we fixed the other issue.. [08:17:06] that's what confused me [08:17:20] or maybe we never went trough with it? [08:17:45] Reception123: no because the CP issue wiki was stuck in mobile [08:32:49] 05:17, 10 August 2019 Anti Sinistram AZ talk contribs changed the settings (setting-wgDefaultRobotPolicy, setting-wgAccountCreationThrottle, setting-wgSVGMetadataCutoff, setting-wgMaxArticleSize, setting-wgMaxImageArea, setting-wgDefaultSkin) for "moderaterightpediawiki" (Cuz) [08:32:50] yup [08:32:58] ^ RhinosF1 [08:33:04] So it's not stuck in mobile [08:33:14] it's set default [08:33:22] wonder why anyone would want default mobile but ok :P [08:33:26] Reception123: strange but ok [08:44:52] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 16 seconds ago with 0 failures [08:54:53] [02miraheze/ssl] 07Reception123 pushed 032 commits to 03master [+0/-2/±2] 13https://git.io/fj5kX [08:54:55] [02miraheze/ssl] 07Reception123 03e3708e0 - remove pc-architect.tk no longer pointing to us [08:54:56] [02miraheze/ssl] 07Reception123 03be18261 - remove ahmsaqib.cf no longer pointing to us [08:56:30] [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fj5kD [08:56:31] [02miraheze/dns] 07Reception123 03094cae6 - remove ahmsaqib.cf zone no longer pointing to us [10:30:15] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5LJ [10:30:16] [02miraheze/services] 07MirahezeSSLBot 03f7a128f - BOT: Updating services config for wikis [10:58:13] hiya JohnLewis [11:20:56] JohnLewis: can u do a CU for me [11:24:55] PROBLEM - aman.info.tm - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'aman.info.tm' expires in 15 day(s) (Wed 28 Aug 2019 11:21:42 AM GMT +0000). [11:25:15] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5tl [11:25:16] [02miraheze/ssl] 07MirahezeSSLBot 0336b540f - Bot: Update SSL cert for aman.info.tm [11:32:55] RECOVERY - aman.info.tm - LetsEncrypt on sslhost is OK: OK - Certificate 'aman.info.tm' will expire on Sun 10 Nov 2019 10:25:07 AM GMT +0000. [11:41:57] PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2648 MB (10% inode=94%); [11:46:31] PROBLEM - wiki.lbcomms.co.za - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.lbcomms.co.za' expires in 15 day(s) (Wed 28 Aug 2019 11:43:43 AM GMT +0000). [11:46:44] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5t5 [11:46:46] [02miraheze/ssl] 07MirahezeSSLBot 0370edc78 - Bot: Update SSL cert for wiki.lbcomms.co.za [11:48:48] PROBLEM - wiki.hackdown.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.hackdown.org' expires in 15 day(s) (Wed 28 Aug 2019 11:45:15 AM GMT +0000). [11:49:01] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5tF [11:49:03] [02miraheze/ssl] 07MirahezeSSLBot 03dad83dd - Bot: Update SSL cert for wiki.hackdown.org [11:49:06] PROBLEM - wiki.meeusen.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.meeusen.net' expires in 15 day(s) (Wed 28 Aug 2019 11:46:49 AM GMT +0000). [11:49:19] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5tb [11:49:21] [02miraheze/ssl] 07MirahezeSSLBot 033619d7d - Bot: Update SSL cert for wiki.meeusen.net [11:50:58] RhinosF1: send me the details can do [11:52:27] PROBLEM - wiki.drones4nature.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.drones4nature.info' expires in 15 day(s) (Wed 28 Aug 2019 11:49:11 AM GMT +0000). [11:52:43] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5th [11:52:44] [02miraheze/ssl] 07MirahezeSSLBot 03683f14f - Bot: Update SSL cert for wiki.drones4nature.info [11:52:48] RECOVERY - wiki.hackdown.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.hackdown.org' will expire on Sun 10 Nov 2019 10:48:55 AM GMT +0000. [11:53:06] RECOVERY - wiki.meeusen.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.meeusen.net' will expire on Sun 10 Nov 2019 10:49:13 AM GMT +0000. [11:54:28] RECOVERY - wiki.lbcomms.co.za - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.lbcomms.co.za' will expire on Sun 10 Nov 2019 10:46:37 AM GMT +0000. [11:54:46] PROBLEM - wikipuk.cl - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wikipuk.cl' expires in 15 day(s) (Wed 28 Aug 2019 11:50:57 AM GMT +0000). [11:54:59] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5qJ [11:55:00] [02miraheze/ssl] 07MirahezeSSLBot 035a1f7ad - Bot: Update SSL cert for wikipuk.cl [11:55:22] PROBLEM - wiki.jacksonheights.nyc - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.jacksonheights.nyc' expires in 15 day(s) (Wed 28 Aug 2019 11:52:08 AM GMT +0000). [11:55:35] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5qU [11:55:37] [02miraheze/ssl] 07MirahezeSSLBot 03ec56316 - Bot: Update SSL cert for wiki.jacksonheights.nyc [11:57:08] PROBLEM - wiki.besuccess.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.besuccess.com' expires in 15 day(s) (Wed 28 Aug 2019 11:53:16 AM GMT +0000). [11:57:14] PROBLEM - speleo.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'speleo.wiki' expires in 15 day(s) (Wed 28 Aug 2019 11:54:35 AM GMT +0000). [11:57:21] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5qT [11:57:23] [02miraheze/ssl] 07MirahezeSSLBot 03a5b4517 - Bot: Update SSL cert for wiki.besuccess.com [11:57:33] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5qk [11:57:34] [02miraheze/ssl] 07MirahezeSSLBot 03c5d7423 - Bot: Update SSL cert for speleo.wiki [11:58:09] PROBLEM - disabled.life - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'disabled.life' expires in 15 day(s) (Wed 28 Aug 2019 11:55:46 AM GMT +0000). [11:58:22] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5qI [11:58:24] [02miraheze/ssl] 07MirahezeSSLBot 03c63efd9 - Bot: Update SSL cert for disabled.life [11:59:04] PROBLEM - wiki.teessidehackspace.org.uk - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.teessidehackspace.org.uk' expires in 15 day(s) (Wed 28 Aug 2019 11:56:45 AM GMT +0000). [11:59:18] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5qt [11:59:20] [02miraheze/ssl] 07MirahezeSSLBot 03ffdb904 - Bot: Update SSL cert for wiki.teessidehackspace.org.uk [12:01:00] PROBLEM - wiki.thunis.eu - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.thunis.eu' expires in 15 day(s) (Wed 28 Aug 2019 11:57:56 AM GMT +0000). [12:01:15] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5qY [12:01:17] [02miraheze/ssl] 07MirahezeSSLBot 036a24e88 - Bot: Update SSL cert for wiki.thunis.eu [12:01:59] PROBLEM - wiki.tulpa.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.tulpa.info' expires in 15 day(s) (Wed 28 Aug 2019 11:59:01 AM GMT +0000). [12:02:14] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5q3 [12:02:15] [02miraheze/ssl] 07MirahezeSSLBot 03e1fc6b2 - Bot: Update SSL cert for wiki.tulpa.info [12:03:00] RECOVERY - wiki.thunis.eu - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.thunis.eu' will expire on Sun 10 Nov 2019 11:01:08 AM GMT +0000. [12:03:04] RECOVERY - wiki.teessidehackspace.org.uk - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.teessidehackspace.org.uk' will expire on Sun 10 Nov 2019 10:59:12 AM GMT +0000. [12:03:08] RECOVERY - wiki.besuccess.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.besuccess.com' will expire on Sun 10 Nov 2019 10:57:15 AM GMT +0000. [12:03:14] RECOVERY - speleo.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'speleo.wiki' will expire on Sun 10 Nov 2019 10:57:27 AM GMT +0000. [12:03:22] RECOVERY - wiki.jacksonheights.nyc - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.jacksonheights.nyc' will expire on Sun 10 Nov 2019 10:55:29 AM GMT +0000. [12:03:55] PROBLEM - lodge.jsnydr.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'lodge.jsnydr.com' expires in 15 day(s) (Wed 28 Aug 2019 12:00:06 PM GMT +0000). [12:04:10] RECOVERY - disabled.life - LetsEncrypt on sslhost is OK: OK - Certificate 'disabled.life' will expire on Sun 10 Nov 2019 10:58:16 AM GMT +0000. [12:04:10] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5qn [12:04:11] [02miraheze/ssl] 07MirahezeSSLBot 03b3d768b - Bot: Update SSL cert for lodge.jsnydr.com [12:04:27] RECOVERY - wiki.drones4nature.info - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.drones4nature.info' will expire on Sun 10 Nov 2019 10:52:36 AM GMT +0000. [12:04:46] RECOVERY - wikipuk.cl - LetsEncrypt on sslhost is OK: OK - Certificate 'wikipuk.cl' will expire on Sun 10 Nov 2019 10:54:52 AM GMT +0000. [12:08:07] PROBLEM - wiki.hanaoto.me - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.hanaoto.me' expires in 15 day(s) (Wed 28 Aug 2019 12:05:48 PM GMT +0000). [12:08:21] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5qC [12:08:22] [02miraheze/ssl] 07MirahezeSSLBot 036852192 - Bot: Update SSL cert for wiki.hanaoto.me [12:09:18] PROBLEM - wiki.worlduniversityandschool.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.worlduniversityandschool.org' expires in 15 day(s) (Wed 28 Aug 2019 12:06:56 PM GMT +0000). [12:09:31] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5q8 [12:09:33] [02miraheze/ssl] 07MirahezeSSLBot 0387b5a11 - Bot: Update SSL cert for wiki.worlduniversityandschool.org [12:10:58] PROBLEM - wiki.inebriation-confederation.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.inebriation-confederation.com' expires in 15 day(s) (Wed 28 Aug 2019 12:08:57 PM GMT +0000). [12:11:13] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5qB [12:11:15] [02miraheze/ssl] 07MirahezeSSLBot 03f295049 - Bot: Update SSL cert for wiki.inebriation-confederation.com [12:11:19] PROBLEM - froggy.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'froggy.info' expires in 15 day(s) (Wed 28 Aug 2019 12:08:00 PM GMT +0000). [12:11:31] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5qR [12:11:33] [02miraheze/ssl] 07MirahezeSSLBot 03130feab - Bot: Update SSL cert for froggy.info [12:12:28] PROBLEM - madgenderscience.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'madgenderscience.wiki' expires in 15 day(s) (Wed 28 Aug 2019 12:10:16 PM GMT +0000). [12:12:40] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5qu [12:12:42] [02miraheze/ssl] 07MirahezeSSLBot 03916b1e4 - Bot: Update SSL cert for madgenderscience.wiki [12:12:58] RECOVERY - wiki.inebriation-confederation.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.inebriation-confederation.com' will expire on Sun 10 Nov 2019 11:11:05 AM GMT +0000. [12:13:18] RECOVERY - wiki.worlduniversityandschool.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.worlduniversityandschool.org' will expire on Sun 10 Nov 2019 11:09:25 AM GMT +0000. [12:13:19] RECOVERY - froggy.info - LetsEncrypt on sslhost is OK: OK - Certificate 'froggy.info' will expire on Sun 10 Nov 2019 11:11:25 AM GMT +0000. [12:13:19] PROBLEM - wiki.autocountsoft.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.autocountsoft.com' expires in 15 day(s) (Wed 28 Aug 2019 12:11:14 PM GMT +0000). [12:13:32] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5qa [12:13:33] [02miraheze/ssl] 07MirahezeSSLBot 0364c2aff - Bot: Update SSL cert for wiki.autocountsoft.com [12:13:55] RECOVERY - lodge.jsnydr.com - LetsEncrypt on sslhost is OK: OK - Certificate 'lodge.jsnydr.com' will expire on Sun 10 Nov 2019 11:04:01 AM GMT +0000. [12:13:59] RECOVERY - wiki.tulpa.info - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.tulpa.info' will expire on Sun 10 Nov 2019 11:02:07 AM GMT +0000. [12:14:07] RECOVERY - wiki.hanaoto.me - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.hanaoto.me' will expire on Sun 10 Nov 2019 11:08:14 AM GMT +0000. [12:23:19] RECOVERY - wiki.autocountsoft.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.autocountsoft.com' will expire on Sun 10 Nov 2019 11:13:25 AM GMT +0000. [12:24:28] RECOVERY - madgenderscience.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'madgenderscience.wiki' will expire on Sun 10 Nov 2019 11:12:34 AM GMT +0000. [13:52:31] Good morning [13:52:47] AmandaCath: hiya [13:54:18] hi [13:54:47] Hi JohnLewis RhinosF1 [13:55:03] how u doing AmandaCath [13:55:42] Just fine, thanks [13:55:51] AmandaCath: nice [14:05:48] PROBLEM - mw1 Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 223 failures. Last run 2 minutes ago with 223 failures. Failed resources (up to 3 shown) [14:06:18] PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Puppet has 238 failures. Last run 2 minutes ago with 238 failures. Failed resources (up to 3 shown) [14:06:24] PROBLEM - cp2 Puppet on cp2 is CRITICAL: CRITICAL: Puppet has 292 failures. Last run 2 minutes ago with 292 failures. Failed resources (up to 3 shown): File[reviwiki.info_private],File[wiki.tulpa.info],File[wiki.tulpa.info_private],File[marinebiodiversitymatrix.org] [14:06:39] Whoa [14:06:39] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 191 failures. Last run 2 minutes ago with 191 failures. Failed resources (up to 3 shown): File[wikiescola.com.br],File[wikiescola.com.br_private],File[wiki.jacksonheights.nyc],File[wiki.jacksonheights.nyc_private] [14:06:40] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 25 failures. Last run 3 minutes ago with 25 failures. Failed resources (up to 3 shown) [14:06:45] JohnLewis: ^ [14:06:48] PROBLEM - misc4 Puppet on misc4 is CRITICAL: CRITICAL: Puppet has 9 failures. Last run 3 minutes ago with 9 failures. Failed resources (up to 3 shown) [14:06:53] This doesn't look good [14:06:53] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Puppet has 251 failures. Last run 3 minutes ago with 251 failures. Failed resources (up to 3 shown) [14:06:57] PROBLEM - cp4 Puppet on cp4 is CRITICAL: CRITICAL: Puppet has 235 failures. Last run 3 minutes ago with 235 failures. Failed resources (up to 3 shown) [14:07:00] Hey icinga-miraheze calm down [14:07:16] PROBLEM - mw3 Puppet on mw3 is CRITICAL: CRITICAL: Puppet has 239 failures. Last run 3 minutes ago with 239 failures. Failed resources (up to 3 shown) [14:07:17] AmandaCath: ill log on in a sec to public but might just be spam [14:07:26] paladox: ^ [14:07:43] I mean, 239 failures is a lot [14:07:50] puppet error is a GC [14:08:05] paladox: GC? [14:08:17] Garbage collection [14:08:17] oh, garbage can? [14:08:24] paladox: good [14:09:17] paladox: so it's not legit? [14:09:36] it'll recover soon enough AmandaCath [14:09:40] Yup, it'll resolve it's self on the next run [14:09:47] RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:09:54] there we go [14:14:21] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 22 seconds ago with 0 failures [14:14:39] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:14:47] RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:14:56] RECOVERY - cp4 Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:15:08] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:15:17] RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:16:24] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:16:38] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [16:10:16] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5sT [16:10:17] [02miraheze/services] 07MirahezeSSLBot 0342f9ab1 - BOT: Updating services config for wikis [16:20:16] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5sG [16:20:18] [02miraheze/services] 07MirahezeSSLBot 03d2d19e5 - BOT: Updating services config for wikis [17:53:34] !log install swapspace on misc2 [17:53:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:55:26] !log remove swapspace from misc2 [17:55:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [18:06:50] !log MariaDB [mhglobal]> UPDATE cw_wikis SET wiki_settings = REPLACE(wiki_settings, '"wgServer":"https:\/\/ahmsaqib.cf"', '') WHERE wiki_dbname = 'ahmsaqibwiki'; [18:06:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [18:15:00] PROBLEM - wiki.bullshit.systems - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.bullshit.systems' expires in 15 day(s) (Wed 28 Aug 2019 06:11:29 PM GMT +0000). [18:15:20] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5ZU [18:15:21] [02miraheze/ssl] 07MirahezeSSLBot 036da06b2 - Bot: Update SSL cert for wiki.bullshit.systems [18:16:17] PROBLEM - madzebrascience.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'madzebrascience.wiki' expires in 15 day(s) (Wed 28 Aug 2019 06:14:14 PM GMT +0000). [18:16:31] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5ZI [18:16:32] [02miraheze/ssl] 07MirahezeSSLBot 039fa7cd9 - Bot: Update SSL cert for madzebrascience.wiki [18:16:54] PROBLEM - wiki.kemeneth.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.kemeneth.com' expires in 15 day(s) (Wed 28 Aug 2019 06:13:02 PM GMT +0000). [18:17:09] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5Zq [18:17:11] [02miraheze/ssl] 07MirahezeSSLBot 03536dc8f - Bot: Update SSL cert for wiki.kemeneth.com [18:18:57] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-1/±1] 13https://git.io/fj5Z3 [18:18:58] [02miraheze/ssl] 07Reception123 03ec4f56b - remove www.runeasy.nl cert wiki (runeasy.miraheze.org) deleted [18:19:16] PROBLEM - paramony.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'paramony.org' expires in 15 day(s) (Wed 28 Aug 2019 06:15:18 PM GMT +0000). [18:19:29] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5ZG [18:19:31] [02miraheze/ssl] 07MirahezeSSLBot 03e2507b5 - Bot: Update SSL cert for paramony.org [18:19:50] PROBLEM - secularknowledge.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'secularknowledge.com' expires in 15 day(s) (Wed 28 Aug 2019 06:17:30 PM GMT +0000). [18:20:04] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5Zn [18:20:04] [02miraheze/ssl] 07MirahezeSSLBot 039e01a5a - Bot: Update SSL cert for secularknowledge.com [18:22:57] RECOVERY - paramony.org - LetsEncrypt on sslhost is OK: OK - Certificate 'paramony.org' will expire on Sun 10 Nov 2019 05:19:22 PM GMT +0000. [18:22:57] RECOVERY - wiki.bullshit.systems - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.bullshit.systems' will expire on Sun 10 Nov 2019 05:15:12 PM GMT +0000. [18:23:01] PROBLEM - vise.dayid.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'vise.dayid.org' expires in 15 day(s) (Wed 28 Aug 2019 06:19:03 PM GMT +0000). [18:23:16] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5ZC [18:23:18] [02miraheze/ssl] 07MirahezeSSLBot 03aeb76e1 - Bot: Update SSL cert for vise.dayid.org [18:23:26] PROBLEM - wiki.nowchess.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.nowchess.org' expires in 15 day(s) (Wed 28 Aug 2019 06:20:04 PM GMT +0000). [18:23:38] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5ZW [18:23:40] [02miraheze/ssl] 07MirahezeSSLBot 030303a3a - Bot: Update SSL cert for wiki.nowchess.org [18:23:50] RECOVERY - secularknowledge.com - LetsEncrypt on sslhost is OK: OK - Certificate 'secularknowledge.com' will expire on Sun 10 Nov 2019 05:19:57 PM GMT +0000. [18:24:17] RECOVERY - madzebrascience.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'madzebrascience.wiki' will expire on Sun 10 Nov 2019 05:16:23 PM GMT +0000. [18:24:19] RECOVERY - wiki.kemeneth.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.kemeneth.com' will expire on Sun 10 Nov 2019 05:17:00 PM GMT +0000. [18:24:24] PROBLEM - wiki.pablojprieto.com.es - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.pablojprieto.com.es' expires in 15 day(s) (Wed 28 Aug 2019 06:20:49 PM GMT +0000). [18:24:37] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5Zl [18:24:39] [02miraheze/ssl] 07MirahezeSSLBot 03a038108 - Bot: Update SSL cert for wiki.pablojprieto.com.es [18:26:00] [02miraheze/ssl] 07Reception123 pushed 032 commits to 03master [+0/-1/±6] 13https://git.io/fj5Z4 [18:26:02] [02miraheze/ssl] 07Reception123 03b16427b - remove www.splat-teams.com cert wiki deleted [18:26:03] [02miraheze/ssl] 07Reception123 0326b6f2a - Merge branch 'master' of https://github.com/miraheze/ssl [18:26:04] [ GitHub - miraheze/ssl: A repository for storing and managing SSL certificates for Miraheze ] - github.com [18:33:09] RECOVERY - wiki.nowchess.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.nowchess.org' will expire on Sun 10 Nov 2019 05:23:32 PM GMT +0000. [18:33:58] RECOVERY - wiki.pablojprieto.com.es - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.pablojprieto.com.es' will expire on Sun 10 Nov 2019 05:24:31 PM GMT +0000. [18:34:19] RECOVERY - vise.dayid.org - LetsEncrypt on sslhost is OK: OK - Certificate 'vise.dayid.org' will expire on Sun 10 Nov 2019 05:23:10 PM GMT +0000. [19:24:34] Do we use cologne blue? [19:24:58] yup [19:24:59] https://meta.miraheze.org/w/index.php?title=Miraheze&useskin=cologneblue [19:25:00] [ Meta ] - meta.miraheze.org [19:25:42] Paladox: It's no longer maintained (https://phabricator.wikimedia.org/T223824) [19:25:43] [ Login ] - phabricator.wikimedia.org [19:28:13] ok [19:29:25] something to remove i guess [19:31:35] Paladox: shall I do it later? Do you want to go down the wgSkipSkins root or completely take out? [19:32:02] take it out would be the best option [19:32:32] you can do it if you want :) [19:32:37] Paladox: okay, will do in about 30-45 mins of that's okay [19:32:41] otherwise i'll do it later [19:32:42] ok [19:34:03] any ideas about usage? [19:36:03] hi Examknow. Welcome :) [19:39:09] We also likely need to run https://m.mediawiki.org/wiki/Special:MyLanguage/Manual:%24wgSkipSkins after and I'll look at adding to the main page. Already added a note to CN [19:39:10] [ Manual:$wgSkipSkins - MediaWiki ] - m.mediawiki.org [19:39:26] Run the UserSettings script mentioned I mean [19:43:55] PROBLEM - mw3 Current Load on mw3 is CRITICAL: CRITICAL - load average: 9.93, 7.06, 5.44 [19:47:55] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 6.58, 7.94, 6.26 [19:51:55] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 4.90, 6.34, 5.99 [20:00:18] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5nA [20:00:19] [02miraheze/services] 07MirahezeSSLBot 035694be2 - BOT: Updating services config for wikis [20:30:24] [02mw-config] 07RhinosF1 opened pull request 03#2726: remove cologne blue - 13https://git.io/fj5cB [20:32:17] [02mw-config] 07RhinosF1 synchronize pull request 03#2726: remove cologne blue - 13https://git.io/fj5cB [20:33:00] Reception123: ^ [20:34:36] hi [20:34:37] Has it been announced? Perhaps we should check where it's enabled and let hsers know [20:34:43] Users* [20:34:51] I'll look into it [20:35:07] Reception123: CN post made (see scrollback) [20:35:41] Ok [20:37:54] Reception123: let me know when it's merged and I can add a note to the news section on the main page (or shall I add a 'due to be' note now) [20:38:13] hola Reception123 [20:38:19] Hispano76_: hiya [20:38:29] hola RhinosF1 :) [20:38:57] Hola [20:39:01] RhinosF1: will do [20:39:02] ;) [20:40:54] RhinosF1, that mailing list link in your CN post doesn't seem relevant :P [20:41:29] Voidwalker: that's the Wikimedia announcement link saying it isn't maintained [20:41:49] Voidwalker: wrong link [20:42:00] heh [20:43:25] Voidwalker: updated [20:45:37] Reception123: jump on discord #general [20:47:27] cool [20:53:41] PROBLEM - db4 Disk Space on db4 is WARNING: DISK WARNING - free space: / 25152 MB (6% inode=96%); [20:54:27] Voidwalker: https://github.com/miraheze/mw-config/blob/master/ManageWikiSettings.php#L701 should that be eg https://github.com/miraheze/mw-config/blob/master/ManageWikiSettings.php#L701 [20:54:28] [ mw-config/ManageWikiSettings.php at master · miraheze/mw-config · GitHub ] - github.com [20:54:29] [ mw-config/ManageWikiSettings.php at master · miraheze/mw-config · GitHub ] - github.com [20:55:02] don't think so [20:55:10] Voidwalker: u did it [20:55:34] Voidwalker: https://github.com/miraheze/mw-config/commit/7dbb2d8cd4aa4a49c68e5ab302e21914e63f2301 maybe it should [20:55:35] [ wgApprovedRevs -> egApprovedRevs (#2706) · miraheze/mw-config@7dbb2d8 · GitHub ] - github.com [20:56:13] I'm going off of https://github.com/wikimedia/mediawiki-extensions-ApprovedRevs/blob/c97194b5d133e46eef0e23b33e555c1626c1d8da/extension.json#L172 [20:56:13] [ mediawiki-extensions-ApprovedRevs/extension.json at c97194b5d133e46eef0e23b33e555c1626c1d8da · wikimedia/mediawiki-extensions-ApprovedRevs · GitHub ] - github.com [20:56:24] and the extension documentation, which lists it as eg [20:56:48] Voidwalker: then maybe it should, unconventional [20:59:18] [02MirahezeMagic] 07RhinosF1 synchronize pull request 03#89: [WIP - DO NOT MERGE] (batch 3) i18n per T4370 - 13https://git.io/fj76K [21:06:24] should https://github.com/miraheze/mw-config/blob/19d5cbd4f0fa3e525496f98defc724c299d0e1de/ManageWikiSettings.php#L768 be 'enabling allow's [21:06:24] [ mw-config/ManageWikiSettings.php at 19d5cbd4f0fa3e525496f98defc724c299d0e1de · miraheze/mw-config · GitHub ] - github.com [21:06:35] s/w's/ws' [21:06:35] RhinosF1 meant to say: should https://github.com/miraheze/mw-config/blob/19d5cbd4f0fa3e525496f98defc724c299d0e1de/ManageWikiSettings.php#L768 be 'enabling allows' [21:07:56] [02MirahezeMagic] 07RhinosF1 synchronize pull request 03#89: [WIP - DO NOT MERGE] (batch 3) i18n per T4370 - 13https://git.io/fj76K [21:08:12] Reception123: thoughts ^^^^ [21:10:14] Hispano76_: yes [21:10:20] uh * RhinosF1 [21:10:34] JohnLewis: will fix [21:11:40] [02mw-config] 07RhinosF1 opened pull request 03#2727: fix SPAG - 13https://git.io/fj5CO [21:11:48] JohnLewis: ^ [21:12:20] [02mw-config] 07JohnFLewis closed pull request 03#2727: fix SPAG - 13https://git.io/fj5CO [21:12:22] [02miraheze/mw-config] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5CZ [21:12:23] [02miraheze/mw-config] 07RhinosF1 030552d10 - fix SPAG (#2727) that does not make sense in current form [21:12:39] [02MirahezeMagic] 07RhinosF1 synchronize pull request 03#89: [WIP - DO NOT MERGE] (batch 3) i18n per T4370 - 13https://git.io/fj76K [21:13:02] JohnLewis: thoughts on Cologne Blue [21:13:31] I asked about it's usage stats earlier [21:13:39] JohnLewis: no one checked [21:13:48] I know :) [21:13:59] but on WMF it's 0.1-0.2% I think [21:14:42] I'd expect something massively different here [21:14:58] well, until someone checks then we won't know [21:15:22] !log starting import for iceriawiki - T4646 [21:15:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [21:15:27] !log delete file F1012413 from phab [21:15:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [21:20:47] [02MirahezeMagic] 07RhinosF1 synchronize pull request 03#89: [WIP - DO NOT MERGE] (batch 3) i18n per T4370 - 13https://git.io/fj76K [21:24:21] [02MirahezeMagic] 07RhinosF1 synchronize pull request 03#89: [WIP - DO NOT MERGE] (batch 3) i18n per T4370 - 13https://git.io/fj76K [21:48:07] JohnLewis i dunno how you'll get those stats? [21:48:14] CologneBlue is a global skin on miraheze [21:48:29] paladox: preferences? [21:49:20] ok [21:50:15] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5CA [21:50:16] [02miraheze/puppet] 07paladox 03057f46e - Remove southparkfan ssh keys [21:52:06] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fj5Ch [21:52:08] [02miraheze/puppet] 07paladox 037018bc6 - Add southparkfan android ssh key back [22:01:51] exciting :P [22:27:01] 503 [22:28:18] Hispano76_: fine for me, what does it say at the bottom [22:28:22] or reload? [22:28:58] hum, work [22:30:36] Hispano76_: probably a temp issue [22:35:12] Hello crusty! If you have any questions feel free to ask and someone should answer soon. [22:36:04] I got a 503 as well fyi [22:36:33] crusty: has it started working again? [22:36:42] yup just reloaded [22:37:26] crusty: good, if you get any errors again, look for a message at the bottom of the page that should give a bit more information on what failed (to us at least). [22:39:41] yeah for sure, i'll send them your way [22:39:44] thanks! [22:39:48] Error 503 Backend fetch failed, forwarded for 2a00:23c4:9e1e:e900:1dfb:4b05:4d6d:e60d, 127.0.0.1 [22:39:48] (Varnish XID 44402916) via cp4 at Mon, 05 Aug 2019 21:00:58 GMT. <-- look like this [22:40:05] ^ that at least tells us what failed when [22:40:20] can I ask what you were doing at the time crusty ? [22:41:14] editing a link in a table on a pretty normal page, no images [22:41:58] crusty: okay, np. Sometimes if you do things that timeout then it can cause a 503 like big imports [22:42:38] https://prnt.sc/oru2en [22:42:39] [ Screenshot by Lightshot ] - prnt.sc [22:42:43] gotcha yeah no files larger than 25kb so far [22:43:26] Hispano76_: thx, that's cp2 then. Again, what were you doing at the time? [22:43:50] crusty: okay, if an import ever fails, let us know on Phabricator and a sysadmin will do it for you [22:44:53] entering a page of 269 985 bytes RhinosF1 [22:45:08] 291* [22:45:15] 291 985* [22:45:45] Hispano76_: erm, i can't see icinga moaning about anything and were up now so just keep us up to date [22:46:41] * RhinosF1 will keep an eye out - error rate is 5% [22:50:33] * RhinosF1 sees the error rate drop [22:52:57] paladox: check the swap on cp2, is that normal? [22:53:54] I’m mobile [22:54:09] paladox: https://misc1.miraheze.org/d/W9MIkA7iz/miraheze-cluster?orgId=1&var-job=node&var-node=cp2.miraheze.org&var-port=9100&fullscreen&panelId=21 [22:54:10] [ Grafana ] - misc1.miraheze.org [22:54:55] RhinosF1: generally that error is not because of cp* in any way [22:55:08] if it was cp it'll end up being a plain 503 or a no-connection error [22:55:16] JohnLewis: okay [22:55:54] Crusty: hi, were you importing a dump or importing something? [22:58:23] no, maybe one or two pictures but less than 25kb [22:58:50] I have less than 10 photos uploaded so far in total [22:59:02] all uploaded individually [22:59:02] Oh