[00:01:20] I have an issue with the admins at Animated Foot Scene Wiki (animatedfeet.miraheze.org). It is the most incompetently run wiki I have ever had the misfortune of contributing to. One of the bureaucrats, DeltaRuneFan2001, keeps insisting that everything regarding policies and templates be run through another bureaucrat (Tycio) who has barely been [00:01:21] active on it. On top of that, DeltaRuneFan2001 constantly types in broken English. [00:02:49] Furthermore, there is a Candidates for Deletion category that has a HUGE backlog, and the admins have done fuck all to get through it. [00:03:35] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 29.48, 22.20, 15.99 [00:05:33] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.68, 20.82, 16.22 [00:06:44] PROBLEM - mw6 APT on mw6 is CRITICAL: APT CRITICAL: 11 packages available for upgrade (2 critical updates). [00:07:32] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.74, 19.00, 16.14 [00:16:31] PROBLEM - jobrunner1 APT on jobrunner1 is CRITICAL: APT CRITICAL: 11 packages available for upgrade (2 critical updates). [01:25:10] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JUXr2 [01:25:12] [02miraheze/services] 07MirahezeSSLBot 0372f8bce - BOT: Updating services config for wikis [01:50:19] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JUXKP [01:50:20] [02miraheze/services] 07MirahezeSSLBot 0331300e4 - BOT: Updating services config for wikis [03:14:20] PROBLEM - test2 APT on test2 is CRITICAL: APT CRITICAL: 11 packages available for upgrade (2 critical updates). [03:33:39] PROBLEM - mw5 APT on mw5 is CRITICAL: APT CRITICAL: 11 packages available for upgrade (2 critical updates). [04:34:25] PROBLEM - misc1 APT on misc1 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (1 critical updates). [05:32:49] PROBLEM - mw7 APT on mw7 is CRITICAL: APT CRITICAL: 11 packages available for upgrade (2 critical updates). [05:43:53] PROBLEM - mw4 APT on mw4 is CRITICAL: APT CRITICAL: 11 packages available for upgrade (2 critical updates). [06:04:20] RECOVERY - test2 APT on test2 is OK: APT OK: 9 packages available for upgrade (0 critical updates). [06:08:25] RECOVERY - misc1 APT on misc1 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [06:17:55] RECOVERY - mw4 APT on mw4 is OK: APT OK: 9 packages available for upgrade (0 critical updates). [06:19:40] RECOVERY - mw5 APT on mw5 is OK: APT OK: 9 packages available for upgrade (0 critical updates). [06:30:31] RECOVERY - jobrunner1 APT on jobrunner1 is OK: APT OK: 9 packages available for upgrade (0 critical updates). [06:40:51] RECOVERY - mw7 APT on mw7 is OK: APT OK: 9 packages available for upgrade (0 critical updates). [06:50:44] RECOVERY - mw6 APT on mw6 is OK: APT OK: 9 packages available for upgrade (0 critical updates). [09:08:08] PROBLEM - db7 Check MariaDB Replication on db7 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 158s [12:05:29] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.16, 18.96, 12.65 [12:10:24] PROBLEM - www.wikimicrofinanza.it - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.wikimicrofinanza.it could not be found [12:10:25] PROBLEM - sa.gyaanipedia.co.in - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for sa.gyaanipedia.co.in could not be found [12:10:26] PROBLEM - bn.gyaanipedia.co.in - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for bn.gyaanipedia.co.in could not be found [12:10:26] PROBLEM - wiki.pupilliam.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.pupilliam.com could not be found [12:10:58] PROBLEM - mon1 Current Load on mon1 is CRITICAL: CRITICAL - load average: 4.47, 4.41, 2.88 [12:12:52] RECOVERY - mon1 Current Load on mon1 is OK: OK - load average: 1.72, 3.40, 2.69 [12:17:05] RECOVERY - www.wikimicrofinanza.it - reverse DNS on sslhost is OK: rDNS OK - www.wikimicrofinanza.it reverse DNS resolves to cp6.miraheze.org [12:17:06] RECOVERY - sa.gyaanipedia.co.in - reverse DNS on sslhost is OK: rDNS OK - sa.gyaanipedia.co.in reverse DNS resolves to cp6.miraheze.org [12:17:07] RECOVERY - bn.gyaanipedia.co.in - reverse DNS on sslhost is OK: rDNS OK - bn.gyaanipedia.co.in reverse DNS resolves to cp7.miraheze.org [12:17:08] RECOVERY - wiki.pupilliam.com - reverse DNS on sslhost is OK: rDNS OK - wiki.pupilliam.com reverse DNS resolves to cp7.miraheze.org [12:18:57] RECOVERY - db7 Check MariaDB Replication on db7 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 1s [12:24:37] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.02, 21.39, 23.89 [12:30:36] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.27, 13.89, 19.49 [13:32:26] Hello Eytirth! If you have any questions, feel free to ask and someone should answer soon. [13:32:32] Yes, hello [13:37:17] PROBLEM - jobrunner2 APT on jobrunner2 is CRITICAL: APT CRITICAL: 11 packages available for upgrade (2 critical updates). [14:13:37] howdy! when / if someone has a moment, I'd appreciate some help on re-activating or re-starting my wiki (it went dormant and may have been deleted) [14:28:52] What is the wiki? [14:30:25] the database name was "vitriolwiki", and the URL (before the Let's Encrypt cert) was "vitriol.miraheze.org" [14:33:32] It does look like it was deleted [14:34:01] @Reception123 you wouldnt happen to have an xml dump of that wiki would you? [14:35:09] @Zppix did you check ManageWiki as well? [14:35:47] @Zppix let me check [14:36:22] not in the recent ones, was it not edited for a long time? [14:36:28] thanks so much for all your help - sorry to add to your workload [14:36:36] no, it hasn't been edited in a long time [14:36:59] like since end of last year or so [14:37:28] oh, I could check older XML backups then [14:37:38] but it may take a while [14:38:20] there's not a ton of content there so it may not be worth it - just some setup i'd like to recover but i can start from scratch if necessary. [14:38:38] would starting from scratch be a better idea, or will i run into conflicts requesting the same name? [14:38:54] No I can recreate the wiki from scratch right now if you make a request [14:39:04] but its up to you [14:39:14] we can always import the XML dump if Reception123 is able to find one [14:39:31] sure thing - i'll get right on that - I appreciate the help! no need for anyone to go to any extra trouble [14:40:10] And I can do the Let's encrypt as well if you wish, just would need the request for that too [14:40:38] Zppix: yeah, though it will take a while because I'd need to download the backups from archive.org, unzip and check [14:40:45] ok [14:40:52] Sphinx-vit: was it a private wiki by the way? If it was unfortunately there won't be any backups [14:41:06] yes it was a private wiki, so that makes sense [14:41:28] Sphinx-vit: oh, sorry then :( I hope it's not too much of an inconvenience for you [14:41:53] if your wiki is meant to be read and you start it up again you could perhaps look into https://meta.miraheze.org/wiki/Dormancy_Policy#Exemptions_from_the_Dormancy_Policy to avoid it being deleted again [14:41:54] [ Dormancy Policy - Miraheze Meta ] - meta.miraheze.org [14:42:47] Sphinx-vit: should finish creating per your request any minute now [14:43:30] awesome, thanks! i'll put in a request for the Let's Encrypt certificate via phabricator once I re-read the doc on that to make sure I do it correctly [14:44:00] Sphinx-vit: ok [14:44:52] RECOVERY - oecumene.org - LetsEncrypt on sslhost is OK: OK - Certificate '*.reg.ru' will expire on Fri 12 Feb 2021 07:13:40 GMT +0000. [14:49:15] SSL request made - thank you so much for your help everyone :-) [14:49:51] Will do now [14:54:18] PROBLEM - oecumene.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:56:25] [02miraheze/ssl] 07Pix1234 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JU1yx [14:56:26] [02miraheze/ssl] 07Pix1234 034043f01 - Update vitriolwiki config [15:00:07] yay - it works! thank you Zppix, Reception123, and RhinosF1! [15:01:37] Sphinx-vit: hopefully it does this one was a bit weird to do as the old ssl was never removed. [15:03:19] so far I'm not having any issues with it - cert looks good when i inspect it and I've navigated around a bit with no problem. I'll let you know if anything acts weird though! [15:03:56] Sphinx-vit: if in about 10 minutes you have issues it should resolve itself pretty quick... but it should be fine... [15:04:30] cool, appreciated [15:45:32] PROBLEM - cp3 Current Load on cp3 is CRITICAL: CRITICAL - load average: 2.61, 2.22, 1.19 [15:47:33] RECOVERY - cp3 Current Load on cp3 is OK: OK - load average: 0.50, 1.53, 1.06 [16:28:12] PROBLEM - db7 Check MariaDB Replication on db7 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 188s [16:30:09] RECOVERY - db7 Check MariaDB Replication on db7 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 0s [16:52:26] * hispano76 greetings [17:15:40] PROBLEM - icclopedia.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'icclopedia.org' expires in 15 day(s) (Thu 15 Oct 2020 17:11:15 GMT +0000). [17:21:26] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JUMU7 [17:21:28] [02miraheze/ssl] 07MirahezeSSLBot 033889b5d - Bot: Update SSL cert for icclopedia.org [17:21:50] PROBLEM - ipv6bolivia.tk - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'ipv6bolivia.tk' expires in 15 day(s) (Thu 15 Oct 2020 17:18:46 GMT +0000). [17:29:05] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JUMTN [17:29:07] [02miraheze/ssl] 07MirahezeSSLBot 037a2cc40 - Bot: Update SSL cert for ipv6bolivia.tk [17:30:24] PROBLEM - munwiki.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'munwiki.info' expires in 15 day(s) (Thu 15 Oct 2020 17:27:53 GMT +0000). [17:36:17] RECOVERY - icclopedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'icclopedia.org' will expire on Mon 28 Dec 2020 16:21:20 GMT +0000. [17:38:26] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JUMk9 [17:38:28] [02miraheze/ssl] 07MirahezeSSLBot 03687c245 - Bot: Update SSL cert for munwiki.info [17:42:12] RECOVERY - ipv6bolivia.tk - LetsEncrypt on sslhost is OK: OK - Certificate 'ipv6bolivia.tk' will expire on Mon 28 Dec 2020 16:28:59 GMT +0000. [17:44:06] PROBLEM - ns1 Current Load on ns1 is CRITICAL: CRITICAL - load average: 3.57, 2.04, 0.93 [17:44:18] RECOVERY - munwiki.info - LetsEncrypt on sslhost is OK: OK - Certificate 'munwiki.info' will expire on Mon 28 Dec 2020 16:38:20 GMT +0000. [17:46:06] RECOVERY - ns1 Current Load on ns1 is OK: OK - load average: 0.20, 1.20, 0.86 [18:15:16] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JUMqG [18:15:17] [02miraheze/services] 07MirahezeSSLBot 0374e2b32 - BOT: Updating services config for wikis [18:54:46] !log add toaqwiki discord hook [18:54:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:01:49] Hi [19:02:55] I have an issue with the logo. I think is because i changed the image that i use for logo from tuscriaturaswiki to intercriaturaswiki. The main page is this: https://tuscriaturas.miraheze.org/wiki/El_Bestiario_del_Hypogripho_Dorado and the logo is this one: https://intercriaturas.miraheze.org/wiki/File:Hipogrifo_logo_libro_v2.png [19:02:57] [ Bestiario del Hypogripho ] - tuscriaturas.miraheze.org [19:02:57] [ File:Hipogrifo logo libro v2.png - InterCriaturas ] - intercriaturas.miraheze.org [19:07:56] I updated but i still not seeing the logo. Do you know what i have to do to see the logo again? [19:14:27] paladox i have an issue with the logo. The main logo of the wiki is not showing. I think is because i changed the image that i use for logo from tuscriaturaswiki to intercriaturaswiki. The main page is this: https://tuscriaturas.miraheze.org/wiki/El_Bestiario_del_Hypogripho_Dorado [19:14:28] https://intercriaturas.miraheze.org/wiki/File:Hipogrifo_logo_libro_v2.png [19:14:28] [ File:Hipogrifo logo libro v2.png - InterCriaturas ] - intercriaturas.miraheze.org [19:14:29] [ Bestiario del Hypogripho ] - tuscriaturas.miraheze.org [19:14:49] hi [19:14:54] the logo works for me [19:15:31] I used Google Chrome: Ctrl+Shift+R [19:16:49] https://imgur.com/a/tAhsgiL [19:16:49] [ Imgur: The magic of the Internet ] - imgur.com [19:16:51] is what i see [19:17:41] Ok, thanks [19:19:55] i see this. https://pasteboard.co/JtmLbe1.png What i need to do to see the logo again? [19:19:56] [ No logo - Image on Pasteboard ] - pasteboard.co [19:23:55] you could try refreshing the page. Or opening the console to see if it's failing to load it [19:44:03] Hello humhum! If you have any questions, feel free to ask and someone should answer soon. [20:02:16] why did it take 40 minutes to get a mention to a user in metawiki? Strange XD [20:02:38] who knows [20:03:07] :) [20:18:03] 🖌️ [20:54:11] >  why did it take 40 minutes to get a mention to a user in metawiki? Strange XD @Hispano76 That's the jobrunner servers' responsibility, and that does seem to be a bit of a long backlog in jobs. If that happens again, I would role ping @System Administrators to quickly check the outstanding jobs on Meta. 🙂 [20:55:20] >  i see this. https://pasteboard.co/JtmLbe1.png What i need to do to see the logo again? wonders how many Pastebin-like competitors there are [20:55:21] [ No logo - Image on Pasteboard ] - pasteboard.co [21:09:11] looks at @Zppix and tells them to look at jobs [21:09:39] looks at @RhinosF1 who has the same access as me [21:10:21] says it's 22:10 here and I'm falling asleep [21:11:41] Its one command though [21:12:14] Why must i be the jobs consultant XD [21:13:33] Because you're awake and functioning [21:14:27] Jobs are 0 on meta [21:14:29] lol this thread from 2:09 pm Pacific time was definitely entertaining 🙂 [21:14:45] Jobrunner1 and 2 sometimes have blips [21:15:04] > Jobs are 0 on meta @Zppix Thanks. Must've been a temporary backlog or something, possibly not related to the jobrunner but rather an issue with the Echo notifications. [21:15:12] Nah [21:15:17] It was probably jobrunner [21:15:19] ah [21:16:14] The current job setup is kinda redneck it legit is just a cron that runs a script for all wikis [21:16:37] We should really move to way WM does it [21:17:08] ^ [21:17:51] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JUMRs [21:17:53] [02miraheze/mw-config] 07paladox 03ced80a2 - ldap: Fix config [21:17:54] [02mw-config] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbvb3 [21:17:56] [02mw-config] 07paladox opened pull request 03#3273: ldap: Fix config - 13https://git.io/JUMRG [21:18:24] [02mw-config] 07paladox closed pull request 03#3273: ldap: Fix config - 13https://git.io/JUMRG [21:18:25] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JUMRC [21:18:27] [02miraheze/mw-config] 07paladox 033f1256a - ldap: Fix config (#3273) [21:18:28] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-1 [21:18:30] [02mw-config] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbvb3 [21:19:30] miraheze/mw-config/paladox-patch-1/ced80a2 - paladox The build has errored. https://travis-ci.org/miraheze/mw-config/builds/731413642 [21:37:45] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JUMEk [21:37:46] [02miraheze/mw-config] 07paladox 03b4f0941 - Ldap wiki will host it's own oath table [21:39:09] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JUMEG [21:39:11] [02miraheze/mw-config] 07paladox 039ad6968 - Fix [21:41:42] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JUME4 [21:41:43] [02miraheze/mw-config] 07paladox 0337c1ade - Fix part 2 [21:48:07] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JUMEH [21:48:08] [02miraheze/mw-config] 07paladox 03f5fedd4 - Update LocalSettings.php [22:00:02] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JUMur [22:00:03] [02miraheze/ssl] 07paladox 03fa7972b - Update wiki.alirezaivaz.ir.crt [22:03:16] RECOVERY - wiki.m.alirezaivaz.ir - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.alirezaivaz.ir' will expire on Mon 28 Dec 2020 20:59:30 GMT +0000. [22:05:10] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JUMzf [22:05:12] [02miraheze/ssl] 07paladox 03d1fd29c - Fix [22:16:54] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JUMzx [22:16:55] [02miraheze/ssl] 07paladox 0383cfa33 - Remove oecumene.org No longer pointing at us. [22:16:57] [02ssl] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vxP9L [22:16:58] [02ssl] 07paladox opened pull request 03#363: Remove oecumene.org - 13https://git.io/JUMzp [22:17:14] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-1/±0] 13https://git.io/JUMge [22:17:16] [02miraheze/ssl] 07paladox 03a6c0cbb - Delete oecumene.org.crt [22:17:17] [02ssl] 07paladox synchronize pull request 03#363: Remove oecumene.org - 13https://git.io/JUMzp [22:17:27] [02ssl] 07paladox closed pull request 03#363: Remove oecumene.org - 13https://git.io/JUMzp [22:17:28] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±1] 13https://git.io/JUMgv [22:17:30] [02miraheze/ssl] 07paladox 03b2cea51 - Remove oecumene.org (#363) * Remove oecumene.org No longer pointing at us. * Delete oecumene.org.crt [22:17:31] [02ssl] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vxP9L [22:17:33] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-1 [22:36:13] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_34 [+0/-0/±1] 13https://git.io/JUM2z [22:36:14] [02miraheze/mediawiki] 07paladox 030d837fc - Update Cosmos [22:38:37] [02ManageWiki] 07paladox commented on pull request 03#186: Update dblist after deleting/undeleting a wiki - 13https://git.io/JUM2K [22:38:45] [02ManageWiki] 07paladox closed pull request 03#186: Update dblist after deleting/undeleting a wiki - 13https://git.io/JUP67 [22:38:46] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JUM2i [22:38:48] [02miraheze/ManageWiki] 07paladox 0303532d0 - Update dblist after deleting/undeleting a wiki (#186) * Update dblist after deleting/undeleting a wiki Bug: T6238 * Update ManageWikiFormFactoryBuilder.php [22:38:49] [02ManageWiki] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vpSns [22:38:51] [02miraheze/ManageWiki] 07paladox deleted branch 03paladox-patch-1 [22:39:13] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_34 [+0/-0/±1] 13https://git.io/JUM2M [22:39:15] [02miraheze/mediawiki] 07paladox 03104e622 - Update ManageWiki