[00:01:37] Universal_Omega, or paladox, there seems to a backlog of jobs piling up on at least `metawiki` and `loginwiki`. Mind taking a look and running jobs? [00:03:41] !log root@jobrunner3:/srv/mediawiki/w/maintenance# sudo -u www-data php runJobs.php --wiki metawiki [00:03:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:15:07] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.60, 4.53, 2.69 [00:17:05] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.45, 4.08, 2.75 [00:20:12] thanks [00:24:42] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 4 seconds ago with 0 failures [00:24:57] PROBLEM - cp12 Puppet on cp12 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:31:23] SPF|Cloud around? [00:40:44] paladox: does https://phabricator.miraheze.org/T6860 still happen? If so, any ideas why? [00:40:45] [ ⚓ T6860 runJobs: json_decode() error (4): Syntax error ] - phabricator.miraheze.org [00:41:11] yes [00:43:10] Any ideas why? [00:51:58] paladox, can you look into the jobrunner on `metawiki`? It didn't move `User:HopkinsTheMovie` to `User:JesseWZT` even though it's notionally completed the Meta wiki rename for that user, and I'm not quite sure why [00:52:23] PROBLEM - db12 Puppet on db12 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:52:32] dmehus you should ask the mediawiki team, i'm busy with something else :) [00:52:57] RECOVERY - cp12 Puppet on cp12 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [00:53:15] paladox, okay, but I don't think it's a MediaWiki issue... it finally moved the page, so it was mega backlogged it looks like [01:02:34] Universal_Omega, can you look into why this global rename hasn't gotten past the first wiki? I'm wondering if maybe there's a backlog of jobs on the wiki, a deleted wiki. I assume we can still run jobs on deleted wikis...assuming so, the wiki in question is `jrapp494wiki` [01:20:22] RECOVERY - db12 Puppet on db12 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:39:20] PROBLEM - jobrunner3 Current Load on jobrunner3 is WARNING: WARNING - load average: 5.27, 4.52, 2.90 [01:41:20] RECOVERY - jobrunner3 Current Load on jobrunner3 is OK: OK - load average: 3.32, 4.07, 2.93 [01:45:31] Universal_Omega, is it possible to pause an import job and resume it later? I know Reception123 said we can do that for other scripts, and maybe that one. Reason I ask is the Callipedia import job is impeding performance significantly across the farm, to the point that my last global rename request is stalled [03:36:47] Renames take awhile dmehus regardless [03:36:52] Its probably not related [06:32:23] RECOVERY - bacula2 APT on bacula2 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [06:38:09] !log reception@jobrunner3:~$ sudo -u www-data php /srv/mediawiki/w/extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=jrapp494wiki --logwiki=metawiki 'IntegraInfinity' 'OfficerJH' [06:38:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:10:55] Reception123, thanks :) [07:11:09] * dmehus has a new MediaWiki maintenance script to read about [07:11:28] [[mw:Manual:fixStuckGlobalRename.php]] [07:11:47] !wiki mw:Manual:fixStuckGlobalRename.php [07:13:04] Hrm, wow, I'm surprised there's no MediaWiki.org page for that script (https://www.mediawiki.org/w/index.php?title=Manual:FixStuckGlobalRename.php&action=edit&redlink=1) [07:13:05] [ Creating Manual:FixStuckGlobalRename.php - MediaWiki ] - www.mediawiki.org [07:44:44] it's a centralauth script that's why heh] [07:48:37] ah [07:54:13] dmehus: renames that have been stuck for more than an hour or so very likely have nothing to do with job queue [08:00:17] dmehus: also https://meta.miraheze.org/w/index.php?title=MediaWiki:Sidebar&diff=166269&oldid=166263&diffmode=source [08:00:18] [ Difference between revisions of "MediaWiki:Sidebar" - Miraheze Meta ] - meta.miraheze.org [08:00:54] I still see i18n after purging that [08:00:55] Urgh [08:01:18] But the duplicate is gone [08:02:39] I created https://meta.miraheze.org/wiki/MediaWiki:Miraheze-sidebar-donate though for now [08:02:40] [ MediaWiki:Miraheze-sidebar-donate - Miraheze Meta ] - meta.miraheze.org [08:02:52] Feel free to change word on that or delete when TWN goes live [08:05:34] RhinosF1, yeah I never saw the duplicate Donate link you saw. Now I see no donate link at all. I don't see why we can't keep the fix in the in the MediaWiki:Sidebar until translatewiki.net updates [08:09:05] RhinosF1, seems fine now. Oh well, I'll monitor it and delete the temporary interface message when translatewiki.net is updated [08:11:55] dmehus: your fix didn't fix anything [08:12:08] The temporary interface message does fix any problem I can see [08:12:26] Try purging the page [08:12:32] If it's dodgy for you [08:13:58] RhinosF1, my fix did fix it. I never saw the i18n message. I saw the "Donate to Miraheze" link below the Miraheze links, but maybe I missed the duplicate above Administration, yeah. Anyway, seems resolved, and I'll delete temporary interface message in a few days [08:14:13] Okay [12:02:15] test [13:30:05] PROBLEM - meregos.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for meregos.com could not be found [15:31:02] LakeToday at 4:15 PM [15:31:02] @Reception123 are you allowed to use deleteArchivedRevisions.php or deleteOldRevisions.php? [15:31:21] paladox: ^ was asked this on Discord. Would it be fine or are there copyright issues regarding deleting old revisions? [15:31:41] Depends on the license [15:31:56] I think that Owen should be asked as i'm not sure from a legal point of view. [15:31:59] yeah afaik it's the default MH one [15:32:23] and ok yeah, thought you might've known but better to be safe then [15:32:43] though it might not be applicable anyway if they're all by the same author (figuring that out) [15:33:22] If it's all one author and that author agrees it's fine [15:34:17] yeah, that's clear :) just didn't really understand what was being requested [16:55:50] RhinosF1: it's the redirects I accidentally created when using DataTransfer [16:56:04] I just wanted to delete the pages and the log saying the page was deleted [16:57:19] Universal_Omega: also, does deleteArchivedRevisions.php and/or deleteOldRevisions.php work for this case then? [17:01:38] PROBLEM - ping4 on cp3 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 346.89 ms [17:03:36] RECOVERY - ping4 on cp3 is OK: PING OK - Packet loss = 0%, RTA = 251.85 ms [17:07:49] @Lake, yeah, looking at https://pokemundo.miraheze.org/wiki/Especial:Lista_de_usu%C3%A1rios, only you and Gtbot2007 have contributed to your PokeMundo wiki. So as long as it's not your wiki's Main Page that Gtbot2007 contributed to, the deleteOldRevisions.php (or whatever) maintenance script can be run [17:07:50] [ Lista de usuários - Pokémundo ] - pokemundo.miraheze.org [17:08:33] those pages are literally just redirects to the actual files [17:09:29] the files are sprites and they sometimes change a bit, so I made the redirects for the page go to the most recent sprite of X character [17:09:33] Oh okay, yeah, that seems fine then :) [17:13:39] PROBLEM - ping4 on cp3 is WARNING: PING WARNING - Packet loss = 0%, RTA = 337.54 ms [17:15:39] RECOVERY - ping4 on cp3 is OK: PING OK - Packet loss = 0%, RTA = 250.85 ms [20:12:41] !sre, there's > 400 jobs on `metawiki` and no sign of being cleared, if someone could please run jobs and consider pausing the `callipediawiki` import in progress [20:13:03] s/in progress/in progress, until the job backlog is cleared [20:13:03] dmehus meant to say: !sre, there's > 400 jobs on `metawiki` and no sign of being cleared, if someone could please run jobs and consider pausing the `callipediawiki` import in progress, until the job backlog is cleared [20:14:34] PROBLEM - ping4 on dbbackup2 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 101.24 ms [20:18:34] RECOVERY - ping4 on dbbackup2 is OK: PING OK - Packet loss = 0%, RTA = 102.38 ms [20:21:53] dmehus: the callipediawiki import is no longer in process [20:22:22] !log reception@jobrunner3:/srv/mediawiki/w/maintenance$ sudo -u www-data php runJobs.php --wiki metawiki [20:22:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:22:41] Fatal error: Allowed memory size of 157286400 bytes exhausted (tried to allocate 221572064 bytes) in /srv/mediawiki/w/extensions/CreateWiki/vendor/php-ai/php-ml/src/ModelManager.php on line 35 [20:22:45] ^ paladox hmm that's not nice [20:22:58] JohnLewis ^ [20:23:00] that seems to be why there's so many jobs [20:30:10] PROBLEM - ping6 on dbbackup2 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 102.04 ms [20:32:10] RECOVERY - ping6 on dbbackup2 is OK: PING OK - Packet loss = 0%, RTA = 103.96 ms [20:40:33] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-3 [+0/-0/±1] 13https://git.io/Jqmsf [20:40:35] [02miraheze/mw-config] 07Universal-Omega 036f08c7b - Add ahinfoboxeswiki configs (T6939) [20:40:36] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-3 - 13https://git.io/vbvb3 [20:40:38] [02mw-config] 07Universal-Omega opened pull request 03#3765: Add ahinfoboxeswiki configs (T6939) - 13https://git.io/JqmsJ [20:41:48] miraheze/mw-config - Universal-Omega the build passed. [21:07:09] Reception123, ack and okay, weird that the jobs were so backlogged again :( [21:07:46] I wonder if we might need to disable the CreateWiki AI scoring? It seems to be a memory hog? [21:26:21] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-5 [+0/-0/±1] 13https://git.io/JqmRX [21:26:22] [02miraheze/mw-config] 07Universal-Omega 03861a217 - Add editcontentmodel to wgRestrictionTypes options (T6939) [21:26:24] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-5 - 13https://git.io/vbvb3 [21:26:25] [02mw-config] 07Universal-Omega opened pull request 03#3766: Add editcontentmodel to wgRestrictionTypes options (T6939) - 13https://git.io/JqmRM [21:27:30] miraheze/mw-config - Universal-Omega the build passed. [21:43:36] I see jobs have been ran again, what were the numbers of each job type in queue? [21:46:46] JohnLewis, I don't know the breakdown in job type, but there approximately ~400 total jobs on `metawiki` at the time it was run [21:49:13] Did you see the error Reception123 pasted ^: [21:49:13] > Fatal error: Allowed memory size of 157286400 bytes exhausted (tried to allocate 221572064 bytes) in /srv/mediawiki/w/extensions/CreateWiki/vendor/php-ai/php-ml/src/ModelManager.php on line 35 [21:49:13] Looks like it was before you disconnected, but not sure if you saw it [21:51:06] It’s been reported three times now and my response doesn’t change - if you don’t run it with enough memory it will fail, which is why the job runner has a higher memory limit for that job [21:54:09] PROBLEM - adadevelopersacademy.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'adadevelopersacademy.wiki' expires in 15 day(s) (Thu 25 Mar 2021 21:46:55 GMT +0000). [21:55:34] JohnLewis, ah, okay, fair enough, so should the SRE/MWE be checking memory usage on the jobrunner before running jobs, to make sure there's available memory? [21:55:59] It’s nothing to do with memory usage on the server [21:56:31] oh, how do you mean then? *is confused* [21:56:45] It’s to do with when running the job, they’re not allocating an appropriately large memory threshold via —max-memory [21:56:56] oh [21:58:08] so `--max-memory` is a parameter that needs to be specified in the command to allocate a sufficiently large enough memory threshold? [21:59:07] Yes for CreateWikiAI jobs [22:00:22] ah, right, that makes sense then. I wonder if we should update the appropriate doc on Meta to make a note of that? [22:03:22] There’s no doc for running jobs because it’s a simple script everyone knows and has been doing since day 1 of getting access. The real question remains why people need to keep running jobs - a question I’ve been trying to answer for a week but no one is giving me any answers or information to assist [22:05:58] yeah... the jobs outstanding by job type aren't something that anyone can view (via one of the applicable dashboards, icinga or grafana), right? It has to be a sysadmin? Hopefully next time this happens we can take a moment to obtain this data for you [22:07:06] There’s zero monitoring for jobs, something else I’ve been requesting mw-admins to look at for over a month [22:12:13] [02mw-config] 07Universal-Omega closed pull request 03#3766: Add editcontentmodel to wgRestrictionTypes options (T6939) - 13https://git.io/JqmRM [22:12:14] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jqmi3 [22:12:16] [02miraheze/mw-config] 07Universal-Omega 0317af0aa - Add editcontentmodel to wgRestrictionTypes options (T6939) (#3766) [22:12:17] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-5 - 13https://git.io/vbvb3 [22:12:19] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-5 [22:13:20] miraheze/mw-config - Universal-Omega the build passed. [22:20:22] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-3 [+0/-0/±1] 13https://git.io/JqmPB [22:20:23] [02miraheze/mw-config] 07Universal-Omega 03b7db888 - Add + [22:20:25] [02mw-config] 07Universal-Omega synchronize pull request 03#3765: Add ahinfoboxeswiki configs (T6939) - 13https://git.io/JqmsJ [22:21:30] miraheze/mw-config - Universal-Omega the build passed. [22:21:32] [02mw-config] 07Universal-Omega closed pull request 03#3764: Add $wmgUseParsoid variable - 13https://git.io/JqLzI [22:21:35] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 - 13https://git.io/vbvb3 [22:21:37] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 [22:22:37] PROBLEM - zw.fontainebleau-avon.fr - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'zw.fontainebleau-avon.fr' expires in 15 day(s) (Thu 25 Mar 2021 22:16:47 GMT +0000). [22:37:23] JohnLewis: is there a config I need to set to make new rights assignable from ManageWiki? I thought it would be `wgManageWikiPermissionsAdditionalRights` but that doesn't allow them to be assigned from ManageWiki. [22:38:58] JohnLewis, ack [22:44:07] JohnLewis, there's 174 jobs again on `metawiki`. Alternatively, paladox or Universal_Omega, if you are around, might be worth gathering the statistical data on the outstanding jobs by job type for John? [22:44:28] There's no urgency to look. They will get run when they get run. [22:44:43] paladox, I don't mean to run them... [22:44:53] I mean for the statistical data John is looking for [22:45:05] oh, that's a mediawiki team thing. [22:45:22] That's not paladoxs duty to do, it's ours dmehus. [22:45:30] yeah I know, just thought if you were free, [22:45:42] Universal_Omega, I know, that's why I mainly pinged you [22:45:46] I'll look when I can. Once I finish what I'm doing now. [22:47:01] [02mw-config] 07Universal-Omega synchronize pull request 03#3765: Add ahinfoboxeswiki configs (T6939) - 13https://git.io/JqmsJ [22:47:03] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-3 [+0/-0/±1] 13https://git.io/JqmSx [22:47:04] [02miraheze/mw-config] 07Universal-Omega 031cfe04e - Update LocalSettings.php [22:47:07] Universal_Omega, well not asking you to run any jobs though, but just noting the jobs outstanding can change quickly, so might be worth taking a break for a couple minutes to grab the statistical data, then resume what you're doing? [22:48:02] miraheze/mw-config - Universal-Omega the build passed. [22:48:34] dmehus: I know, doing now. [22:48:44] Universal_Omega, okay, cool, thanks [22:49:26] There's no jobs outstanding now, though :( [22:51:08] dmehus: oh, next time let me know and I'll check. [22:56:07] Universal_Omega well there was > 200 jobs outstanding 7 minutes ago hence why I wanted you to drop everything you were doing for a couple minutes, but okay. 😜 [22:57:59] dmehus: I did end up trying to look immediately, but by the time I got my computer to load correctly, it was to late. Sorry. [22:58:50] dmehus: this was all there was by the time I could look https://www.irccloud.com/pastebin/t5sgXXrK/ [22:58:50] [ Snippet | IRCCloud ] - www.irccloud.com [23:01:46] PROBLEM - www.thesimswiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.thesimswiki.com' expires in 15 day(s) (Thu 25 Mar 2021 22:57:41 GMT +0000). [23:04:22] PROBLEM - wiki.thesimswiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.thesimswiki.com' expires in 15 day(s) (Thu 25 Mar 2021 22:57:41 GMT +0000). [23:05:02] Universal_Omega, okay, no problem. I'll just watch for a pile of jobs to pile up again and ping you again next time [23:05:23] Thanks @Doug! [23:05:27] PROBLEM - thesimswiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.thesimswiki.com' expires in 15 day(s) (Thu 25 Mar 2021 22:57:41 GMT +0000). [23:09:47] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqmdE [23:09:49] [02miraheze/mw-config] 07Universal-Omega 034071e95 - Fix wgRestrictionTypes default to match that of core [23:10:49] miraheze/mw-config - Universal-Omega the build passed. [23:10:49] PROBLEM - cp12 Disk Space on cp12 is WARNING: DISK WARNING - free space: / 4232 MB (10% inode=96%); [23:11:35] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqmdV [23:11:37] [02miraheze/mw-config] 07Universal-Omega 03e6ff014 - Fix wgRestrictionTypes default to match that of core and LS.php [23:12:37] miraheze/mw-config - Universal-Omega the build passed. [23:16:17] PROBLEM - www.pornwiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.pornwiki.org' expires in 15 day(s) (Thu 25 Mar 2021 23:13:54 GMT +0000). [23:16:24] PROBLEM - www.bluepageswiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.bluepageswiki.org' expires in 15 day(s) (Thu 25 Mar 2021 23:13:20 GMT +0000). [23:16:53] PROBLEM - pornwiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.pornwiki.org' expires in 15 day(s) (Thu 25 Mar 2021 23:13:54 GMT +0000). [23:16:53] PROBLEM - baharna.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.baharna.org' expires in 15 day(s) (Thu 25 Mar 2021 23:11:26 GMT +0000). [23:17:40] PROBLEM - bluevies.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.bluevies.org' expires in 15 day(s) (Thu 25 Mar 2021 23:12:56 GMT +0000). [23:17:49] PROBLEM - www.erikapedia.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.erikapedia.com' expires in 15 day(s) (Thu 25 Mar 2021 23:15:05 GMT +0000). [23:18:03] PROBLEM - www.baharna.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.baharna.org' expires in 15 day(s) (Thu 25 Mar 2021 23:11:26 GMT +0000). [23:18:21] PROBLEM - www.portalsofphereon.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.portalsofphereon.com' expires in 15 day(s) (Thu 25 Mar 2021 23:14:02 GMT +0000). [23:18:46] PROBLEM - portalsofphereon.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.portalsofphereon.com' expires in 15 day(s) (Thu 25 Mar 2021 23:14:02 GMT +0000). [23:18:48] PROBLEM - www.iceria.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.iceria.org' expires in 15 day(s) (Thu 25 Mar 2021 23:15:54 GMT +0000). [23:20:12] PROBLEM - rothwell-leeds.co.uk - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.rothwell-leeds.co.uk' expires in 15 day(s) (Thu 25 Mar 2021 23:14:10 GMT +0000). [23:20:24] PROBLEM - lukaibrisimovic.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.lukaibrisimovic.org' expires in 15 day(s) (Thu 25 Mar 2021 23:16:38 GMT +0000). [23:20:42] PROBLEM - petrawiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.petrawiki.org' expires in 15 day(s) (Thu 25 Mar 2021 23:13:44 GMT +0000). [23:20:56] PROBLEM - www.zenbuddhism.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.zenbuddhism.info' expires in 15 day(s) (Thu 25 Mar 2021 23:12:35 GMT +0000). [23:21:05] PROBLEM - www.lukaibrisimovic.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.lukaibrisimovic.org' expires in 15 day(s) (Thu 25 Mar 2021 23:16:38 GMT +0000). [23:21:06] PROBLEM - www.wikimicrofinanza.it - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.wikimicrofinanza.it' expires in 15 day(s) (Thu 25 Mar 2021 23:12:18 GMT +0000). [23:21:52] PROBLEM - www.bluevies.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.bluevies.org' expires in 15 day(s) (Thu 25 Mar 2021 23:12:56 GMT +0000). [23:22:05] PROBLEM - erikapedia.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.erikapedia.com' expires in 15 day(s) (Thu 25 Mar 2021 23:15:05 GMT +0000). [23:22:07] PROBLEM - iceria.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.iceria.org' expires in 15 day(s) (Thu 25 Mar 2021 23:15:54 GMT +0000). [23:22:35] PROBLEM - www.rothwell-leeds.co.uk - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.rothwell-leeds.co.uk' expires in 15 day(s) (Thu 25 Mar 2021 23:14:10 GMT +0000). [23:22:59] PROBLEM - www.mcpk.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.mcpk.wiki' expires in 15 day(s) (Thu 25 Mar 2021 23:16:51 GMT +0000). [23:23:08] PROBLEM - www.lab612.at - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.lab612.at' expires in 15 day(s) (Thu 25 Mar 2021 23:16:16 GMT +0000). [23:23:30] PROBLEM - www.johanloopmans.nl - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.johanloopmans.nl' expires in 15 day(s) (Thu 25 Mar 2021 23:16:07 GMT +0000). [23:24:52] PROBLEM - johanloopmans.nl - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.johanloopmans.nl' expires in 15 day(s) (Thu 25 Mar 2021 23:16:07 GMT +0000). [23:42:58] JohnLewis: Will setting `$wgSitename = $wi->config->settings['wgSitename'][$wi->dbname];` at the top of LS.php (just under `setVariables` work to allow access to that throughout LS)? (Based on https://github.com/miraheze/CreateWiki/blob/master/includes/WikiInitialise.php#L58 it looks like it might - also seems to when testing locally) [23:42:59] [ CreateWiki/WikiInitialise.php at master · miraheze/CreateWiki · GitHub ] - github.com [23:43:32] paladox: any reason why puppet has been disabled on test3 for so long? [23:43:43] no [23:43:56] Can I re-enable then, paladox? [23:44:02] yeh [23:44:06] thanks! [23:48:12] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYvy [23:48:13] [02miraheze/ssl] 07MirahezeSSLBot 03b077b61 - Bot: Update SSL cert for www.pornwiki.org [23:48:31] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYvS [23:48:32] [02miraheze/ssl] 07MirahezeSSLBot 03bc3f2c0 - Bot: Update SSL cert for www.bluepageswiki.org [23:49:06] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYvQ [23:49:07] [02miraheze/ssl] 07MirahezeSSLBot 03f06e4b4 - Bot: Update SSL cert for www.pornwiki.org [23:49:20] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYv5 [23:49:22] [02miraheze/ssl] 07MirahezeSSLBot 03b3d9357 - Bot: Update SSL cert for www.baharna.org [23:49:32] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYvd [23:49:34] [02miraheze/ssl] 07MirahezeSSLBot 03f72f121 - Bot: Update SSL cert for www.bluevies.org [23:49:46] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYvN [23:49:47] [02miraheze/ssl] 07MirahezeSSLBot 03c1b02a7 - Bot: Update SSL cert for www.erikapedia.com [23:49:57] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYvA [23:49:58] [02miraheze/ssl] 07MirahezeSSLBot 03162c7d6 - Bot: Update SSL cert for www.baharna.org [23:50:07] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYvx [23:50:09] [02miraheze/ssl] 07MirahezeSSLBot 03d3dd4cf - Bot: Update SSL cert for www.portalsofphereon.com [23:50:31] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYvp [23:50:33] [02miraheze/ssl] 07MirahezeSSLBot 032f53761 - Bot: Update SSL cert for www.iceria.org [23:50:42] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYvh [23:50:43] [02miraheze/ssl] 07MirahezeSSLBot 03a542590 - Bot: Update SSL cert for www.rothwell-leeds.co.uk [23:50:53] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYvj [23:50:54] [02miraheze/ssl] 07MirahezeSSLBot 030741b10 - Bot: Update SSL cert for www.lukaibrisimovic.org [23:51:05] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYfe [23:51:06] [02miraheze/ssl] 07MirahezeSSLBot 03da34ae1 - Bot: Update SSL cert for www.petrawiki.org [23:51:10] PROBLEM - test3 Puppet on test3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 minute ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [23:51:17] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYff [23:51:19] [02miraheze/ssl] 07MirahezeSSLBot 03335d3f3 - Bot: Update SSL cert for www.zenbuddhism.info [23:51:29] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYfJ [23:51:30] [02miraheze/ssl] 07MirahezeSSLBot 03b835282 - Bot: Update SSL cert for www.lukaibrisimovic.org [23:51:46] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYfU [23:51:47] [02miraheze/ssl] 07MirahezeSSLBot 0300bbd83 - Bot: Update SSL cert for www.wikimicrofinanza.it [23:51:57] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYfT [23:51:59] [02miraheze/ssl] 07MirahezeSSLBot 037417a3a - Bot: Update SSL cert for www.bluevies.org [23:52:20] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYfI [23:52:22] [02miraheze/ssl] 07MirahezeSSLBot 03ece3b64 - Bot: Update SSL cert for www.erikapedia.com [23:52:31] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYft [23:52:32] [02miraheze/ssl] 07MirahezeSSLBot 035e1b0a8 - Bot: Update SSL cert for www.iceria.org [23:52:43] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYfq [23:52:44] [02miraheze/ssl] 07MirahezeSSLBot 03585817c - Bot: Update SSL cert for www.rothwell-leeds.co.uk [23:53:06] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYfm [23:53:07] [02miraheze/ssl] 07MirahezeSSLBot 03d264a8c - Bot: Update SSL cert for www.mcpk.wiki [23:53:23] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYfY [23:53:25] [02miraheze/ssl] 07MirahezeSSLBot 03bb1d810 - Bot: Update SSL cert for www.lab612.at [23:53:36] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqYfO [23:53:38] [02miraheze/ssl] 07MirahezeSSLBot 033925a65 - Bot: Update SSL cert for www.johanloopmans.nl [23:55:02] PROBLEM - cp10 Puppet on cp10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 6 seconds ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [23:55:03] PROBLEM - cp12 Puppet on cp12 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [23:57:10] PROBLEM - cp11 Puppet on cp11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [23:59:38] PROBLEM - www.christipedia.nl - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.christipedia.nl' expires in 15 day(s) (Thu 25 Mar 2021 23:56:36 GMT +0000). [23:59:39] PROBLEM - christipedia.nl - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.christipedia.nl' expires in 15 day(s) (Thu 25 Mar 2021 23:56:36 GMT +0000).