[00:02:51] RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 3760 MB (15% inode=93%); [00:14:01] PROBLEM - bacula2 APT on bacula2 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [00:34:35] Hello Koishi16! If you have any questions, feel free to ask and someone should answer soon. [00:35:54] What if my logo is too big and only show it partly? [00:36:41] Logos display best when sized at 135px or smaller, if they are bigger than that, there isn't all that much you can do but shrink it down [00:37:03] Thanks [00:42:50] PROBLEM - mw7 APT on mw7 is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [00:43:28] PROBLEM - gluster1 APT on gluster1 is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [02:09:51] PROBLEM - mon1 APT on mon1 is CRITICAL: APT CRITICAL: 6 packages available for upgrade (4 critical updates). [02:36:19] PROBLEM - ns2 APT on ns2 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [02:51:33] PROBLEM - mw6 APT on mw6 is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [03:36:05] PROBLEM - pcgamingwiki.elijahpepe.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - pcgamingwiki.elijahpepe.com reverse DNS resolves to stage-sfo2.qencode-encoder-54f4c60cd21111ea92000a08eec0dbba [03:38:03] PROBLEM - cp6 APT on cp6 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [03:39:05] PROBLEM - services2 APT on services2 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [03:40:37] Hello koushi! If you have any questions, feel free to ask and someone should answer soon. [03:41:14] How can I add comments below the pages? [03:56:22] PROBLEM - puppet2 APT on puppet2 is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [04:10:19] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJgZR [04:10:21] [02miraheze/services] 07MirahezeSSLBot 03448b7cf - BOT: Updating services config for wikis [04:11:05] PROBLEM - pcgamingwiki.elijahpepe.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for pcgamingwiki.elijahpepe.com could not be found [04:41:53] PROBLEM - gluster2 APT on gluster2 is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [04:46:05] PROBLEM - pcgamingwiki.elijahpepe.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - pcgamingwiki.elijahpepe.com reverse DNS resolves to stage-sfo2.qencode-encoder-14d717b6d21e11ea92000a08eec0dbba [04:52:09] PROBLEM - ldap1 APT on ldap1 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [04:57:17] [02mw-config] 07Reception123 closed pull request 03#3193: Add Mermaid extension T5204 - 13https://git.io/JJzR8 [04:57:19] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/JJgnM [04:57:20] [02miraheze/mw-config] 07Reception123 03da828b3 - Add Mermaid extension T5204 (#3193) [04:57:22] [02mw-config] 07Reception123 deleted branch 03Reception123-patch-2 - 13https://git.io/vbvb3 [04:57:23] [02miraheze/mw-config] 07Reception123 deleted branch 03Reception123-patch-2 [05:04:47] PROBLEM - test2 Puppet on test2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki config] [05:05:00] !log sudo -u www-data php /srv/mediawiki/w/maintenance/mergeMessageFileList.php --output /srv/mediawiki/config/ExtensionMessageFiles.php --wiki loginwiki [05:05:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:05:46] !log sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildLocalisationCache.php --wiki loginwiki on mw* and jbr1 [05:05:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:07:28] PROBLEM - cp9 APT on cp9 is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [05:09:33] !log sudo apt-get upgrade on ns2/mw6/cp6/services2/puppet2/gluster2 [05:09:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:10:20] RECOVERY - ns2 APT on ns2 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [05:10:27] RECOVERY - cp6 APT on cp6 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [05:11:19] RECOVERY - services2 APT on services2 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [05:11:43] RECOVERY - mw6 APT on mw6 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [05:11:55] RECOVERY - gluster2 APT on gluster2 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [05:12:19] PROBLEM - db12 APT on db12 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [05:12:24] RECOVERY - puppet2 APT on puppet2 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [05:12:41] PROBLEM - mw4 Current Load on mw4 is CRITICAL: CRITICAL - load average: 10.48, 7.03, 4.86 [05:12:41] PROBLEM - mw5 Current Load on mw5 is CRITICAL: CRITICAL - load average: 8.72, 6.79, 4.80 [05:12:57] PROBLEM - miraheze.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for miraheze.com could not be found [05:13:01] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.52, 18.38, 13.62 [05:13:46] PROBLEM - services2 Puppet on services2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:14:05] PROBLEM - cloud1 Current Load on cloud1 is WARNING: WARNING - load average: 20.49, 19.80, 14.82 [05:14:07] PROBLEM - cp6 Puppet on cp6 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:14:39] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 6.28, 6.68, 4.99 [05:14:40] RECOVERY - mw5 Current Load on mw5 is OK: OK - load average: 5.10, 6.21, 4.84 [05:15:01] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 17.00, 18.00, 14.08 [05:16:06] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 13.90, 17.47, 14.56 [05:18:17] !log sudo -u www-data php deleteBatch.php --wiki reallifevillainswiki --r "Requested - T5987" /home/reception/rlv.txt [05:18:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:19:47] RECOVERY - miraheze.com - reverse DNS on sslhost is OK: rDNS OK - miraheze.com reverse DNS resolves to cp6.miraheze.org [05:23:48] RECOVERY - services2 Puppet on services2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:24:07] RECOVERY - cp6 Puppet on cp6 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:29:31] PROBLEM - test2 APT on test2 is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [05:33:45] !log sudo apt-get upgrade on test2 [05:33:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:35:31] RECOVERY - test2 APT on test2 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [05:45:33] PROBLEM - cp3 APT on cp3 is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [05:56:05] PROBLEM - pcgamingwiki.elijahpepe.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for pcgamingwiki.elijahpepe.com could not be found [06:01:51] RECOVERY - mon1 APT on mon1 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [06:02:50] RECOVERY - mw7 APT on mw7 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [06:07:27] RECOVERY - cp9 APT on cp9 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [06:14:37] !log sudo apt-get upgrade on cp3 [06:14:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:15:28] RECOVERY - cp3 APT on cp3 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [06:27:58] RECOVERY - bacula2 APT on bacula2 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [06:28:23] !log sudo apt-get upgrade on bacula2 [06:28:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:30:07] !log reception@jobrunner1:/srv/mediawiki/w/maintenance$ sudo -u www-data php namespaceDupes.php --wiki reallifevillainswiki --fix [06:30:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:39:39] !log reception@jobrunner1:/srv/mediawiki/w/maintenance$ sudo -u www-data php cleanupTitles.php --wiki reallifevillainswiki [06:39:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:40:46] RECOVERY - mw4 APT on mw4 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [06:43:26] RECOVERY - gluster1 APT on gluster1 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [06:52:09] RECOVERY - ldap1 APT on ldap1 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [06:52:19] !log sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=tuscriaturaswiki /home/reception/Wikipedia-20200709122726.xml [06:52:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:52:49] !log sudo apt-get upgrade on ldap1 [06:52:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:58:05] RECOVERY - db12 APT on db12 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [06:59:15] RECOVERY - mw5 APT on mw5 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [08:04:13] PROBLEM - ns1 Current Load on ns1 is WARNING: WARNING - load average: 0.80, 1.85, 1.03 [08:06:13] RECOVERY - ns1 Current Load on ns1 is OK: OK - load average: 0.04, 0.85, 0.81 [08:30:26] [02ssl] 07RhinosF1 opened pull request 03#347: Remove domains pointing in an invalid way - 13https://git.io/JJgz3 [08:31:00] Reception123: ^ [08:31:33] [02ssl] 07RhinosF1 synchronize pull request 03#347: Remove domains pointing in an invalid way - 13https://git.io/JJgz3 [08:32:06] Reception123: can you please remove and unset for them wikis [08:35:48] PROBLEM - db7 APT on db7 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [08:40:00] ok [08:40:55] [02ssl] 07Reception123 closed pull request 03#347: Remove domains pointing in an invalid way - 13https://git.io/JJgz3 [08:40:56] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-5/±1] 13https://git.io/JJggT [08:40:58] [02miraheze/ssl] 07RhinosF1 03018104a - Remove domains not poiniting at us (#347) T5988 [08:42:05] Reception123: ty, they can all be unset [08:45:14] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJggg [08:45:15] [02miraheze/services] 07MirahezeSSLBot 03ffb5c67 - BOT: Updating services config for wikis [08:45:43] done except one [08:45:44] MariaDB [mhglobal]> SELECT * FROM cw_wikis WHERE wiki_url = 'wiki.glaucuslinux.org'; [08:52:43] Reception123: ack [09:20:44] [02ssl] 07RhinosF1 opened pull request 03#348: More - 13https://git.io/JJgVG [09:20:58] Reception123: ^ [09:22:47] PROBLEM - cp7 APT on cp7 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [09:26:29] !log sudo apt-get upgrade on cp7 [09:26:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:26:47] RECOVERY - cp7 APT on cp7 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [09:27:06] [02ssl] 07Reception123 closed pull request 03#348: More - 13https://git.io/JJgVG [09:27:08] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-2/±1] 13https://git.io/JJgVh [09:27:09] [02miraheze/ssl] 07RhinosF1 03696066b - More (#348) * Remove domains not poiniting at us T5988 * remove more T5988 [09:50:21] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJgrR [09:50:23] [02miraheze/services] 07MirahezeSSLBot 03f0d3664 - BOT: Updating services config for wikis [10:03:39] PROBLEM - rdb2 APT on rdb2 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [10:39:34] hi JohnLewis [11:14:10] hi [11:22:23] hi [12:03:38] Reception123: I'll ping you once I merge, just sanity checking the patches. [12:04:56] ok [12:05:25] PROBLEM - phab1 APT on phab1 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [12:06:00] !log sudo apt-get upgrade on phab1 [12:06:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:07:26] RECOVERY - phab1 APT on phab1 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [12:07:36] Reception123: https://github.com/wikimedia/mediawiki-extensions-GlobalUsage/blob/12dafb63c128de7807230db41a06f1481f674c14/sql/GlobalUsage.sql will be needed on comonswiki but I'm standing by and ready! [12:07:37] [ mediawiki-extensions-GlobalUsage/GlobalUsage.sql at 12dafb63c128de7807230db41a06f1481f674c14 · wikimedia/mediawiki-extensions-GlobalUsage · GitHub ] - github.com [12:07:59] Reception123: ping me when you have the sql ready [12:08:09] and mergeMessageLists etc via salt-ssh [12:08:36] Reception123: merging on your +1 [12:09:33] RhinosF1: oh that's actually weird, how do we do the SSL since it's now managed with ManageWiki but this would be a global ext? [12:10:09] *SQL [12:10:12] Reception123: you mean sql and it's on comonswiki that table per the config patch [12:10:32] RhinosF1: ah, so the SQL only needs to be run for commonswiki [12:10:39] ye [12:10:48] Reception123: merging in 3 [12:10:52] ok [12:11:17] [02mw-config] 07RhinosF1 closed pull request 03#3047: Installing GlobalUsage extension - 13https://git.io/JfmuK [12:11:18] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/JJgS8 [12:11:20] [02miraheze/mw-config] 07examknow 03c473a7d - Installing GlobalUsage extension (#3047) [12:11:50] Reception123: over to you, I'm watching for issues [12:13:15] well let's begin [12:13:32] Reception123: ping me once sql and i18n is done [12:13:42] ok [12:13:45] i have a revert ready just in case [12:15:25] Reception123: can you get that sql done asap please? [12:15:45] !log sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php --wiki commonswiki /srv/mediawiki/w/extensions/GlobalUsage/sql/GlobalUsage.sql [12:15:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:16:21] Reception123: sql looks good [12:16:21] !log sudo -u www-data php /srv/mediawiki/w/maintenance/mergeMessageFileList.php --output /srv/mediawiki/config/ExtensionMessageFiles.php --wiki loginwiki on mw* and jbr1 [12:16:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:17:01] !log sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildLocalisationCache.php --wiki loginwiki on mw* and jbr1 [12:17:04] RhinosF1: all done [12:17:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:17:14] Reception123: ty [12:17:56] Reception123: No impact [12:18:12] it's installed for sure [12:18:31] hmm [12:18:33] wgGlobalUsageDatabase is set [12:18:44] RhinosF1: "Run refreshGlobalimagelinks.php on all wikis in your farm. This will take a long time, but only needs to be done once when installing the extension. php extensions/GlobalUsage/maintenance/refreshGlobalimagelinks.php --pages=existing,nonexisting" [12:18:54] Reception123: yeah that, [12:18:57] need to do that [12:19:00] I'll do it in a screen [12:19:32] RhinosF1: oh ok I'll leave it to you then :) remember John's email regarding changes with foreachindblist though [12:19:56] Reception123: getting it up [12:24:12] !log rhinos@jobrunner1:~$ sudo -u www-data nice -19 usr/local/bin/foreachwikiindblist /srv/mediawiki/w/cache/databases.json /srv/mediawiki/w/extensions/GlobalUsage/maintenance/refreshGlobalimagelinks.php --pages=existing,nonexisting [12:24:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:43:41] PROBLEM - mw4 APT on mw4 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [12:43:58] Reception123: can you apt upgradey stuff [12:45:26] K [12:46:27] !log sudo apt-get upgrade on mw4 [12:46:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:47:39] RECOVERY - mw4 APT on mw4 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [12:49:34] Reception123: don't reboot rdb2 without checking which is the master [12:50:33] RhinosF1: why would I reboot? [12:51:05] Reception123: if any update needed it [12:51:38] Ah, well these are just small updates that are quickly done [12:51:41] No need for reboots [12:52:02] [02puppet] 07RhinosF1 opened pull request 03#1461: +2 crons for GlobalUsage - 13https://git.io/JJgQO [12:52:36] [02puppet] 07RhinosF1 synchronize pull request 03#1461: +2 crons for GlobalUsage - 13https://git.io/JJgQO [12:52:41] ok [12:57:07] [02puppet] 07RhinosF1 edited pull request 03#1461: +2 crons for GlobalUsage - 13https://git.io/JJgQO [12:57:16] JohnLewis: ^ merge then [12:59:47] Does the 2nd one really need to be ran fortnightly? [13:00:29] JohnLewis: clean cache, depends how often we care about ensuring it's up to date [13:01:07] mkay [13:01:25] [02puppet] 07JohnFLewis closed pull request 03#1461: +2 crons for GlobalUsage - 13https://git.io/JJgQO [13:01:26] [02miraheze/puppet] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJg7f [13:01:28] [02miraheze/puppet] 07RhinosF1 03d98641b - +2 crons for GlobalUsage (#1461) * +2 crons for GlobalUsage needs a change to https://github.com/miraheze/MirahezeMagic/blob/master/maintenance/populateWikibaseSitesTable.php#L55 make it not wikibase specific first * Update jobrunner.pp [13:01:28] [ MirahezeMagic/populateWikibaseSitesTable.php at master · miraheze/MirahezeMagic · GitHub ] - github.com [13:04:16] [02puppet] 07Reception123 commented on pull request 03#1402: add wildcards for googleusercontent and imgbox T5614 - 13https://git.io/JJg7Y [13:04:18] [02puppet] 07Reception123 closed pull request 03#1402: add wildcards for googleusercontent and imgbox T5614 - 13https://git.io/Jf94J [13:05:00] [02puppet] 07JohnFLewis closed pull request 03#1160: vanish: Remove data schemas from CSP - 13https://git.io/Je9Qb [13:06:46] [02puppet] 07JohnFLewis commented on pull request 03#1382: Introduce new decommission role - 13https://git.io/JJg70 [13:09:30] [02puppet] 07RhinosF1 opened pull request 03#1462: + onlinewebfonts to CSP - 13https://git.io/JJg7D [13:09:45] [02puppet] 07RhinosF1 commented on pull request 03#1373: Add onlinewebfonts to whitelist - 13https://git.io/JJg79 [13:09:46] [02puppet] 07RhinosF1 closed pull request 03#1373: Add onlinewebfonts to whitelist - 13https://git.io/Jf43N [13:16:01] [02puppet] 07JohnFLewis synchronize pull request 03#1390: Increase hst to 31536000 - 13https://git.io/JfMyE [13:16:05] [02miraheze/puppet] 07JohnFLewis pushed 03275 commits to 03paladox-patch-5 [+42/-34/±468] 13https://git.io/JJg5G [13:16:07] [02miraheze/puppet] 07JohnFLewis 037e21f0f - Merge branch 'master' into paladox-patch-5 [13:16:10] [02puppet] 07JohnFLewis closed pull request 03#1390: Increase hst to 31536000 - 13https://git.io/JfMyE [13:16:11] [02miraheze/puppet] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JJg5n [13:16:13] [02miraheze/puppet] 07paladox 031a5dfe3 - Increase hst to 31536000 (#1390) * Increase hst to 31536000 See https://hstspreload.org/?domain=miraheze.org * Update mediawiki.conf Co-authored-by: John F. Lewis [13:16:14] [ HSTS Preload List Submission ] - hstspreload.org [13:17:00] [02puppet] 07JohnFLewis commented on pull request 03#1414: icinga2: Update pki and api classes - 13https://git.io/JJg5W [13:17:32] [02puppet] 07JohnFLewis commented on pull request 03#1424: Cloud: Add ssh::server::permit_root - 13https://git.io/JJg5E [13:17:51] [02puppet] 07JohnFLewis closed pull request 03#1462: + onlinewebfonts to CSP - 13https://git.io/JJg7D [13:17:53] [02miraheze/puppet] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJg5u [13:17:54] [02miraheze/puppet] 07RhinosF1 037f9bb3f - + onlinewebfonts (#1462) [13:18:22] [02mw-config] 07RhinosF1 synchronize pull request 03#2970: Add DT extensions for testing - 13https://git.io/JvNyj [13:19:18] [02mw-config] 07JohnFLewis closed pull request 03#3088: ManageWikiSettings: Stop using null values - 13https://git.io/JfVD5 [13:19:20] [02miraheze/mw-config] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJg5r [13:19:21] [02miraheze/mw-config] 07paladox 03b662ac2 - ManageWikiSettings: Stop using null values (#3088) [13:23:54] [02MirahezeMagic] 07JohnFLewis closed pull request 03#139: attempt to fix script by adding chr - 13https://git.io/JJuhh [13:24:30] [02miraheze/mw-config] 07RhinosF1 pushed 034 commits to 03master [+0/-0/±3] 13https://git.io/JJg5Q [13:24:31] [02miraheze/mw-config] 07RhinosF1 030abe05f - Update extension-list [13:24:33] [02miraheze/mw-config] 07RhinosF1 03b666ca3 - Drop Poll related settings [13:24:34] [02miraheze/mw-config] 07RhinosF1 03a241bee - remove Poll [13:24:36] [02miraheze/mw-config] 07RhinosF1 03542360a - Merge branch 'master' into master [13:25:23] JohnLewis, Reception123: That wasn't supposed to do that but it's got no diff so it's good. [13:25:27] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.88, 20.81, 15.41 [13:25:40] PROBLEM - gluster1 Current Load on gluster1 is CRITICAL: CRITICAL - load average: 8.34, 6.10, 3.43 [13:29:05] [02MirahezeMagic] 07JohnFLewis commented on pull request 03#128: Create migrateLocalToGlobal.php - 13https://git.io/JJgdG [13:29:06] [02MirahezeMagic] 07JohnFLewis closed pull request 03#128: Create migrateLocalToGlobal.php - 13https://git.io/JfQHU [13:29:42] [02landing] 07JohnFLewis commented on pull request 03#38: Update to bootstrap 4.5.0 - 13https://git.io/JJgd8 [13:30:47] [02IncidentReporting] 07JohnFLewis closed pull request 03#12: Fix bug where ->i_responders could be empty - 13https://git.io/JfT98 [13:31:29] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 33.29, 25.39, 19.03 [13:31:42] [02DataDump] 07JohnFLewis reviewed pull request 03#6 commit - 13https://git.io/JJgdw [13:31:58] PROBLEM - cp3 Stunnel Http for mw5 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [13:32:56] [02miraheze/DataDump] 07translatewiki pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/JJgd1 [13:32:57] [02miraheze/DataDump] 07translatewiki 03c7da5a7 - Localisation updates from https://translatewiki.net. [13:32:58] [ Main page - translatewiki.net ] - translatewiki.net [13:32:58] [02miraheze/CreateWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJgdM [13:33:00] [02miraheze/CreateWiki] 07translatewiki 03dc12cce - Localisation updates from https://translatewiki.net. [13:33:01] [ Main page - translatewiki.net ] - translatewiki.net [13:33:01] [02miraheze/MatomoAnalytics] 07translatewiki pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/JJgdD [13:33:03] [02miraheze/MatomoAnalytics] 07translatewiki 030b8413d - Localisation updates from https://translatewiki.net. [13:33:04] [ Main page - translatewiki.net ] - translatewiki.net [13:33:04] [02miraheze/ManageWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JJgdy [13:33:06] [02miraheze/ManageWiki] 07translatewiki 03b9c8164 - Localisation updates from https://translatewiki.net. [13:33:07] [ Main page - translatewiki.net ] - translatewiki.net [13:33:08] [02miraheze/MirahezeMagic] 07translatewiki pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JJgdS [13:33:09] [02miraheze/MirahezeMagic] 07translatewiki 03056161c - Localisation updates from https://translatewiki.net. [13:33:10] [ Main page - translatewiki.net ] - translatewiki.net [13:33:22] PROBLEM - espiral.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for espiral.org could not be found [13:33:27] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.83, 23.14, 18.97 [13:33:39] PROBLEM - wiki.erisly.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.erisly.com could not be found [13:33:58] RECOVERY - cp3 Stunnel Http for mw5 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15609 bytes in 1.005 second response time [13:35:40] PROBLEM - gluster1 Current Load on gluster1 is WARNING: WARNING - load average: 6.03, 7.36, 5.73 [13:37:12] PROBLEM - db7 Check MariaDB Replication on db7 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 164s [13:39:10] RECOVERY - db7 Check MariaDB Replication on db7 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 2s [13:39:40] RECOVERY - gluster1 Current Load on gluster1 is OK: OK - load average: 6.06, 6.65, 5.81 [13:40:04] RECOVERY - espiral.org - reverse DNS on sslhost is OK: rDNS OK - espiral.org reverse DNS resolves to cp6.miraheze.org [13:40:19] RECOVERY - wiki.erisly.com - reverse DNS on sslhost is OK: rDNS OK - wiki.erisly.com reverse DNS resolves to cp6.miraheze.org [13:40:38] [02mw-config] 07RhinosF1 synchronize pull request 03#2970: Add DT extensions for testing - 13https://git.io/JvNyj [13:41:01] [02mw-config] 07RhinosF1 closed pull request 03#2970: Add DT extensions for testing - 13https://git.io/JvNyj [13:41:26] [02mw-config] 07RhinosF1 commented on pull request 03#2970: Add DT extensions for testing - 13https://git.io/JJgFm [13:41:27] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.48, 18.53, 18.42 [13:44:37] PROBLEM - db13 APT on db13 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [13:46:29] PROBLEM - mail1 APT on mail1 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [13:48:24] !log pull in security updates [13:48:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:48:28] RECOVERY - mail1 APT on mail1 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [13:48:36] RECOVERY - db13 APT on db13 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [13:48:50] JohnLewis: on? For? [13:49:17] RhinosF1: icinga alerts? [13:49:42] JohnLewis: every server then pretty much [13:49:58] Log messages should be clear [13:50:24] it is clear, security updates [13:51:34] RECOVERY - rdb2 APT on rdb2 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [13:52:13] RECOVERY - db7 APT on db7 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [13:52:50] PROBLEM - rdb2 Puppet on rdb2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:02:50] RECOVERY - rdb2 Puppet on rdb2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:22:06] PROBLEM - jobrunner1 APT on jobrunner1 is CRITICAL: APT CRITICAL: 6 packages available for upgrade (6 critical updates). [14:25:50] PROBLEM - wiki.teessidehackspace.org.uk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.teessidehackspace.org.uk could not be found [14:32:30] RECOVERY - wiki.teessidehackspace.org.uk - reverse DNS on sslhost is OK: rDNS OK - wiki.teessidehackspace.org.uk reverse DNS resolves to cp7.miraheze.org [14:33:29] PROBLEM - db11 APT on db11 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [15:46:28] teeside? :) [15:46:36] caught my attention :p [15:52:57] PROBLEM - mw4 Current Load on mw4 is WARNING: WARNING - load average: 7.88, 5.79, 4.51 [15:54:58] RECOVERY - mw4 Current Load on mw4 is OK: OK - load average: 3.45, 4.80, 4.30 [15:56:39] PROBLEM - mw5 APT on mw5 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [16:28:36] hello! I noticed Miraheze's Kartographer is using Wikimedia's tile server, which eventually returns 429s. I think we need to switch it use one of the OSM tile servers https://wiki.openstreetmap.org/wiki/Tile_servers [16:28:37] [ Tile servers - OpenStreetMap Wiki ] - wiki.openstreetmap.org [16:33:53] that's a very good idea since WMF tile servers are being used too much afaict [16:35:28] yeah they've had many problems. But I don't think Miraheze was ever one of them [16:38:44] PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2646 MB (10% inode=93%); [16:45:13] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJ2U4 [16:45:14] [02miraheze/services] 07MirahezeSSLBot 037944660 - BOT: Updating services config for wikis [16:49:54] PROBLEM - franchise.franchising.org.ua - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'franchise.franchising.org.ua' expires in 15 day(s) (Sat 15 Aug 2020 16:44:59 GMT +0000). [16:51:11] PROBLEM - rdb1 APT on rdb1 is CRITICAL: APT CRITICAL: 4 packages available for upgrade (4 critical updates). [16:52:23] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJ2UQ [16:52:25] [02miraheze/ssl] 07MirahezeSSLBot 03fd723df - Bot: Update SSL cert for franchise.franchising.org.ua [17:03:42] RECOVERY - franchise.franchising.org.ua - LetsEncrypt on sslhost is OK: OK - Certificate 'franchise.franchising.org.ua' will expire on Wed 28 Oct 2020 15:52:15 GMT +0000. [17:05:11] PROBLEM - services1 APT on services1 is CRITICAL: APT CRITICAL: 5 packages available for upgrade (4 critical updates). [17:07:36] !log sudo apt-get upgrade on services1 [17:07:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:09:10] RECOVERY - services1 APT on services1 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [18:00:13] PROBLEM - wiki.veloren.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.veloren.net reverse DNS resolves to HSI-KBW-078-042-033-234.hsi3.kabel-badenwuerttemberg.de [18:39:31] Is this enough info for request? https://phabricator.miraheze.org/T5989 [18:39:32] [ ⚓ T5989 New protection level on sesupportwiki ] - phabricator.miraheze.org [18:39:40] @Reception123 ^ [18:40:12] Yes [18:40:50] Great. Thanks [19:32:53] Can someone please lock the user ETWGdbfnjkmazrTh‎‎ for vandalism-only account. They have made several invalid wiki requests, and attempted to vandalise my user talk as per the abuse filters. They are also clearly not here. [19:33:25] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:35:21] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [19:52:44] > @Stewards Can someone please lock the user ETWGdbfnjkmazrTh‎‎ for vandalism-only account. They have made several invalid wiki requests, and attempted to vandalise my user talk as per the abuse filters. They are also clearly not here to build miraheze. @TFFfan I've reported this in #cvt where it may get more attention, but this is a reasonable to report it. [20:12:29] * hispano76 greetings [20:20:46] Blocked [20:25:06] hispano76: not you, welcome :P [20:26:04] haha :) mutante [20:36:19] Will review the filters later, but imo, the local block should be fine [20:40:44] @Void: kinda busy, blocked to bring under control and someone can endorse, extend, lock etc. [20:41:13] Hello Majavah! If you have any questions, feel free to ask and someone should answer soon. [20:42:46] Hello :) [20:50:51] Hey Majavah [20:55:13] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJ2Gi [20:55:15] [02miraheze/services] 07MirahezeSSLBot 035061e05 - BOT: Updating services config for wikis [20:55:35] I tried to start lurking here, not to cause wide attention :P [20:57:45] Majavah: we have a huge welcoming party [20:57:55] Obviously only on irc [21:00:55] not a fan [21:01:04] * Majavah attempts to go back into lurking only [21:04:13] PROBLEM - cp9 Current Load on cp9 is WARNING: WARNING - load average: 1.79, 1.83, 1.05 [21:06:13] RECOVERY - cp9 Current Load on cp9 is OK: OK - load average: 0.34, 1.28, 0.95 [21:07:06] Himmalerin: hi [21:07:45] For responding to a declined wiki request asking for a more specific purpose do I edit the wiki request or just make a new one? [21:08:03] hello! [21:08:17] Himmalerin: either, if editing, drop the person a note saying you've updated it that declinded it [21:08:53] Editing seems a little cleaner (less total requests) so I'll go with that, thanks! [21:20:19] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJ2Z2 [21:20:20] [02miraheze/services] 07MirahezeSSLBot 036ec3f72 - BOT: Updating services config for wikis [21:20:49] [02mw-config] 07Amanda-Catherine opened pull request 03#3194: Add "editor" user group and protection level to sesupportwiki T5989 - 13https://git.io/JJ2Zw [21:31:35] Looking at localsettings.php - Can be added import option from Wikipedia to Meta? [21:31:35] https://cdn.discordapp.com/attachments/435711390544560128/738509115038367794/IMG_20200730_233010.jpg [21:37:40] Yes [21:53:06] > Looking at localsettings.php - Can be added import option from Wikipedia to Meta? Is this for transwiki importation on Miraheze? If so, I assumed that used the Special:Interwiki table, but I guess it's a different configuration file? [21:53:34] never uses transwiki importation; always prefers file upload [21:53:47] Interwiki table is managed on wiki [21:54:01] We don't use the messy interwiki.php stuff [21:55:44] >  We don't use the messy interwiki.php stuff interwiki.php is a configuration file that can be used to pull from the interwiki table on wiki in other uses, such as transwiki imports, but it's messy? [21:56:02] just making sure I understood correctly [21:56:49] You have a page on wiki for that, and then run a script that converts it to a config file called Interwiki.php and then that is hooked via LS.php to manage Soecual:Interwiki [22:05:55] >  You have a page on wiki for that, and then run a script that converts it to a config file called Interwiki.php and then that is hooked via LS.php to manage Soecual:Interwiki Ah, thanks! 🙂