[00:03:52] PROBLEM - graylog2 Puppet on graylog2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:05:34] @Reception123 https://phabricator.miraheze.org/T7279 [00:05:35] [ ⚓ T7279 Polls can't be created on crappygameswiki ] - phabricator.miraheze.org [00:08:16] paladox [00:09:01] SPF|Cloud [00:15:29] southmarkfan[m] [00:15:39] southparkfan[m] [00:16:00] SPF is asleep probably, be calm [00:16:10] Someone will take a look asap [00:17:11] ok [00:26:27] @Lake 💯 :thistbh: ^ [00:31:47] RECOVERY - graylog2 Puppet on graylog2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:46:07] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org [01:22:36] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [01:36:26] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org [01:52:09] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [02:06:00] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org [02:14:51] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [02:28:40] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org [02:37:29] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [02:51:19] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org [03:00:08] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [03:13:57] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org [03:36:35] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [03:50:10] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JseeR [03:50:11] [02miraheze/services] 07MirahezeSSLBot 03afdcdc4 - BOT: Updating services config for wikis [04:11:06] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org [04:19:58] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [05:35:57] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org [05:51:41] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [05:59:55] [02mediawiki] 07Reception123 closed pull request 03#1690: Bump extensions/PollNY from `edbf58e` to `4d7b0e5` - 13https://git.io/J3AU7 [05:59:56] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jse4s [05:59:58] [02miraheze/mediawiki] 07dependabot[bot] 03e195921 - Bump extensions/PollNY from `edbf58e` to `4d7b0e5` (#1690) [05:59:59] [02mediawiki] 07Reception123 deleted branch 03dependabot/submodules/REL1_35/extensions/PollNY-4d7b0e5 - 13https://git.io/vbL5b [06:00:01] [02miraheze/mediawiki] 07Reception123 deleted branch 03dependabot/submodules/REL1_35/extensions/PollNY-4d7b0e5 [06:00:43] [02mediawiki] 07Reception123 closed pull request 03#1533: Bump extensions/PollNY from `031fb61` to `fd25bc2` - 13https://git.io/J3Rqb [06:00:44] [02mediawiki] 07dependabot[bot] commented on pull request 03#1533: Bump extensions/PollNY from `031fb61` to `fd25bc2` - 13https://git.io/Jse4u [06:00:46] [02miraheze/mediawiki] 07Reception123 deleted branch 03dependabot/submodules/REL1_36/extensions/PollNY-fd25bc2 [06:00:47] [02mediawiki] 07Reception123 deleted branch 03dependabot/submodules/REL1_36/extensions/PollNY-fd25bc2 - 13https://git.io/vbL5b [06:02:08] [02mediawiki] 07Reception123 closed pull request 03#1431: Bump extensions/PortableInfobox from `7efce76` to `f713ac0` - 13https://git.io/JYSNv [06:02:10] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jse4M [06:02:11] [02miraheze/mediawiki] 07dependabot[bot] 03fa27af0 - Bump extensions/PortableInfobox from `7efce76` to `f713ac0` (#1431) [06:02:35] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_35/extensions/PortableInfobox-f713ac0 [06:02:36] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_35/extensions/PortableInfobox-f713ac0 - 13https://git.io/vbL5b [06:03:04] [02mediawiki] 07Reception123 closed pull request 03#1496: Bump extensions/AbuseFilter from `e8ce676` to `2758de4` - 13https://git.io/JO5DV [06:03:05] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/Jse4N [06:03:07] [02miraheze/mediawiki] 07dependabot[bot] 03094ffb0 - Bump extensions/AbuseFilter from `e8ce676` to `2758de4` (#1496) [06:03:13] [02mediawiki] 07Reception123 closed pull request 03#1507: Bump extensions/LanguageSelector from `4b2035a` to `c4d0405` - 13https://git.io/JO5yt [06:03:14] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/Jse4x [06:03:16] [02miraheze/mediawiki] 07dependabot[bot] 036ae278d - Bump extensions/LanguageSelector from `4b2035a` to `c4d0405` (#1507) [06:03:18] [02mediawiki] 07Reception123 closed pull request 03#1515: Bump extensions/UrlGetParameters from `f08eb68` to `b938af0` - 13https://git.io/JO5y0 [06:03:20] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/Jse4h [06:03:21] [02miraheze/mediawiki] 07dependabot[bot] 0368e7f2a - Bump extensions/UrlGetParameters from `f08eb68` to `b938af0` (#1515) [06:03:37] [02mediawiki] 07Reception123 closed pull request 03#1517: Bump extensions/PipeEscape from `7d59f4c` to `8dd9026` - 13https://git.io/JO5y2 [06:03:38] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/JseBe [06:03:40] [02miraheze/mediawiki] 07dependabot[bot] 035fec0d6 - Bump extensions/PipeEscape from `7d59f4c` to `8dd9026` (#1517) [06:03:42] [02mediawiki] 07Reception123 closed pull request 03#1521: Bump extensions/NewUserNotif from `e25bee2` to `78eccd9` - 13https://git.io/JO5yM [06:03:43] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/JseBf [06:03:45] [02miraheze/mediawiki] 07dependabot[bot] 03fdd7587 - Bump extensions/NewUserNotif from `e25bee2` to `78eccd9` (#1521) [06:03:46] [02mediawiki] 07Reception123 closed pull request 03#1523: Bump extensions/Collection from `0ee7b44` to `f980d70` - 13https://git.io/JO5yH [06:03:48] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/JseBJ [06:03:49] [02miraheze/mediawiki] 07dependabot[bot] 03d2bd462 - Bump extensions/Collection from `0ee7b44` to `f980d70` (#1523) [06:03:53] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/AbuseFilter-2758de4 - 13https://git.io/vbL5b [06:03:55] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/AbuseFilter-2758de4 [06:03:57] [02mediawiki] 07Reception123 closed pull request 03#1530: Bump extensions/CentralAuth from `25d6296` to `a9fe007` - 13https://git.io/J3RqX [06:03:58] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/JseBI [06:03:59] [02miraheze/mediawiki] 07dependabot[bot] 035b0c9e8 - Bump extensions/CentralAuth from `25d6296` to `a9fe007` (#1530) [06:04:02] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/LanguageSelector-c4d0405 - 13https://git.io/vbL5b [06:04:04] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/LanguageSelector-c4d0405 [06:04:05] [02mediawiki] 07Reception123 closed pull request 03#1536: Bump extensions/SimpleBlogPage from `a343b47` to `e553839` - 13https://git.io/J3Rmf [06:04:07] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/JseBq [06:04:08] [02miraheze/mediawiki] 07dependabot[bot] 03310e008 - Bump extensions/SimpleBlogPage from `a343b47` to `e553839` (#1536) [06:04:10] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/UrlGetParameters-b938af0 [06:04:11] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/UrlGetParameters-b938af0 - 13https://git.io/vbL5b [06:04:14] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/PipeEscape-8dd9026 - 13https://git.io/vbL5b [06:04:16] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/PipeEscape-8dd9026 [06:04:17] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/NewUserNotif-78eccd9 [06:04:19] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/NewUserNotif-78eccd9 - 13https://git.io/vbL5b [06:04:20] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/Collection-f980d70 [06:04:22] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/Collection-f980d70 - 13https://git.io/vbL5b [06:04:23] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/CentralAuth-a9fe007 - 13https://git.io/vbL5b [06:04:25] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/CentralAuth-a9fe007 [06:04:28] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/SimpleBlogPage-e553839 [06:04:29] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/SimpleBlogPage-e553839 - 13https://git.io/vbL5b [06:05:15] [02mediawiki] 07Reception123 closed pull request 03#1541: Bump extensions/MassMessage from `8f27c62` to `054fbd1` - 13https://git.io/J3RmE [06:05:16] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/JseBG [06:05:18] [02miraheze/mediawiki] 07dependabot[bot] 032f492d8 - Bump extensions/MassMessage from `8f27c62` to `054fbd1` (#1541) [06:05:24] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/MassMessage-054fbd1 [06:05:25] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/MassMessage-054fbd1 - 13https://git.io/vbL5b [06:05:30] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org [06:05:30] [02mediawiki] 07Reception123 closed pull request 03#1542: Bump skins/liberty from `00262dc` to `25d70e8` - 13https://git.io/J3Rma [06:05:32] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/JseBC [06:05:33] [02miraheze/mediawiki] 07dependabot[bot] 0399bd302 - Bump skins/liberty from `00262dc` to `25d70e8` (#1542) [06:05:38] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/skins/liberty-25d70e8 [06:05:39] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/skins/liberty-25d70e8 - 13https://git.io/vbL5b [06:05:53] [02mediawiki] 07Reception123 closed pull request 03#1544: Bump extensions/mw-ScratchBlocks4 from `74ca1cd` to `b71d5cd` - 13https://git.io/J3RmX [06:05:54] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/JseBB [06:05:56] [02miraheze/mediawiki] 07dependabot[bot] 035f4aaad - Bump extensions/mw-ScratchBlocks4 from `74ca1cd` to `b71d5cd` (#1544) [06:06:12] [02mw-config] 07Reception123 commented on pull request 03#3858: Add wgScratchBlocks4BlockVersion to ManageWiki - 13https://git.io/JseBu [06:06:14] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/mw-ScratchBlocks4-b71d5cd - 13https://git.io/vbL5b [06:06:15] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_36/extensions/mw-ScratchBlocks4-b71d5cd [06:06:38] [02mediawiki] 07Reception123 closed pull request 03#1559: Bump extensions/Maps from `56bc21f` to `11d8882` - 13https://git.io/J3o14 [06:06:39] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JseBr [06:06:41] [02miraheze/mediawiki] 07dependabot[bot] 0389ad0c4 - Bump extensions/Maps from `56bc21f` to `11d8882` (#1559) [06:06:45] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_35/extensions/Maps-11d8882 [06:06:47] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_35/extensions/Maps-11d8882 - 13https://git.io/vbL5b [06:06:48] [02mediawiki] 07Reception123 closed pull request 03#1564: Bump extensions/CreateWiki from `f424044` to `4c29852` - 13https://git.io/J3o1X [06:06:50] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JseBK [06:06:51] [02miraheze/mediawiki] 07dependabot[bot] 03bd019b7 - Bump extensions/CreateWiki from `f424044` to `4c29852` (#1564) [06:06:54] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_35/extensions/CreateWiki-4c29852 - 13https://git.io/vbL5b [06:06:56] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_35/extensions/CreateWiki-4c29852 [06:07:05] [02mediawiki] 07Reception123 closed pull request 03#1566: Bump vendor from `327fc31` to `6070660` - 13https://git.io/J3o19 [06:07:06] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JseBX [06:07:08] [02miraheze/mediawiki] 07dependabot[bot] 037a4336a - Bump vendor from `327fc31` to `6070660` (#1566) [06:07:14] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_35/vendor-6070660 - 13https://git.io/vbL5b [06:07:16] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_35/vendor-6070660 [06:07:17] [02mediawiki] 07Reception123 closed pull request 03#1568: Bump extensions/Purge from `47b21bc` to `3fadfbd` - 13https://git.io/J3o1F [06:07:19] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JseBM [06:07:20] [02miraheze/mediawiki] 07dependabot[bot] 037383092 - Bump extensions/Purge from `47b21bc` to `3fadfbd` (#1568) [06:07:22] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_35/extensions/Purge-3fadfbd - 13https://git.io/vbL5b [06:07:23] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_35/extensions/Purge-3fadfbd [06:07:56] [02mediawiki] 07Reception123 closed pull request 03#1707: Bump extensions/DataTransfer from `066b2ea` to `4f2e642` - 13https://git.io/J3AT2 [06:07:57] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JseB7 [06:07:59] [02miraheze/mediawiki] 07dependabot[bot] 03fa8f356 - Bump extensions/DataTransfer from `066b2ea` to `4f2e642` (#1707) [06:08:09] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_35/extensions/DataTransfer-4f2e642 - 13https://git.io/vbL5b [06:08:11] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_35/extensions/DataTransfer-4f2e642 [06:10:40] PROBLEM - mw10 Puppet on mw10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [06:11:07] [02mediawiki] 07Reception123 closed pull request 03#1626: Bump extensions/DarkMode from `73d8847` to `267873a` - 13https://git.io/J3AvW [06:11:09] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JseRm [06:11:10] [02miraheze/mediawiki] 07dependabot[bot] 03b705061 - Bump extensions/DarkMode from `73d8847` to `267873a` (#1626) [06:11:12] PROBLEM - mw9 Puppet on mw9 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [06:11:13] PROBLEM - mw11 Puppet on mw11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [06:11:15] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_35/extensions/DarkMode-267873a - 13https://git.io/vbL5b [06:11:16] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_35/extensions/DarkMode-267873a [06:11:26] PROBLEM - mw8 Puppet on mw8 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [06:14:18] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [06:16:15] !log cd /srv/mediawiki/w/extensions && sudo -u www-data git reset --hard origin/REL1_35 && sudo -u www-data git pull ; sudo -u www-data git submodule update' on mw*/jbr* [06:16:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:18:23] PROBLEM - jobrunner3 Puppet on jobrunner3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [06:19:04] !log ALTER TABLE poll_choice MODIFY COLUMN pc_vote_count int NOT NULL default 0; on crappygameswiki [06:19:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:28:07] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org [06:33:25] RECOVERY - mw8 Puppet on mw8 is OK: OK: Puppet is currently enabled, last run 2 seconds ago with 0 failures [06:34:22] RECOVERY - jobrunner3 Puppet on jobrunner3 is OK: OK: Puppet is currently enabled, last run 53 seconds ago with 0 failures [06:34:39] RECOVERY - mw10 Puppet on mw10 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:35:12] RECOVERY - mw9 Puppet on mw9 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:35:13] RECOVERY - mw11 Puppet on mw11 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:37:44] PROBLEM - test3 Puppet on test3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [06:43:21] Umm... why am I stuck in dark mode now? CC @R4356th. I turned it on, switched skins, and now can't turn it off... [06:43:54] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [06:45:24] It is only disableable in vector... [06:45:41] Every other skin I've tried it can be enabled but not disabled. [06:46:26] Oh wait it can only be enabled or disabled in vector? Then going to a skin after enabling it can't be disabled... [06:54:14] Hmm [07:32:10] PROBLEM - wiki.xysspon.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.xysspon.com could not be found [07:32:11] PROBLEM - wiki.ricochetuniverse.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.ricochetuniverse.com could not be found [07:32:11] PROBLEM - www.pornwiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.pornwiki.org could not be found [07:32:12] PROBLEM - viileapedia.ga - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for viileapedia.ga could not be found [07:32:12] PROBLEM - wiki.ciptamedia.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.ciptamedia.org could not be found [07:39:03] RECOVERY - viileapedia.ga - reverse DNS on sslhost is OK: rDNS OK - viileapedia.ga reverse DNS resolves to cp11.miraheze.org [07:39:04] RECOVERY - wiki.ricochetuniverse.com - reverse DNS on sslhost is OK: rDNS OK - wiki.ricochetuniverse.com reverse DNS resolves to cp11.miraheze.org [07:39:04] RECOVERY - www.pornwiki.org - reverse DNS on sslhost is OK: rDNS OK - www.pornwiki.org reverse DNS resolves to cp10.miraheze.org [07:39:04] RECOVERY - wiki.ciptamedia.org - reverse DNS on sslhost is OK: rDNS OK - wiki.ciptamedia.org reverse DNS resolves to cp10.miraheze.org [07:39:09] RECOVERY - wiki.xysspon.com - reverse DNS on sslhost is OK: rDNS OK - wiki.xysspon.com reverse DNS resolves to cp11.miraheze.org [07:39:12] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org [07:54:56] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [08:29:29] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org [09:05:57] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [09:19:47] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org [09:28:38] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [10:03:12] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org [10:18:56] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [10:53:55] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org [11:09:54] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [11:23:43] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org [11:29:19] PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 2.15, 3.72, 2.21 [11:31:19] RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 0.85, 2.68, 2.00 [11:32:34] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [11:46:23] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org [12:02:06] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [12:04:27] Crepusculum: can't repro. Which skins did you use? [12:04:54] (Also, this should probably be in -offtopic.) [12:15:54] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org [12:24:44] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [12:38:09] PROBLEM - mw8 Puppet on mw8 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/mnt/mediawiki-static] [12:38:34] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org [13:08:06] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [14:03:19] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org [14:16:25] PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 1.96, 3.74, 2.51 [14:18:20] RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 0.40, 2.58, 2.23 [14:39:55] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [14:59:04] Hmm, I'm getting very high load times [15:14:30] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org [15:15:20] PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 10.52, 10.47, 5.35 [15:16:31] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.56, 19.50, 15.50 [15:18:31] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 9.26, 15.64, 14.58 [15:21:19] PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 0.75, 3.83, 3.95 [15:25:20] RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 2.00, 2.45, 3.33 [15:31:28] [02mw-config] 07R4356th commented on pull request 03#3858: Add wgScratchBlocks4BlockVersion to ManageWiki - 13https://git.io/JsfWP [15:35:04] Remember when we merged horribletvshowepisodeswiki into terribletvshowswiki? Now let's do the same but for the opposite wiki [15:35:06] https://phabricator.miraheze.org/T7284 [15:35:07] [ ⚓ T7284 Merge marveloustvshowepisodeswiki into besttvshowswiki ] - phabricator.miraheze.org [15:36:38] paladox Reception123 RhinosF1 [15:37:10] A task is enough [15:39:21] ok but this is the last import request that requires staff attention. The terrifictvnetworks merge can be done easily since the dumps are so tiny and I have them downloaded [15:44:03] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [16:10:25] @MarioMario456, so you don't need all the merges done that Blubabluba9990 identified on SN? [16:11:11] only the ones needed for the rebrand. the terrific tv networks merge will be done by myself since the dumps are tiny [16:12:06] I'll keep the names of the music wikis as is, but I'll continue my plan to take over horrible music wiki [16:19:18] ok [16:39:19] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org [17:01:08] Just made up a joke... [17:01:17] paladox, or SPF|Cloud, did you hear what they did to the avid programmer with a GitHub addiction? [17:08:58] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [17:09:54] paladox, or SPF|Cloud, they had him committed [17:10:49] Crepusculum: ^ [17:13:30] Lol. [17:16:48] RhinosF1: ^ [17:20:02] dmehus: funny but it don't need a ping [17:21:03] RhinosF1: heh, ok :P [17:22:46] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org [17:45:24] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [17:48:00] PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 2.82, 4.14, 2.75 [17:49:54] RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 0.79, 2.99, 2.49 [18:11:38] My wiki appears to hav3 some trouble loading thumbnails right now [18:13:54] Link? [18:14:18] Dreamverse.miraheze.org [18:14:29] The images affected appear to be different client to client. [18:17:19] PROBLEM - cp10 Current Load on cp10 is CRITICAL: CRITICAL - load average: 3.99, 4.13, 2.70 [18:18:52] Why isn't icinga-miraheze crying [18:19:16] it wasn't loading for me earlier, but now everything is loading fine [18:19:19] RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 1.45, 3.07, 2.48 [18:19:50] SPF|Cloud, paladox: one can someone fix gluster on mw8, 2) why isn't the check alerting. There's a gluster client one 3) I thought we made it auto correct, it's being failing since 1pm [18:19:55] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp11.miraheze.org [18:20:03] @Lake: it's 25% of traffic [18:20:24] https://icinga.miraheze.org/monitoring/service/show?host=mw8&service=mw8%20Check%20Gluster%20Clients is not OK [18:20:25] [ Icinga Web 2 Login ] - icinga.miraheze.org [18:20:49] !log remount gluster on mw8 [18:21:11] !log test [18:21:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:21:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:21:37] paladox: we just got really slow [18:22:02] !log test2 [18:22:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:22:16] Betterish [18:22:53] paladox: do you know answer to 2 & 3? [18:27:48] paladox: https://icinga.miraheze.org/monitoring/service/show?host=mw8&service=mw8%20Puppet [18:27:49] [ Icinga Web 2 Login ] - icinga.miraheze.org [18:27:52] Still showing crashed [18:28:34] What's going on here? [18:28:48] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [18:29:58] https://phabricator.miraheze.org/T7287 [18:29:59] [ ⚓ T7287 /mnt/mediawiki-static failures on mw8; missing thumbnails in places ] - phabricator.miraheze.org [18:30:08] Somethings not right [18:31:31] Oh puppet is every 10 minutes [18:31:37] So of course I won't see a recovery [18:33:19] Every 30 [18:33:23] Let's check in 5 [18:35:25] RECOVERY - mw8 Puppet on mw8 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:35:53] Yey [18:35:57] Everything works now [18:36:30] Good [18:36:32] Thanks [18:36:56] Not a clue why it failed [18:37:01] That was supposed to be fix [18:37:04] Fixed* [18:44:18] [02miraheze/RemovePII] 07RhinosF1 pushed 031 commit to 03main [+1/-0/±0] 13https://git.io/Jsfph [18:44:19] [02miraheze/RemovePII] 07RhinosF1 03e730d7f - Initial commit [18:44:21] [02RemovePII] 07RhinosF1 created branch 03main - 13https://git.io/Jsfpj [18:45:54] RhinosF1: can the 'main' branch be renamed to 'master' like other extensions? [18:46:28] Crepusculum: master is on the way out [18:46:32] We're just very slow [18:47:03] !log create RemovePII extension [18:47:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:48:03] Crepusculum: you created the RemovePII extension version 1.0 already? :) [18:48:17] RhinosF1: I know, but regardless I dislike the name 'main' anyways, and it can be switched later with everything else I think, if we rename other extensions. I like the term 'master' better personally but if we have to eventually id rather still have this be master and changed later. But I guess up to you. [18:48:27] dmehus: see Phabricator task for it. [18:48:43] Crepusculum: Oh is that what you linked me on IRC? [18:48:47] Yes. [18:48:49] cool [18:48:51] looking [18:49:09] Crepusculum: we should at some point [18:49:15] So I'd rather we pick a name now [18:49:25] That we want to stick too [18:49:50] I don't like 'main' that much either [18:49:50] RhinosF1: personally, I prefer master but that's just a personal opinion. [18:50:07] Crepusculum: it's deemed a bit offensive in some cases [18:50:11] It's industry wide [18:51:21] RhinosF1: I don't understand, how is 'master' deemed offensive? [18:51:25] https://phabricator.wikimedia.org/T254646 [18:51:27] [ ⚓ T254646 Reconsidering how we name things ] - phabricator.wikimedia.org [18:51:29] Crepusculum: ^ [18:52:15] https://github.com/github/renaming [18:52:15] [ GitHub - github/renaming: Guidance for changing the default branch name for GitHub repositories ] - github.com [18:52:24] lol are they complaining because master and slave are terms used in certain relationship cultures? [18:52:39] Google git master main and its a lot better than I can explain [18:52:47] Dirty terminology has always been used in the computer industry, like `mount` for mounting hard drives [18:52:50] dmehus: it's about control [18:52:56] I mean we use kill too [18:53:01] yeah [18:53:06] still I disagree with that [18:53:35] that's Wikimedia's overly liberal community kowtowing to special interest groups :P [18:53:54] I mean git and GitHub have nothing to do with wikimedia [18:54:01] And they long made changes [18:54:07] That was back in October [18:54:09] well GitHub isn't proposing to rename it, are they? [18:54:13] RhinosF1: for now can we stick to master until we find a new name? I 'really' don't like 'main' for the branches. But if we have to use that so be it I guess. [18:54:15] Yes [18:54:28] dmehus: that's why we're having this discussion [18:54:33] dmehus: github did rename master to main as the default branch. [18:54:35] RhinosF1: ack [18:54:46] Crepusculum: Let's see what Reception123 thinks [18:54:58] But we should have a discussion [18:55:05] Crepusculum: yeah I think sticking with `master` for now is fine. I don't see the need to make interim changes if `main` isn't decided on by GitHub [18:55:13] dmehus: it was back in October. [18:55:19] Main has been default since then [18:55:31] Look at the mirahezebots.org repo as my best example [18:55:33] RhinosF1: so why haven't we renamed our existing master branches? [18:55:40] dmehus: because effort [18:55:46] if we don't have to, I'd say stick with `master` for consistency [18:55:52] It took a while for the rename tools to come out [18:55:54] No need to make change for the sake of change [18:55:57] And it's time consuming [18:56:03] I think that's a sysadmin rule, Reception123 told me [18:56:21] I mean sticking with standards and being socially correct isn't for the sake of change [18:56:34] But I do note bots just uses dev/release [18:56:44] well, yes, but you said yourself it's a lot of effort [18:56:48] PROBLEM - wikipariksha.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wikipariksha.com could not be found [18:56:52] to change all our master branches to main [18:57:07] It's effort [18:57:08] It also requires updating the submodules also in order to change them. [18:57:16] either we change all of our master branches to main, or we stick with master. I'm indifferent to either, but do want consistency [18:57:17] It's ^ that's the big bit [18:57:32] But I do have the script for switching submodule branches [18:57:55] Crepusculum: if we do want to stick then I'd rather SRE change the default for new repos too [18:58:01] So let's hear from one of them too [18:58:10] Otherwise every new repo will cause this debate [18:58:18] RhinosF1: sounds good. [18:59:00] well, in fairness, if we do change the default for new repos, we should go to the trouble of updating our existing repo branch names for consistency [18:59:37] The default is currently main [18:59:48] dmehus: that is what we are saying, to change the default to whatever we will use everywhere, the default is main right now. [18:59:51] I'm saying if we want master then we should reset that [19:00:30] PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 3.88, 3.89, 2.80 [19:00:32] Crepusculum, ah, makes sense, yeah [19:00:43] RhinosF1, yeah [19:01:26] The change was forced on 1st October [19:02:25] RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 0.73, 2.71, 2.49 [19:03:08] ah [19:03:15] so it's been default since October 1st [19:03:19] Yep [19:03:21] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: rDNS OK - lcn.zfc.id.lv reverse DNS resolves to cp10.miraheze.org [19:03:25] We don't create new repos very often then [19:03:31] when was the last repo we created? [19:03:34] We've created 1 in that time [19:03:36] before RemovePII? [19:03:39] ah [19:03:41] Yep [19:03:42] RECOVERY - wikipariksha.com - reverse DNS on sslhost is OK: rDNS OK - wikipariksha.com reverse DNS resolves to cp10.miraheze.org [19:03:55] That was the ext updater repo for work I haven't started yet [19:04:01] oh [19:04:13] The Dependabot replacement [19:04:20] ah, interesting [19:04:26] There was GlobalNewFiles repo in the time also. I renamed the branch that time though. [19:04:34] Ah ok [19:04:37] Wasn't GlobalNewFiles in August? [19:04:46] I remember RhinosF1 testing it in August or September [19:04:52] oh wait [19:04:56] that's GlobalUsage [19:05:02] September was a long time ago [19:05:03] I get that confused with GlobalNewFiles lol [19:05:06] Yeah I did global usage [19:05:10] yeah [19:05:14] that's what I remembered [19:05:20] That took a while to get working because long script to set up [19:05:24] yeah [19:05:59] I mean it worked for new things [19:06:08] But old files took a while for it to see [19:06:17] Probably one of the longest scripts I've ever ran [19:12:12] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lcn.zfc.id.lv could not be found [19:14:34] [02RemovePII] 07Universal-Omega opened pull request 03#1: Create SpecialGenerateRandomHash (T7286: phase one) - 13https://git.io/JsJvS [19:16:31] [02RemovePII] 07Universal-Omega synchronize pull request 03#1: Create SpecialGenerateRandomHash (T7286: phase one) - 13https://git.io/JsJvS [19:17:02]