[00:02:45] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JqKiI [00:02:46] [02miraheze/puppet] 07paladox 03de86b40 - Update puppetserver_logback.xml [00:02:48] [02puppet] 07paladox synchronize pull request 03#1707: puppetserver/puppetdb: use logrotate - 13https://git.io/JqK0s [00:03:44] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JqKPd [00:03:46] [02miraheze/puppet] 07paladox 0343abccb - Update puppetdb_logback.xml [00:03:47] [02puppet] 07paladox synchronize pull request 03#1707: puppetserver/puppetdb: use logrotate - 13https://git.io/JqK0s [00:04:10] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JqKX2 [00:04:11] [02miraheze/puppet] 07paladox 037c351c7 - Update puppetserver-request-logging.xml [00:04:13] [02puppet] 07paladox synchronize pull request 03#1707: puppetserver/puppetdb: use logrotate - 13https://git.io/JqK0s [00:04:24] [02mediawiki] 07dependabot[bot] opened pull request 03#1280: Bump extensions/SocialProfile from `e0286cb` to `fbb875b` - 13https://git.io/JqKXx [00:04:26] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/SocialProfile-fbb875b [+0/-0/±1] 13https://git.io/JqKXp [00:04:27] [02miraheze/mediawiki] 07dependabot[bot] 03f830051 - Bump extensions/SocialProfile from `e0286cb` to `fbb875b` [00:04:29] [02mediawiki] 07dependabot[bot] labeled pull request 03#1280: Bump extensions/SocialProfile from `e0286cb` to `fbb875b` - 13https://git.io/JqKXx [00:04:30] [02mediawiki] 07dependabot[bot] labeled pull request 03#1280: Bump extensions/SocialProfile from `e0286cb` to `fbb875b` - 13https://git.io/JqKXx [00:04:32] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/SocialProfile-fbb875b - 13https://git.io/vbL5b [00:04:33] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/3D-2549882 [+0/-0/±1] 13https://git.io/JqK1v [00:04:35] [02miraheze/mediawiki] 07dependabot[bot] 03fe44bbe - Bump extensions/3D from `f29cb2a` to `2549882` [00:04:36] [02mediawiki] 07dependabot[bot] opened pull request 03#1281: Bump extensions/3D from `f29cb2a` to `2549882` - 13https://git.io/JqK1J [00:04:38] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/3D-2549882 - 13https://git.io/vbL5b [00:04:39] [02mediawiki] 07dependabot[bot] labeled pull request 03#1281: Bump extensions/3D from `f29cb2a` to `2549882` - 13https://git.io/JqK1J [00:04:41] [02mediawiki] 07dependabot[bot] labeled pull request 03#1281: Bump extensions/3D from `f29cb2a` to `2549882` - 13https://git.io/JqK1J [00:04:42] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/Graph-09f0338 [+0/-0/±1] 13https://git.io/JqK1I [00:04:44] [02miraheze/mediawiki] 07dependabot[bot] 0311b9221 - Bump extensions/Graph from `69ec90a` to `09f0338` [00:04:45] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/Graph-09f0338 - 13https://git.io/vbL5b [00:04:47] [02mediawiki] 07dependabot[bot] opened pull request 03#1282: Bump extensions/Graph from `69ec90a` to `09f0338` - 13https://git.io/JqK1t [00:04:48] [02mediawiki] 07dependabot[bot] labeled pull request 03#1282: Bump extensions/Graph from `69ec90a` to `09f0338` - 13https://git.io/JqK1t [00:04:50] [02mediawiki] 07dependabot[bot] labeled pull request 03#1282: Bump extensions/Graph from `69ec90a` to `09f0338` - 13https://git.io/JqK1t [00:04:51] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/LocalisationUpdate-f1cc54c [+0/-0/±1] 13https://git.io/JqK1Y [00:04:53] [02miraheze/mediawiki] 07dependabot[bot] 037296eeb - Bump extensions/LocalisationUpdate from `4813f8c` to `f1cc54c` [00:04:54] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/LocalisationUpdate-f1cc54c - 13https://git.io/vbL5b [00:04:56] [02mediawiki] 07dependabot[bot] opened pull request 03#1283: Bump extensions/LocalisationUpdate from `4813f8c` to `f1cc54c` - 13https://git.io/JqK1s [00:04:57] [02mediawiki] 07dependabot[bot] labeled pull request 03#1283: Bump extensions/LocalisationUpdate from `4813f8c` to `f1cc54c` - 13https://git.io/JqK1s [00:04:59] [02mediawiki] 07dependabot[bot] labeled pull request 03#1283: Bump extensions/LocalisationUpdate from `4813f8c` to `f1cc54c` - 13https://git.io/JqK1s [00:05:00] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/MatomoAnalytics-aacdf8c [+0/-0/±1] 13https://git.io/JqK1n [00:05:02] [02miraheze/mediawiki] 07dependabot[bot] 03698a145 - Bump extensions/MatomoAnalytics from `411142e` to `aacdf8c` [00:05:03] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/MatomoAnalytics-aacdf8c - 13https://git.io/vbL5b [00:05:05] [02mediawiki] 07dependabot[bot] opened pull request 03#1284: Bump extensions/MatomoAnalytics from `411142e` to `aacdf8c` - 13https://git.io/JqK1l [00:05:06] [02mediawiki] 07dependabot[bot] labeled pull request 03#1284: Bump extensions/MatomoAnalytics from `411142e` to `aacdf8c` - 13https://git.io/JqK1l [00:05:07] [02mediawiki] 07dependabot[bot] labeled pull request 03#1284: Bump extensions/MatomoAnalytics from `411142e` to `aacdf8c` - 13https://git.io/JqK1l [00:05:09] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/ContributionScores-6fac009 [+0/-0/±1] 13https://git.io/JqK1g [00:05:11] [02miraheze/mediawiki] 07dependabot[bot] 034fee494 - Bump extensions/ContributionScores from `0c38613` to `6fac009` [00:05:12] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/ContributionScores-6fac009 - 13https://git.io/vbL5b [00:05:14] [02mediawiki] 07dependabot[bot] opened pull request 03#1285: Bump extensions/ContributionScores from `0c38613` to `6fac009` - 13https://git.io/JqK1a [00:05:15] [02mediawiki] 07dependabot[bot] labeled pull request 03#1285: Bump extensions/ContributionScores from `0c38613` to `6fac009` - 13https://git.io/JqK1a [00:05:17] [02mediawiki] 07dependabot[bot] labeled pull request 03#1285: Bump extensions/ContributionScores from `0c38613` to `6fac009` - 13https://git.io/JqK1a [00:05:18] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/AJAXPoll-3830cb4 [+0/-0/±1] 13https://git.io/JqK1o [00:05:20] [02miraheze/mediawiki] 07dependabot[bot] 03636b505 - Bump extensions/AJAXPoll from `8f2cb6a` to `3830cb4` [00:05:21] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/AJAXPoll-3830cb4 - 13https://git.io/vbL5b [00:05:23] [02mediawiki] 07dependabot[bot] opened pull request 03#1286: Bump extensions/AJAXPoll from `8f2cb6a` to `3830cb4` - 13https://git.io/JqK16 [00:05:24] [02mediawiki] 07dependabot[bot] labeled pull request 03#1286: Bump extensions/AJAXPoll from `8f2cb6a` to `3830cb4` - 13https://git.io/JqK16 [00:05:26] [02mediawiki] 07dependabot[bot] labeled pull request 03#1286: Bump extensions/AJAXPoll from `8f2cb6a` to `3830cb4` - 13https://git.io/JqK16 [00:05:27] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/Maps-d4f8f90 [+0/-0/±1] 13https://git.io/JqK19 [00:05:29] [02miraheze/mediawiki] 07dependabot[bot] 03591337e - Bump extensions/Maps from `79cac63` to `d4f8f90` [00:05:30] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/Maps-d4f8f90 - 13https://git.io/vbL5b [00:05:32] [02mediawiki] 07dependabot[bot] opened pull request 03#1287: Bump extensions/Maps from `79cac63` to `d4f8f90` - 13https://git.io/JqK1Q [00:05:33] [02mediawiki] 07dependabot[bot] labeled pull request 03#1287: Bump extensions/Maps from `79cac63` to `d4f8f90` - 13https://git.io/JqK1Q [00:05:35] [02mediawiki] 07dependabot[bot] labeled pull request 03#1287: Bump extensions/Maps from `79cac63` to `d4f8f90` - 13https://git.io/JqK1Q [00:05:36] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/skins/MinervaNeue-97729f6 [+0/-0/±1] 13https://git.io/JqK1j [00:05:38] [02miraheze/mediawiki] 07dependabot[bot] 03e9c998e - Bump skins/MinervaNeue from `5141b06` to `97729f6` [00:05:39] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/skins/MinervaNeue-97729f6 - 13https://git.io/vbL5b [00:05:41] [02mediawiki] 07dependabot[bot] opened pull request 03#1288: Bump skins/MinervaNeue from `5141b06` to `97729f6` - 13https://git.io/JqKMe [00:05:42] [02mediawiki] 07dependabot[bot] labeled pull request 03#1288: Bump skins/MinervaNeue from `5141b06` to `97729f6` - 13https://git.io/JqKMe [00:05:44] [02mediawiki] 07dependabot[bot] labeled pull request 03#1288: Bump skins/MinervaNeue from `5141b06` to `97729f6` - 13https://git.io/JqKMe [00:05:45] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/GeoCrumbs-ff63e3b [+0/-0/±1] 13https://git.io/JqKMU [00:05:47] [02miraheze/mediawiki] 07dependabot[bot] 03b38116d - Bump extensions/GeoCrumbs from `20fbdc8` to `ff63e3b` [00:05:48] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/GeoCrumbs-ff63e3b - 13https://git.io/vbL5b [00:05:50] [02mediawiki] 07dependabot[bot] opened pull request 03#1289: Bump extensions/GeoCrumbs from `20fbdc8` to `ff63e3b` - 13https://git.io/JqKMk [00:05:51] [02mediawiki] 07dependabot[bot] labeled pull request 03#1289: Bump extensions/GeoCrumbs from `20fbdc8` to `ff63e3b` - 13https://git.io/JqKMk [00:05:53] [02mediawiki] 07dependabot[bot] labeled pull request 03#1289: Bump extensions/GeoCrumbs from `20fbdc8` to `ff63e3b` - 13https://git.io/JqKMk [00:05:54] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/OAuth-5a56193 [+0/-0/±1] 13https://git.io/JqKMO [00:05:56] [02miraheze/mediawiki] 07dependabot[bot] 032781bde - Bump extensions/OAuth from `c66d882` to `5a56193` [00:05:57] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/OAuth-5a56193 - 13https://git.io/vbL5b [00:05:59] [02mediawiki] 07dependabot[bot] opened pull request 03#1290: Bump extensions/OAuth from `c66d882` to `5a56193` - 13https://git.io/JqKM3 [00:06:00] [02mediawiki] 07dependabot[bot] labeled pull request 03#1290: Bump extensions/OAuth from `c66d882` to `5a56193` - 13https://git.io/JqKM3 [00:06:02] [02mediawiki] 07dependabot[bot] labeled pull request 03#1290: Bump extensions/OAuth from `c66d882` to `5a56193` - 13https://git.io/JqKM3 [00:06:03] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/PollNY-ab834a4 [+0/-0/±1] 13https://git.io/JqKM8 [00:06:05] [02miraheze/mediawiki] 07dependabot[bot] 030ad95b3 - Bump extensions/PollNY from `668f186` to `ab834a4` [00:06:06] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/PollNY-ab834a4 - 13https://git.io/vbL5b [00:06:08] [02mediawiki] 07dependabot[bot] opened pull request 03#1291: Bump extensions/PollNY from `668f186` to `ab834a4` - 13https://git.io/JqKMB [00:06:09] [02mediawiki] 07dependabot[bot] labeled pull request 03#1291: Bump extensions/PollNY from `668f186` to `ab834a4` - 13https://git.io/JqKMB [00:06:11] [02mediawiki] 07dependabot[bot] labeled pull request 03#1291: Bump extensions/PollNY from `668f186` to `ab834a4` - 13https://git.io/JqKMB [00:06:12] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/Kartographer-ce6de47 [+0/-0/±1] 13https://git.io/JqKMg [00:06:14] [02miraheze/mediawiki] 07dependabot[bot] 03204dbcd - Bump extensions/Kartographer from `7272b2b` to `ce6de47` [00:06:15] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/Kartographer-ce6de47 - 13https://git.io/vbL5b [00:06:17] [02mediawiki] 07dependabot[bot] opened pull request 03#1292: Bump extensions/Kartographer from `7272b2b` to `ce6de47` - 13https://git.io/JqKMa [00:06:18] [02mediawiki] 07dependabot[bot] labeled pull request 03#1292: Bump extensions/Kartographer from `7272b2b` to `ce6de47` - 13https://git.io/JqKMa [00:06:20] [02mediawiki] 07dependabot[bot] labeled pull request 03#1292: Bump extensions/Kartographer from `7272b2b` to `ce6de47` - 13https://git.io/JqKMa [00:06:21] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/WikiForum-4db12ac [+0/-0/±1] 13https://git.io/JqKMw [00:06:23] [02miraheze/mediawiki] 07dependabot[bot] 036823947 - Bump extensions/WikiForum from `218044c` to `4db12ac` [00:06:24] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/WikiForum-4db12ac - 13https://git.io/vbL5b [00:06:26] [02mediawiki] 07dependabot[bot] opened pull request 03#1293: Bump extensions/WikiForum from `218044c` to `4db12ac` - 13https://git.io/JqKMr [00:06:27] [02mediawiki] 07dependabot[bot] labeled pull request 03#1293: Bump extensions/WikiForum from `218044c` to `4db12ac` - 13https://git.io/JqKMr [00:06:29] [02mediawiki] 07dependabot[bot] labeled pull request 03#1293: Bump extensions/WikiForum from `218044c` to `4db12ac` - 13https://git.io/JqKMr [00:06:30] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/ApprovedRevs-801399b [+0/-0/±1] 13https://git.io/JqKMH [00:06:32] [02miraheze/mediawiki] 07dependabot[bot] 03d0fe50c - Bump extensions/ApprovedRevs from `916a0ea` to `801399b` [00:06:33] [02mediawiki] 07dependabot[bot] opened pull request 03#1294: Bump extensions/ApprovedRevs from `916a0ea` to `801399b` - 13https://git.io/JqKMQ [00:06:35] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/ApprovedRevs-801399b - 13https://git.io/vbL5b [00:06:36] [02mediawiki] 07dependabot[bot] labeled pull request 03#1294: Bump extensions/ApprovedRevs from `916a0ea` to `801399b` - 13https://git.io/JqKMQ [00:06:38] [02mediawiki] 07dependabot[bot] labeled pull request 03#1294: Bump extensions/ApprovedRevs from `916a0ea` to `801399b` - 13https://git.io/JqKMQ [00:06:39] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/QuizGame-e5119a2 [+0/-0/±1] 13https://git.io/JqKMd [00:06:41] [02miraheze/mediawiki] 07dependabot[bot] 034998afc - Bump extensions/QuizGame from `c8268e3` to `e5119a2` [00:06:42] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/QuizGame-e5119a2 - 13https://git.io/vbL5b [00:06:44] [02mediawiki] 07dependabot[bot] opened pull request 03#1295: Bump extensions/QuizGame from `c8268e3` to `e5119a2` - 13https://git.io/JqKMx [00:06:45] [02mediawiki] 07dependabot[bot] labeled pull request 03#1295: Bump extensions/QuizGame from `c8268e3` to `e5119a2` - 13https://git.io/JqKMx [00:06:47] [02mediawiki] 07dependabot[bot] labeled pull request 03#1295: Bump extensions/QuizGame from `c8268e3` to `e5119a2` - 13https://git.io/JqKMx [00:06:48] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/Translate-04be1fd [+0/-0/±1] 13https://git.io/JqKDf [00:06:50] [02miraheze/mediawiki] 07dependabot[bot] 035369bb0 - Bump extensions/Translate from `6d5bcca` to `04be1fd` [00:06:51] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/Translate-04be1fd - 13https://git.io/vbL5b [00:06:53] [02mediawiki] 07dependabot[bot] opened pull request 03#1296: Bump extensions/Translate from `6d5bcca` to `04be1fd` - 13https://git.io/JqKDU [00:06:54] [02mediawiki] 07dependabot[bot] labeled pull request 03#1296: Bump extensions/Translate from `6d5bcca` to `04be1fd` - 13https://git.io/JqKDU [00:06:56] [02mediawiki] 07dependabot[bot] labeled pull request 03#1296: Bump extensions/Translate from `6d5bcca` to `04be1fd` - 13https://git.io/JqKDU [00:06:57] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/NukeDPL-82a69fc [+0/-0/±1] 13https://git.io/JqKDG [00:06:59] [02miraheze/mediawiki] 07dependabot[bot] 03b50fc84 - Bump extensions/NukeDPL from `6858907` to `82a69fc` [00:07:00] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/NukeDPL-82a69fc - 13https://git.io/vbL5b [00:07:02] [02mediawiki] 07dependabot[bot] opened pull request 03#1297: Bump extensions/NukeDPL from `6858907` to `82a69fc` - 13https://git.io/JqKDW [00:07:03] [02mediawiki] 07dependabot[bot] labeled pull request 03#1297: Bump extensions/NukeDPL from `6858907` to `82a69fc` - 13https://git.io/JqKDW [00:07:05] [02mediawiki] 07dependabot[bot] labeled pull request 03#1297: Bump extensions/NukeDPL from `6858907` to `82a69fc` - 13https://git.io/JqKDW [00:07:06] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/SyntaxHighlight_GeSHi-75d4fba [+0/-0/±1] 13https://git.io/JqKDR [00:07:08] [02miraheze/mediawiki] 07dependabot[bot] 03ba066de - Bump extensions/SyntaxHighlight_GeSHi from `5fecf46` to `75d4fba` [00:07:09] [02mediawiki] 07dependabot[bot] opened pull request 03#1298: Bump extensions/SyntaxHighlight_GeSHi from `5fecf46` to `75d4fba` - 13https://git.io/JqKD0 [00:07:11] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/SyntaxHighlight_GeSHi-75d4fba - 13https://git.io/vbL5b [00:07:12] [02mediawiki] 07dependabot[bot] labeled pull request 03#1298: Bump extensions/SyntaxHighlight_GeSHi from `5fecf46` to `75d4fba` - 13https://git.io/JqKD0 [00:07:14] [02mediawiki] 07dependabot[bot] labeled pull request 03#1298: Bump extensions/SyntaxHighlight_GeSHi from `5fecf46` to `75d4fba` - 13https://git.io/JqKD0 [00:07:15] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/ManageWiki-ce4f5ee [+0/-0/±1] 13https://git.io/JqKDa [00:07:17] [02miraheze/mediawiki] 07dependabot[bot] 03954eb25 - Bump extensions/ManageWiki from `9557f3f` to `ce4f5ee` [00:07:18] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/ManageWiki-ce4f5ee - 13https://git.io/vbL5b [00:07:20] [02mediawiki] 07dependabot[bot] opened pull request 03#1299: Bump extensions/ManageWiki from `9557f3f` to `ce4f5ee` - 13https://git.io/JqKDw [00:07:21] [02mediawiki] 07dependabot[bot] labeled pull request 03#1299: Bump extensions/ManageWiki from `9557f3f` to `ce4f5ee` - 13https://git.io/JqKDw [00:07:23] [02mediawiki] 07dependabot[bot] labeled pull request 03#1299: Bump extensions/ManageWiki from `9557f3f` to `ce4f5ee` - 13https://git.io/JqKDw [00:07:24] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/IncidentReporting-633ceae [+0/-0/±1] 13https://git.io/JqKDP [00:07:26] [02miraheze/mediawiki] 07dependabot[bot] 03e06b492 - Bump extensions/IncidentReporting from `4c85e3f` to `633ceae` [00:07:27] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/IncidentReporting-633ceae - 13https://git.io/vbL5b [00:07:29] [02mediawiki] 07dependabot[bot] opened pull request 03#1300: Bump extensions/IncidentReporting from `4c85e3f` to `633ceae` - 13https://git.io/JqKDX [00:07:30] [02mediawiki] 07dependabot[bot] labeled pull request 03#1300: Bump extensions/IncidentReporting from `4c85e3f` to `633ceae` - 13https://git.io/JqKDX [00:07:32] [02mediawiki] 07dependabot[bot] labeled pull request 03#1300: Bump extensions/IncidentReporting from `4c85e3f` to `633ceae` - 13https://git.io/JqKDX [00:07:33] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/HAWelcome-3d81ae1 [+0/-0/±1] 13https://git.io/JqKDH [00:07:35] [02miraheze/mediawiki] 07dependabot[bot] 030dbcd84 - Bump extensions/HAWelcome from `93bc28f` to `3d81ae1` [00:07:36] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/HAWelcome-3d81ae1 - 13https://git.io/vbL5b [00:07:38] [02mediawiki] 07dependabot[bot] opened pull request 03#1301: Bump extensions/HAWelcome from `93bc28f` to `3d81ae1` - 13https://git.io/JqKD7 [00:07:39] [02mediawiki] 07dependabot[bot] labeled pull request 03#1301: Bump extensions/HAWelcome from `93bc28f` to `3d81ae1` - 13https://git.io/JqKD7 [00:07:41] [02mediawiki] 07dependabot[bot] labeled pull request 03#1301: Bump extensions/HAWelcome from `93bc28f` to `3d81ae1` - 13https://git.io/JqKD7 [00:07:42] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/MsCatSelect-6dddf39 [+0/-0/±1] 13https://git.io/JqKDd [00:07:44] [02miraheze/mediawiki] 07dependabot[bot] 03a25550a - Bump extensions/MsCatSelect from `8b91b1a` to `6dddf39` [00:07:45] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/MsCatSelect-6dddf39 - 13https://git.io/vbL5b [00:07:47] [02mediawiki] 07dependabot[bot] opened pull request 03#1302: Bump extensions/MsCatSelect from `8b91b1a` to `6dddf39` - 13https://git.io/JqKDF [00:07:48] [02mediawiki] 07dependabot[bot] labeled pull request 03#1302: Bump extensions/MsCatSelect from `8b91b1a` to `6dddf39` - 13https://git.io/JqKDF [00:07:50] [02mediawiki] 07dependabot[bot] labeled pull request 03#1302: Bump extensions/MsCatSelect from `8b91b1a` to `6dddf39` - 13https://git.io/JqKDF [00:07:51] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/TwitterTag-0c2cee6 [+0/-0/±1] 13https://git.io/JqKyv [00:07:53] [02miraheze/mediawiki] 07dependabot[bot] 035fef7a6 - Bump extensions/TwitterTag from `56bf26b` to `0c2cee6` [00:07:54] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/TwitterTag-0c2cee6 - 13https://git.io/vbL5b [00:07:56] [02mediawiki] 07dependabot[bot] opened pull request 03#1303: Bump extensions/TwitterTag from `56bf26b` to `0c2cee6` - 13https://git.io/JqKyJ [00:07:57] [02mediawiki] 07dependabot[bot] labeled pull request 03#1303: Bump extensions/TwitterTag from `56bf26b` to `0c2cee6` - 13https://git.io/JqKyJ [00:07:59] [02mediawiki] 07dependabot[bot] labeled pull request 03#1303: Bump extensions/TwitterTag from `56bf26b` to `0c2cee6` - 13https://git.io/JqKyJ [00:08:00] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/WikiCategoryTagCloud-eaa931b [+0/-0/±1] 13https://git.io/JqKyq [00:08:02] [02miraheze/mediawiki] 07dependabot[bot] 03851b4be - Bump extensions/WikiCategoryTagCloud from `8d08994` to `eaa931b` [00:08:03] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/WikiCategoryTagCloud-eaa931b - 13https://git.io/vbL5b [00:08:05] [02mediawiki] 07dependabot[bot] opened pull request 03#1304: Bump extensions/WikiCategoryTagCloud from `8d08994` to `eaa931b` - 13https://git.io/JqKym [00:08:06] [02mediawiki] 07dependabot[bot] labeled pull request 03#1304: Bump extensions/WikiCategoryTagCloud from `8d08994` to `eaa931b` - 13https://git.io/JqKym [00:08:08] [02mediawiki] 07dependabot[bot] labeled pull request 03#1304: Bump extensions/WikiCategoryTagCloud from `8d08994` to `eaa931b` - 13https://git.io/JqKym [00:08:09] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/RandomSelection-9063e58 [+0/-0/±1] 13https://git.io/JqKyc [00:08:11] [02miraheze/mediawiki] 07dependabot[bot] 03498c7e7 - Bump extensions/RandomSelection from `79518ac` to `9063e58` [00:08:13] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/RandomSelection-9063e58 - 13https://git.io/vbL5b [00:08:14] [02mediawiki] 07dependabot[bot] opened pull request 03#1305: Bump extensions/RandomSelection from `79518ac` to `9063e58` - 13https://git.io/JqKyW [00:08:15] [02mediawiki] 07dependabot[bot] labeled pull request 03#1305: Bump extensions/RandomSelection from `79518ac` to `9063e58` - 13https://git.io/JqKyW [00:08:17] [02mediawiki] 07dependabot[bot] labeled pull request 03#1305: Bump extensions/RandomSelection from `79518ac` to `9063e58` - 13https://git.io/JqKyW [00:08:18] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/CreateWiki-a2c05b2 [+0/-0/±1] 13https://git.io/JqKyR [00:08:20] [02miraheze/mediawiki] 07dependabot[bot] 030cbb242 - Bump extensions/CreateWiki from `f8eec3f` to `a2c05b2` [00:08:21] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/CreateWiki-a2c05b2 - 13https://git.io/vbL5b [00:08:23] [02mediawiki] 07dependabot[bot] opened pull request 03#1306: Bump extensions/CreateWiki from `f8eec3f` to `a2c05b2` - 13https://git.io/JqKyE [00:08:24] [02mediawiki] 07dependabot[bot] labeled pull request 03#1306: Bump extensions/CreateWiki from `f8eec3f` to `a2c05b2` - 13https://git.io/JqKyE [00:08:26] [02mediawiki] 07dependabot[bot] labeled pull request 03#1306: Bump extensions/CreateWiki from `f8eec3f` to `a2c05b2` - 13https://git.io/JqKyE [00:08:28] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/Purge-47b21bc [+0/-0/±1] 13https://git.io/JqKyi [00:08:29] [02miraheze/mediawiki] 07dependabot[bot] 0327ff4c8 - Bump extensions/Purge from `218b277` to `47b21bc` [00:08:30] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/Purge-47b21bc - 13https://git.io/vbL5b [00:08:32] [02mediawiki] 07dependabot[bot] opened pull request 03#1307: Bump extensions/Purge from `218b277` to `47b21bc` - 13https://git.io/JqKyP [00:08:33] [02mediawiki] 07dependabot[bot] labeled pull request 03#1307: Bump extensions/Purge from `218b277` to `47b21bc` - 13https://git.io/JqKyP [00:08:35] [02mediawiki] 07dependabot[bot] labeled pull request 03#1307: Bump extensions/Purge from `218b277` to `47b21bc` - 13https://git.io/JqKyP [00:08:36] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/ProtectSite-76ba0e8 [+0/-0/±1] 13https://git.io/JqKyD [00:08:38] [02miraheze/mediawiki] 07dependabot[bot] 03a9c9084 - Bump extensions/ProtectSite from `ba3a7e1` to `76ba0e8` [00:08:39] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/ProtectSite-76ba0e8 - 13https://git.io/vbL5b [00:08:41] [02mediawiki] 07dependabot[bot] opened pull request 03#1308: Bump extensions/ProtectSite from `ba3a7e1` to `76ba0e8` - 13https://git.io/JqKyS [00:08:42] [02mediawiki] 07dependabot[bot] labeled pull request 03#1308: Bump extensions/ProtectSite from `ba3a7e1` to `76ba0e8` - 13https://git.io/JqKyS [00:08:44] [02mediawiki] 07dependabot[bot] labeled pull request 03#1308: Bump extensions/ProtectSite from `ba3a7e1` to `76ba0e8` - 13https://git.io/JqKyS [00:08:45] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/MirahezeMagic-f4c85c0 [+0/-0/±1] 13https://git.io/JqKyd [00:08:47] [02miraheze/mediawiki] 07dependabot[bot] 03fc28124 - Bump extensions/MirahezeMagic from `d9e7dd4` to `f4c85c0` [00:08:48] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/MirahezeMagic-f4c85c0 - 13https://git.io/vbL5b [00:08:50] [02mediawiki] 07dependabot[bot] opened pull request 03#1309: Bump extensions/MirahezeMagic from `d9e7dd4` to `f4c85c0` - 13https://git.io/JqKyN [00:08:52] [02mediawiki] 07dependabot[bot] labeled pull request 03#1309: Bump extensions/MirahezeMagic from `d9e7dd4` to `f4c85c0` - 13https://git.io/JqKyN [00:08:53] [02mediawiki] 07dependabot[bot] labeled pull request 03#1309: Bump extensions/MirahezeMagic from `d9e7dd4` to `f4c85c0` - 13https://git.io/JqKyN [00:08:54] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/VoteNY-c7a829a [+0/-0/±1] 13https://git.io/JqKSv [00:08:56] [02miraheze/mediawiki] 07dependabot[bot] 0373a9939 - Bump extensions/VoteNY from `80c2e22` to `c7a829a` [00:08:57] [02mediawiki] 07dependabot[bot] opened pull request 03#1310: Bump extensions/VoteNY from `80c2e22` to `c7a829a` - 13https://git.io/JqKSf [00:08:59] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/VoteNY-c7a829a - 13https://git.io/vbL5b [00:09:00] [02mediawiki] 07dependabot[bot] labeled pull request 03#1310: Bump extensions/VoteNY from `80c2e22` to `c7a829a` - 13https://git.io/JqKSf [00:09:02] [02mediawiki] 07dependabot[bot] labeled pull request 03#1310: Bump extensions/VoteNY from `80c2e22` to `c7a829a` - 13https://git.io/JqKSf [00:09:03] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/Comments-c70f7d5 [+0/-0/±1] 13https://git.io/JqKSI [00:09:05] [02miraheze/mediawiki] 07dependabot[bot] 037a0d9b8 - Bump extensions/Comments from `d75fa77` to `c70f7d5` [00:09:06] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/Comments-c70f7d5 - 13https://git.io/vbL5b [00:09:08] [02mediawiki] 07dependabot[bot] opened pull request 03#1311: Bump extensions/Comments from `d75fa77` to `c70f7d5` - 13https://git.io/JqKSt [00:09:09] [02mediawiki] 07dependabot[bot] labeled pull request 03#1311: Bump extensions/Comments from `d75fa77` to `c70f7d5` - 13https://git.io/JqKSt [00:09:11] [02mediawiki] 07dependabot[bot] labeled pull request 03#1311: Bump extensions/Comments from `d75fa77` to `c70f7d5` - 13https://git.io/JqKSt [00:09:12] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/Editcount-b9a6080 [+0/-0/±1] 13https://git.io/JqKSO [00:09:14] [02miraheze/mediawiki] 07dependabot[bot] 0369e5925 - Bump extensions/Editcount from `a6599e6` to `b9a6080` [00:09:16] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/Editcount-b9a6080 - 13https://git.io/vbL5b [00:09:17] [02mediawiki] 07dependabot[bot] opened pull request 03#1312: Bump extensions/Editcount from `a6599e6` to `b9a6080` - 13https://git.io/JqKSs [00:09:18] [02mediawiki] 07dependabot[bot] labeled pull request 03#1312: Bump extensions/Editcount from `a6599e6` to `b9a6080` - 13https://git.io/JqKSs [00:09:20] [02mediawiki] 07dependabot[bot] labeled pull request 03#1312: Bump extensions/Editcount from `a6599e6` to `b9a6080` - 13https://git.io/JqKSs [00:09:21] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/Validator-1293482 [+0/-0/±1] 13https://git.io/JqKSc [00:09:23] [02miraheze/mediawiki] 07dependabot[bot] 038f287fb - Bump extensions/Validator from `48bd379` to `1293482` [00:09:24] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/Validator-1293482 - 13https://git.io/vbL5b [00:09:26] [02mediawiki] 07dependabot[bot] opened pull request 03#1313: Bump extensions/Validator from `48bd379` to `1293482` - 13https://git.io/JqKSW [00:09:27] [02mediawiki] 07dependabot[bot] labeled pull request 03#1313: Bump extensions/Validator from `48bd379` to `1293482` - 13https://git.io/JqKSW [00:09:29] [02mediawiki] 07dependabot[bot] labeled pull request 03#1313: Bump extensions/Validator from `48bd379` to `1293482` - 13https://git.io/JqKSW [00:09:30] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/CommentStreams-6fbe4c1 [+0/-0/±1] 13https://git.io/JqKSg [00:09:32] [02miraheze/mediawiki] 07dependabot[bot] 036d0212a - Bump extensions/CommentStreams from `22ebac2` to `6fbe4c1` [00:09:33] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/CommentStreams-6fbe4c1 - 13https://git.io/vbL5b [00:09:35] [02mediawiki] 07dependabot[bot] opened pull request 03#1314: Bump extensions/CommentStreams from `22ebac2` to `6fbe4c1` - 13https://git.io/JqKS2 [00:09:36] [02mediawiki] 07dependabot[bot] labeled pull request 03#1314: Bump extensions/CommentStreams from `22ebac2` to `6fbe4c1` - 13https://git.io/JqKS2 [00:09:38] [02mediawiki] 07dependabot[bot] labeled pull request 03#1314: Bump extensions/CommentStreams from `22ebac2` to `6fbe4c1` - 13https://git.io/JqKS2 [00:09:39] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/GlobalNewFiles-d831a1a [+0/-0/±1] 13https://git.io/JqKSo [00:09:41] [02miraheze/mediawiki] 07dependabot[bot] 03634b37a - Bump extensions/GlobalNewFiles from `027a20a` to `d831a1a` [00:09:42] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/GlobalNewFiles-d831a1a - 13https://git.io/vbL5b [00:09:44] [02mediawiki] 07dependabot[bot] opened pull request 03#1315: Bump extensions/GlobalNewFiles from `027a20a` to `d831a1a` - 13https://git.io/JqKS6 [00:09:45] [02mediawiki] 07dependabot[bot] labeled pull request 03#1315: Bump extensions/GlobalNewFiles from `027a20a` to `d831a1a` - 13https://git.io/JqKS6 [00:09:47] [02mediawiki] 07dependabot[bot] labeled pull request 03#1315: Bump extensions/GlobalNewFiles from `027a20a` to `d831a1a` - 13https://git.io/JqKS6 [00:09:48] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/Wikibase-14df0d2 [+0/-0/±1] 13https://git.io/JqKS9 [00:09:50] [02miraheze/mediawiki] 07dependabot[bot] 031834f2b - Bump extensions/Wikibase from `5ebc9d0` to `14df0d2` [00:09:51] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/Wikibase-14df0d2 - 13https://git.io/vbL5b [00:09:53] [02mediawiki] 07dependabot[bot] opened pull request 03#1316: Bump extensions/Wikibase from `5ebc9d0` to `14df0d2` - 13https://git.io/JqKSb [00:09:54] [02mediawiki] 07dependabot[bot] labeled pull request 03#1316: Bump extensions/Wikibase from `5ebc9d0` to `14df0d2` - 13https://git.io/JqKSb [00:09:56] [02mediawiki] 07dependabot[bot] labeled pull request 03#1316: Bump extensions/Wikibase from `5ebc9d0` to `14df0d2` - 13https://git.io/JqKSb [00:09:57] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/VEForAll-8c90b8d [+0/-0/±1] 13https://git.io/JqKSp [00:09:59] [02miraheze/mediawiki] 07dependabot[bot] 0341c6a9d - Bump extensions/VEForAll from `38290cd` to `8c90b8d` [00:10:00] [02mediawiki] 07dependabot[bot] opened pull request 03#1317: Bump extensions/VEForAll from `38290cd` to `8c90b8d` - 13https://git.io/JqKSh [00:10:02] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/VEForAll-8c90b8d - 13https://git.io/vbL5b [00:10:03] [02mediawiki] 07dependabot[bot] labeled pull request 03#1317: Bump extensions/VEForAll from `38290cd` to `8c90b8d` - 13https://git.io/JqKSh [00:10:05] [02mediawiki] 07dependabot[bot] labeled pull request 03#1317: Bump extensions/VEForAll from `38290cd` to `8c90b8d` - 13https://git.io/JqKSh [00:10:06] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/BlogPage-c4dbf5b [+0/-0/±1] 13https://git.io/JqK9U [00:10:08] [02miraheze/mediawiki] 07dependabot[bot] 032961bfb - Bump extensions/BlogPage from `7951ba8` to `c4dbf5b` [00:10:09] [02mediawiki] 07dependabot[bot] opened pull request 03#1318: Bump extensions/BlogPage from `7951ba8` to `c4dbf5b` - 13https://git.io/JqK9T [00:10:11] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/BlogPage-c4dbf5b - 13https://git.io/vbL5b [00:10:13] [02mediawiki] 07dependabot[bot] labeled pull request 03#1318: Bump extensions/BlogPage from `7951ba8` to `c4dbf5b` - 13https://git.io/JqK9T [00:10:14] [02mediawiki] 07dependabot[bot] labeled pull request 03#1318: Bump extensions/BlogPage from `7951ba8` to `c4dbf5b` - 13https://git.io/JqK9T [00:10:15] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/UniversalLanguageSelector-f31b0c9 [+0/-0/±1] 13https://git.io/JqK9Y [00:10:17] [02miraheze/mediawiki] 07dependabot[bot] 031e3eecc - Bump extensions/UniversalLanguageSelector from `0b92c67` to `f31b0c9` [00:10:18] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/UniversalLanguageSelector-f31b0c9 - 13https://git.io/vbL5b [00:10:20] [02mediawiki] 07dependabot[bot] opened pull request 03#1319: Bump extensions/UniversalLanguageSelector from `0b92c67` to `f31b0c9` - 13https://git.io/JqK93 [00:10:21] [02mediawiki] 07dependabot[bot] labeled pull request 03#1319: Bump extensions/UniversalLanguageSelector from `0b92c67` to `f31b0c9` - 13https://git.io/JqK93 [00:10:23] [02mediawiki] 07dependabot[bot] labeled pull request 03#1319: Bump extensions/UniversalLanguageSelector from `0b92c67` to `f31b0c9` - 13https://git.io/JqK93 [00:10:24] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/skins/Medik-8794a72 [+0/-0/±1] 13https://git.io/JqK9l [00:10:26] [02miraheze/mediawiki] 07dependabot[bot] 035e6ce72 - Bump skins/Medik from `921c3b8` to `8794a72` [00:10:27] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/skins/Medik-8794a72 - 13https://git.io/vbL5b [00:10:29] [02mediawiki] 07dependabot[bot] opened pull request 03#1320: Bump skins/Medik from `921c3b8` to `8794a72` - 13https://git.io/JqK98 [00:10:30] [02mediawiki] 07dependabot[bot] labeled pull request 03#1320: Bump skins/Medik from `921c3b8` to `8794a72` - 13https://git.io/JqK98 [00:10:32] [02mediawiki] 07dependabot[bot] labeled pull request 03#1320: Bump skins/Medik from `921c3b8` to `8794a72` - 13https://git.io/JqK98 [00:10:33] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/WikiSEO-c0579da [+0/-0/±1] 13https://git.io/JqK9E [00:10:35] [02miraheze/mediawiki] 07dependabot[bot] 035366bb5 - Bump extensions/WikiSEO from `b72a071` to `c0579da` [00:10:36] [02mediawiki] 07dependabot[bot] opened pull request 03#1321: Bump extensions/WikiSEO from `b72a071` to `c0579da` - 13https://git.io/JqK9g [00:10:38] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/WikiSEO-c0579da - 13https://git.io/vbL5b [00:10:39] [02mediawiki] 07dependabot[bot] labeled pull request 03#1321: Bump extensions/WikiSEO from `b72a071` to `c0579da` - 13https://git.io/JqK9g [00:10:41] [02mediawiki] 07dependabot[bot] labeled pull request 03#1321: Bump extensions/WikiSEO from `b72a071` to `c0579da` - 13https://git.io/JqK9g [00:10:42] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/GettingStarted-5ec1c75 [+0/-0/±1] 13https://git.io/JqK9i [00:10:44] [02miraheze/mediawiki] 07dependabot[bot] 037097c27 - Bump extensions/GettingStarted from `32cb951` to `5ec1c75` [00:10:45] [02mediawiki] 07dependabot[bot] opened pull request 03#1322: Bump extensions/GettingStarted from `32cb951` to `5ec1c75` - 13https://git.io/JqK9P [00:10:47] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/GettingStarted-5ec1c75 - 13https://git.io/vbL5b [00:10:48] [02mediawiki] 07dependabot[bot] labeled pull request 03#1322: Bump extensions/GettingStarted from `32cb951` to `5ec1c75` - 13https://git.io/JqK9P [00:10:50] [02mediawiki] 07dependabot[bot] labeled pull request 03#1322: Bump extensions/GettingStarted from `32cb951` to `5ec1c75` - 13https://git.io/JqK9P [00:10:51] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/RottenLinks-511fdfd [+0/-0/±1] 13https://git.io/JqK9y [00:10:53] [02miraheze/mediawiki] 07dependabot[bot] 035db27f6 - Bump extensions/RottenLinks from `1f4b396` to `511fdfd` [00:10:54] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/RottenLinks-511fdfd - 13https://git.io/vbL5b [00:10:56] [02mediawiki] 07dependabot[bot] opened pull request 03#1323: Bump extensions/RottenLinks from `1f4b396` to `511fdfd` - 13https://git.io/JqK99 [00:10:57] [02mediawiki] 07dependabot[bot] labeled pull request 03#1323: Bump extensions/RottenLinks from `1f4b396` to `511fdfd` - 13https://git.io/JqK99 [00:10:59] [02mediawiki] 07dependabot[bot] labeled pull request 03#1323: Bump extensions/RottenLinks from `1f4b396` to `511fdfd` - 13https://git.io/JqK99 [00:11:00] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/cldr-c03d2bb [+0/-0/±1] 13https://git.io/JqK9F [00:11:02] [02miraheze/mediawiki] 07dependabot[bot] 03c880750 - Bump extensions/cldr from `02f2895` to `c03d2bb` [00:11:03] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/cldr-c03d2bb - 13https://git.io/vbL5b [00:11:05] [02mediawiki] 07dependabot[bot] opened pull request 03#1324: Bump extensions/cldr from `02f2895` to `c03d2bb` - 13https://git.io/JqK9N [00:11:06] [02mediawiki] 07dependabot[bot] labeled pull request 03#1324: Bump extensions/cldr from `02f2895` to `c03d2bb` - 13https://git.io/JqK9N [00:11:08] [02mediawiki] 07dependabot[bot] labeled pull request 03#1324: Bump extensions/cldr from `02f2895` to `c03d2bb` - 13https://git.io/JqK9N [00:11:09] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/Cargo-3d811e2 [+0/-0/±1] 13https://git.io/JqK9p [00:11:11] [02miraheze/mediawiki] 07dependabot[bot] 033d3daef - Bump extensions/Cargo from `8fc3204` to `3d811e2` [00:11:12] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/Cargo-3d811e2 - 13https://git.io/vbL5b [00:11:14] [02mediawiki] 07dependabot[bot] opened pull request 03#1325: Bump extensions/Cargo from `8fc3204` to `3d811e2` - 13https://git.io/JqKHe [00:11:15] [02mediawiki] 07dependabot[bot] labeled pull request 03#1325: Bump extensions/Cargo from `8fc3204` to `3d811e2` - 13https://git.io/JqKHe [00:11:17] [02mediawiki] 07dependabot[bot] labeled pull request 03#1325: Bump extensions/Cargo from `8fc3204` to `3d811e2` - 13https://git.io/JqKHe [00:11:18] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/MediaWikiChat-ca65d0d [+0/-0/±1] 13https://git.io/JqKHI [00:11:20] [02miraheze/mediawiki] 07dependabot[bot] 03275d96a - Bump extensions/MediaWikiChat from `eadada7` to `ca65d0d` [00:11:21] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/MediaWikiChat-ca65d0d - 13https://git.io/vbL5b [00:11:23] [02mediawiki] 07dependabot[bot] opened pull request 03#1326: Bump extensions/MediaWikiChat from `eadada7` to `ca65d0d` - 13https://git.io/JqKHq [00:11:24] [02mediawiki] 07dependabot[bot] labeled pull request 03#1326: Bump extensions/MediaWikiChat from `eadada7` to `ca65d0d` - 13https://git.io/JqKHq [00:11:26] [02mediawiki] 07dependabot[bot] labeled pull request 03#1326: Bump extensions/MediaWikiChat from `eadada7` to `ca65d0d` - 13https://git.io/JqKHq [00:11:27] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/skins/Citizen-b68a2bd [+0/-0/±1] 13https://git.io/JqKHO [00:11:29] [02miraheze/mediawiki] 07dependabot[bot] 039a9e161 - Bump skins/Citizen from `09b399c` to `b68a2bd` [00:11:30] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/skins/Citizen-b68a2bd - 13https://git.io/vbL5b [00:11:32] [02mediawiki] 07dependabot[bot] opened pull request 03#1327: Bump skins/Citizen from `09b399c` to `b68a2bd` - 13https://git.io/JqKH3 [00:11:33] [02mediawiki] 07dependabot[bot] labeled pull request 03#1327: Bump skins/Citizen from `09b399c` to `b68a2bd` - 13https://git.io/JqKH3 [00:11:35] [02mediawiki] 07dependabot[bot] labeled pull request 03#1327: Bump skins/Citizen from `09b399c` to `b68a2bd` - 13https://git.io/JqKH3 [00:11:36] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/DataDump-cc6c400 [+0/-0/±1] 13https://git.io/JqKHW [00:11:38] [02miraheze/mediawiki] 07dependabot[bot] 03ca23bb0 - Bump extensions/DataDump from `dee406e` to `cc6c400` [00:11:39] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/DataDump-cc6c400 - 13https://git.io/vbL5b [00:11:41] [02mediawiki] 07dependabot[bot] opened pull request 03#1328: Bump extensions/DataDump from `dee406e` to `cc6c400` - 13https://git.io/JqKHl [00:11:42] [02mediawiki] 07dependabot[bot] labeled pull request 03#1328: Bump extensions/DataDump from `dee406e` to `cc6c400` - 13https://git.io/JqKHl [00:11:44] [02mediawiki] 07dependabot[bot] labeled pull request 03#1328: Bump extensions/DataDump from `dee406e` to `cc6c400` - 13https://git.io/JqKHl [00:11:45] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/Commentbox-5ecfa85 [+0/-0/±1] 13https://git.io/JqKHB [00:11:47] [02miraheze/mediawiki] 07dependabot[bot] 031c198eb - Bump extensions/Commentbox from `c816a9b` to `5ecfa85` [00:11:48] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/Commentbox-5ecfa85 - 13https://git.io/vbL5b [00:11:50] [02mediawiki] 07dependabot[bot] opened pull request 03#1329: Bump extensions/Commentbox from `c816a9b` to `5ecfa85` - 13https://git.io/JqKH0 [00:11:51] [02mediawiki] 07dependabot[bot] labeled pull request 03#1329: Bump extensions/Commentbox from `c816a9b` to `5ecfa85` - 13https://git.io/JqKH0 [00:11:53] [02mediawiki] 07dependabot[bot] labeled pull request 03#1329: Bump extensions/Commentbox from `c816a9b` to `5ecfa85` - 13https://git.io/JqKH0 [00:11:54] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/PageForms-08fbb02 [+0/-0/±1] 13https://git.io/JqKHa [00:11:56] [02miraheze/mediawiki] 07dependabot[bot] 035e519b6 - Bump extensions/PageForms from `ff424c0` to `08fbb02` [00:11:57] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/PageForms-08fbb02 - 13https://git.io/vbL5b [00:11:59] [02mediawiki] 07dependabot[bot] opened pull request 03#1330: Bump extensions/PageForms from `ff424c0` to `08fbb02` - 13https://git.io/JqKHV [00:12:00] [02mediawiki] 07dependabot[bot] labeled pull request 03#1330: Bump extensions/PageForms from `ff424c0` to `08fbb02` - 13https://git.io/JqKHV [00:12:02] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+1/-0/±0] 13https://git.io/JqKHw [00:12:03] [02miraheze/puppet] 07paladox 03844c21c - Create puppetserver.logrotate.conf [00:12:05] [02mediawiki] 07dependabot[bot] labeled pull request 03#1330: Bump extensions/PageForms from `ff424c0` to `08fbb02` - 13https://git.io/JqKHV [00:12:06] [02puppet] 07paladox synchronize pull request 03#1707: puppetserver/puppetdb: use logrotate - 13https://git.io/JqK0s [00:12:08] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/extensions/WikiDiscover-a258459 [+0/-0/±1] 13https://git.io/JqKHP [00:12:09] [02miraheze/mediawiki] 07dependabot[bot] 03a5a26f3 - Bump extensions/WikiDiscover from `b0051c5` to `a258459` [00:12:11] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/extensions/WikiDiscover-a258459 - 13https://git.io/vbL5b [00:12:12] [02mediawiki] 07dependabot[bot] opened pull request 03#1331: Bump extensions/WikiDiscover from `b0051c5` to `a258459` - 13https://git.io/JqKH1 [00:12:14] [02mediawiki] 07dependabot[bot] labeled pull request 03#1331: Bump extensions/WikiDiscover from `b0051c5` to `a258459` - 13https://git.io/JqKH1 [00:12:15] [02mediawiki] 07dependabot[bot] labeled pull request 03#1331: Bump extensions/WikiDiscover from `b0051c5` to `a258459` - 13https://git.io/JqKH1 [00:12:17] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_35/skins/Cosmos-b00f5ec [+0/-0/±1] 13https://git.io/JqKHS [00:12:18] [02miraheze/mediawiki] 07dependabot[bot] 030c3d8bd - Bump skins/Cosmos from `7ea0eb3` to `b00f5ec` [00:12:20] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_35/skins/Cosmos-b00f5ec - 13https://git.io/vbL5b [00:12:21] [02mediawiki] 07dependabot[bot] opened pull request 03#1332: Bump skins/Cosmos from `7ea0eb3` to `b00f5ec` - 13https://git.io/JqKHH [00:12:23] [02mediawiki] 07dependabot[bot] labeled pull request 03#1332: Bump skins/Cosmos from `7ea0eb3` to `b00f5ec` - 13https://git.io/JqKHH [00:12:24] [02mediawiki] 07dependabot[bot] labeled pull request 03#1332: Bump skins/Cosmos from `7ea0eb3` to `b00f5ec` - 13https://git.io/JqKHH [00:12:26] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JqK7v [00:12:27] [02miraheze/puppet] 07paladox 03f2a0c27 - Update client.pp [00:12:29] [02puppet] 07paladox synchronize pull request 03#1707: puppetserver/puppetdb: use logrotate - 13https://git.io/JqK0s [00:12:30] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+1/-0/±0] 13https://git.io/JqK5e [00:12:32] [02miraheze/puppet] 07paladox 039c39e42 - Create puppetdb.logrotate.conf [00:12:33] [02puppet] 07paladox synchronize pull request 03#1707: puppetserver/puppetdb: use logrotate - 13https://git.io/JqK0s [00:12:47] [02puppet] 07paladox closed pull request 03#1707: puppetserver/puppetdb: use logrotate - 13https://git.io/JqK0s [00:12:49] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+2/-0/±5] 13https://git.io/JqKxB [00:12:51] [02miraheze/puppet] 07paladox 035ed1673 - puppetserver/puppetdb: use logrotate (#1707) [00:12:52] [02puppet] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vbiAS [00:12:54] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-2 [00:20:06] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/Jq6IJ [00:20:07] [02miraheze/puppet] 07paladox 03fccb275 - Fix class being used in logback [00:20:09] [02puppet] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vbiAS [00:20:11] [02puppet] 07paladox opened pull request 03#1708: Fix class being used in logback - 13https://git.io/Jq6It [00:20:20] [02puppet] 07paladox edited pull request 03#1708: puppetserver/puppetdb: Fix class being used in logback - 13https://git.io/Jq6It [00:20:36] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/Jq6IH [00:20:37] [02miraheze/puppet] 07paladox 0384ee89c - Update puppetdb_logback.xml [00:20:39] [02puppet] 07paladox synchronize pull request 03#1708: puppetserver/puppetdb: Fix class being used in logback - 13https://git.io/Jq6It [00:20:47] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/Jq6Ij [00:20:49] [02miraheze/puppet] 07paladox 031341909 - Update puppetserver-request-logging.xml [00:20:51] [02puppet] 07paladox synchronize pull request 03#1708: puppetserver/puppetdb: Fix class being used in logback - 13https://git.io/Jq6It [00:21:01] [02puppet] 07paladox closed pull request 03#1708: puppetserver/puppetdb: Fix class being used in logback - 13https://git.io/Jq6It [00:21:02] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/Jq6Ll [00:21:04] [02miraheze/puppet] 07paladox 030ed3b8b - puppetserver/puppetdb: Fix class being used in logback (#1708) [00:22:00] [02mediawiki] 07paladox closed pull request 03#1331: Bump extensions/WikiDiscover from `b0051c5` to `a258459` - 13https://git.io/JqKH1 [00:22:01] [02miraheze/mediawiki] 07paladox pushed 032 commits to 03REL1_35 [+0/-0/±2] 13https://git.io/Jq6tQ [00:22:03] [02miraheze/mediawiki] 07paladox 03a8d7c8f - Merge pull request #1331 from miraheze/dependabot/submodules/REL1_35/extensions/WikiDiscover-a258459 [00:22:04] [02miraheze/mediawiki] 07paladox deleted branch 03dependabot/submodules/REL1_35/extensions/WikiDiscover-a258459 [00:22:06] [02mediawiki] 07paladox deleted branch 03dependabot/submodules/REL1_35/extensions/WikiDiscover-a258459 - 13https://git.io/vbL5b [00:24:20] [02puppet] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vbiAS [00:24:22] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-2 [00:24:59] PROBLEM - cp12 Puppet on cp12 is CRITICAL: CRITICAL: Puppet has 113 failures. Last run 2 minutes ago with 113 failures. Failed resources (up to 3 shown): File[en.petrawiki.org_private],File[de.gyaanipedia.com],File[de.gyaanipedia.com_private],File[www.bluevies.org] [00:26:01] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/Jq6GW [00:26:02] [02miraheze/puppet] 07paladox 0360a130a - gluster: Add logrotate [00:26:03] [02puppet] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vbiAS [00:26:05] [02puppet] 07paladox opened pull request 03#1709: gluster: Add logrotate - 13https://git.io/Jq6GB [00:27:13] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+1/-0/±0] 13https://git.io/Jq6nL [00:27:14] [02miraheze/puppet] 07paladox 03414dfb9 - Create glusterfs-common.logrotate.conf [00:27:16] [02puppet] 07paladox synchronize pull request 03#1709: gluster: Add logrotate - 13https://git.io/Jq6GB [00:27:47] [02puppet] 07paladox edited pull request 03#1709: gluster: Add logrotate - 13https://git.io/Jq6GB [00:28:11] [02puppet] 07paladox closed pull request 03#1709: gluster: Add logrotate - 13https://git.io/Jq6GB [00:28:13] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/Jq6cW [00:28:14] [02miraheze/puppet] 07paladox 036d6911e - gluster: Add logrotate (#1709) [00:28:16] [02puppet] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vbiAS [00:28:17] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-2 [00:44:22] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 2.50, 1.71, 1.12 [00:46:19] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 0.93, 1.33, 1.05 [00:53:18] RECOVERY - cp12 Puppet on cp12 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:55:16] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jq6pJ [00:55:18] [02miraheze/mw-config] 07paladox 03dea5e43 - Increase memcache timeout to 500ms [00:56:14] miraheze/mw-config - paladox the build passed. [00:57:23] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.69, 3.67, 4.00 [00:59:25] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 5.08, 4.07, 4.10 [00:59:28] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqifC [00:59:29] [02miraheze/mw-config] 07paladox 0354e28ac - Test memcached persistent [01:00:32] miraheze/mw-config - paladox the build passed. [01:03:45] well, I don't know if this is related to all these commits, but When I try to create a Cargo table it gives me an error ("Fatal exception of type 'MWException' ") [01:03:45] https://cdn.discordapp.com/attachments/808001911868489748/820824555886608404/unknown.png [01:03:58] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqiLb [01:04:00] [02miraheze/mw-config] 07paladox 0355ffc40 - Revert "Test memcached persistent" [01:04:59] miraheze/mw-config - paladox the build passed. [01:05:34] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/Jqimk [01:05:36] [02miraheze/puppet] 07paladox 03d3e575f - mediawiki: Use nutcracker for memcached [01:05:37] [02puppet] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vbiAS [01:05:40] [02puppet] 07paladox opened pull request 03#1710: mediawiki: Use nutcracker for memcached - 13https://git.io/Jqims [01:06:29] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JqiYK [01:06:31] [02miraheze/puppet] 07paladox 03a0c060b - Update init.pp [01:06:32] [02puppet] 07paladox synchronize pull request 03#1710: mediawiki: Use nutcracker for memcached - 13https://git.io/Jqims [01:10:55] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/Jqinu [01:10:57] [02miraheze/puppet] 07paladox 034752dcb - Update nutcracker.yml.erb [01:10:58] [02puppet] 07paladox synchronize pull request 03#1710: mediawiki: Use nutcracker for memcached - 13https://git.io/Jqims [01:11:22] [02puppet] 07paladox closed pull request 03#1710: mediawiki: Use nutcracker for memcached - 13https://git.io/Jqims [01:11:23] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/Jqicq [01:11:25] [02miraheze/puppet] 07paladox 035655456 - mediawiki: Use nutcracker for memcached (#1710) [01:11:26] [02puppet] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vbiAS [01:11:28] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-2 [01:16:38] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqiBg [01:16:39] [02miraheze/mw-config] 07paladox 037737ff3 - Switch to nutcracker for memcached [01:17:39] miraheze/mw-config - paladox the build passed. [01:18:06] PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 226s [01:18:25] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JqiEU [01:18:27] [02miraheze/puppet] 07paladox 035394c16 - nutcracker: Remove systemd override [01:18:29] [02puppet] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vbiAS [01:18:30] [02puppet] 07paladox opened pull request 03#1711: nutcracker: Remove systemd override - 13https://git.io/JqiEI [01:18:39] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-1/±0] 13https://git.io/JqiE2 [01:18:41] [02miraheze/puppet] 07paladox 0318ada58 - Delete nutcracker-systemd-override.conf.erb [01:18:42] [02puppet] 07paladox synchronize pull request 03#1711: nutcracker: Remove systemd override - 13https://git.io/JqiEI [01:18:49] [02puppet] 07paladox closed pull request 03#1711: nutcracker: Remove systemd override - 13https://git.io/JqiEI [01:18:50] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-1/±1] 13https://git.io/JqiEy [01:18:52] [02miraheze/puppet] 07paladox 03aded47f - nutcracker: Remove systemd override (#1711) [01:18:54] [02puppet] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vbiAS [01:18:55] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-2 [01:20:07] RECOVERY - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 95s [01:21:33] !log rm /etc/systemd/system/nutcracker.service.d/puppet-override.conf ; systemctl daemon-reload ; service nutcracker restart on mw* and jobrunner* [01:21:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:37:31] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.69, 6.89, 6.06 [01:39:31] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.02, 6.91, 6.17 [01:39:31] PROBLEM - jobrunner4 Current Load on jobrunner4 is WARNING: WARNING - load average: 5.69, 4.49, 2.97 [01:41:31] RECOVERY - jobrunner4 Current Load on jobrunner4 is OK: OK - load average: 4.48, 4.36, 3.11 [01:43:30] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.13, 7.34, 6.49 [01:45:30] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.46, 6.57, 6.31 [01:47:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.03, 3.67, 4.00 [01:51:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.08, 3.94, 4.03 [02:34:17] PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 304s [02:36:12] PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is WARNING: MariaDB replication - both - WARNING - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 109s [02:38:07] RECOVERY - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 0s [02:53:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.74, 3.73, 4.00 [02:55:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.48, 3.88, 4.02 [03:00:06] PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 247s [03:02:07] PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is WARNING: MariaDB replication - both - WARNING - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 159s [03:04:06] RECOVERY - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 1s [03:46:24] PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 314s [03:49:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.15, 3.58, 3.91 [03:51:23] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.03, 3.67, 3.90 [03:58:06] RECOVERY - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 0s [04:03:23] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.42, 3.94, 4.00 [04:07:20] [02miraheze/IncidentReporting] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/Jq1rv [04:07:21] [02miraheze/IncidentReporting] 07Universal-Omega 03815e29d - Update SpecialIncidentReports.php [04:07:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.11, 4.05, 4.03 [04:07:23] [02IncidentReporting] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/fh5YJ [04:07:32] [02IncidentReporting] 07Universal-Omega opened pull request 03#17: T4420: Introduce stats for IncidentReports (WIP) - 13https://git.io/Jq1rG [04:08:27] miraheze/IncidentReporting - Universal-Omega the build passed. [04:16:07] PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is WARNING: MariaDB replication - both - WARNING - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 165s [04:18:06] RECOVERY - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 0s [04:20:45] PROBLEM - ping4 on cp3 is WARNING: PING WARNING - Packet loss = 0%, RTA = 336.38 ms [04:21:23] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.01, 3.61, 3.89 [04:31:23] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.24, 3.78, 3.78 [04:33:23] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.54, 3.59, 3.70 [04:35:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.73, 3.98, 3.83 [04:36:45] RECOVERY - ping4 on cp3 is OK: PING OK - Packet loss = 0%, RTA = 253.37 ms [04:39:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.67, 3.99, 3.89 [04:41:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.53, 4.23, 3.99 [04:45:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.04, 3.85, 3.91 [04:49:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.35, 3.88, 3.89 [04:55:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.47, 3.91, 3.94 [04:59:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.54, 4.16, 4.02 [05:01:07] [02mediawiki] 07Universal-Omega closed pull request 03#1328: Bump extensions/DataDump from `dee406e` to `cc6c400` - 13https://git.io/JqKHl [05:01:09] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JqMEP [05:01:10] [02miraheze/mediawiki] 07dependabot[bot] 03c9dc105 - Bump extensions/DataDump from `dee406e` to `cc6c400` (#1328) [05:01:12] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/DataDump-cc6c400 [05:01:14] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/DataDump-cc6c400 - 13https://git.io/vbL5b [05:01:28] [02mediawiki] 07Universal-Omega closed pull request 03#1332: Bump skins/Cosmos from `7ea0eb3` to `b00f5ec` - 13https://git.io/JqKHH [05:01:29] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JqMuT [05:01:31] [02miraheze/mediawiki] 07dependabot[bot] 03feabed2 - Bump skins/Cosmos from `7ea0eb3` to `b00f5ec` (#1332) [05:01:32] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/skins/Cosmos-b00f5ec - 13https://git.io/vbL5b [05:01:34] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/skins/Cosmos-b00f5ec [05:02:34] [02mediawiki] 07Universal-Omega closed pull request 03#1329: Bump extensions/Commentbox from `c816a9b` to `5ecfa85` - 13https://git.io/JqKH0 [05:02:35] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JqMz8 [05:02:37] [02miraheze/mediawiki] 07dependabot[bot] 03d57636f - Bump extensions/Commentbox from `c816a9b` to `5ecfa85` (#1329) [05:02:38] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/Commentbox-5ecfa85 - 13https://git.io/vbL5b [05:02:40] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/Commentbox-5ecfa85 [05:03:04] [02mediawiki] 07Universal-Omega closed pull request 03#1323: Bump extensions/RottenLinks from `1f4b396` to `511fdfd` - 13https://git.io/JqK99 [05:03:05] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JqMz7 [05:03:07] [02miraheze/mediawiki] 07dependabot[bot] 03c1149bd - Bump extensions/RottenLinks from `1f4b396` to `511fdfd` (#1323) [05:03:09] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/RottenLinks-511fdfd [05:03:10] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/RottenLinks-511fdfd - 13https://git.io/vbL5b [05:03:54] [02mediawiki] 07Universal-Omega closed pull request 03#1315: Bump extensions/GlobalNewFiles from `027a20a` to `d831a1a` - 13https://git.io/JqKS6 [05:03:55] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JqM2I [05:03:57] [02miraheze/mediawiki] 07dependabot[bot] 033125ca4 - Bump extensions/GlobalNewFiles from `027a20a` to `d831a1a` (#1315) [05:03:58] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/GlobalNewFiles-d831a1a - 13https://git.io/vbL5b [05:04:00] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/GlobalNewFiles-d831a1a [05:04:22] [02mediawiki] 07Universal-Omega closed pull request 03#1309: Bump extensions/MirahezeMagic from `d9e7dd4` to `f4c85c0` - 13https://git.io/JqKyN [05:04:23] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JqM2D [05:04:25] [02miraheze/mediawiki] 07dependabot[bot] 035c0e901 - Bump extensions/MirahezeMagic from `d9e7dd4` to `f4c85c0` (#1309) [05:04:27] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/MirahezeMagic-f4c85c0 - 13https://git.io/vbL5b [05:04:28] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/MirahezeMagic-f4c85c0 [05:04:46] [02mediawiki] 07Universal-Omega closed pull request 03#1306: Bump extensions/CreateWiki from `f8eec3f` to `a2c05b2` - 13https://git.io/JqKyE [05:04:48] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JqMaO [05:04:49] [02miraheze/mediawiki] 07dependabot[bot] 0357d89d6 - Bump extensions/CreateWiki from `f8eec3f` to `a2c05b2` (#1306) [05:04:51] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/CreateWiki-a2c05b2 - 13https://git.io/vbL5b [05:04:52] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/CreateWiki-a2c05b2 [05:05:20] [02mediawiki] 07Universal-Omega closed pull request 03#1299: Bump extensions/ManageWiki from `9557f3f` to `ce4f5ee` - 13https://git.io/JqKDw [05:05:22] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JqMaN [05:05:23] [02miraheze/mediawiki] 07dependabot[bot] 03f053063 - Bump extensions/ManageWiki from `9557f3f` to `ce4f5ee` (#1299) [05:05:25] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/ManageWiki-ce4f5ee - 13https://git.io/vbL5b [05:05:26] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/ManageWiki-ce4f5ee [05:05:58] [02mediawiki] 07Universal-Omega closed pull request 03#1284: Bump extensions/MatomoAnalytics from `411142e` to `aacdf8c` - 13https://git.io/JqK1l [05:05:59] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JqMVr [05:06:01] [02miraheze/mediawiki] 07dependabot[bot] 032aae5d9 - Bump extensions/MatomoAnalytics from `411142e` to `aacdf8c` (#1284) [05:06:02] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/MatomoAnalytics-aacdf8c [05:06:04] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/MatomoAnalytics-aacdf8c - 13https://git.io/vbL5b [05:07:07] [02mediawiki] 07Universal-Omega closed pull request 03#1300: Bump extensions/IncidentReporting from `4c85e3f` to `633ceae` - 13https://git.io/JqKDX [05:07:09] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JqMr3 [05:07:10] [02miraheze/mediawiki] 07dependabot[bot] 03b972431 - Bump extensions/IncidentReporting from `4c85e3f` to `633ceae` (#1300) [05:07:12] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/IncidentReporting-633ceae [05:07:13] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/IncidentReporting-633ceae - 13https://git.io/vbL5b [05:08:25] [02mediawiki] 07Universal-Omega closed pull request 03#1313: Bump extensions/Validator from `48bd379` to `1293482` - 13https://git.io/JqKSW [05:08:27] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JqMoA [05:08:28] [02miraheze/mediawiki] 07dependabot[bot] 037cdf024 - Bump extensions/Validator from `48bd379` to `1293482` (#1313) [05:08:30] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/Validator-1293482 [05:08:31] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/Validator-1293482 - 13https://git.io/vbL5b [05:10:21] PROBLEM - jobrunner4 Puppet on jobrunner4 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [05:37:44] PROBLEM - mw9 Puppet on mw9 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [05:38:06] PROBLEM - mw10 Puppet on mw10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [05:38:41] PROBLEM - test3 Puppet on test3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 4 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [05:39:17] PROBLEM - mw11 Puppet on mw11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [05:39:47] PROBLEM - mw8 Puppet on mw8 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [05:39:58] PROBLEM - jobrunner3 Puppet on jobrunner3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [05:51:23] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.79, 3.76, 3.97 [06:01:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.64, 3.97, 3.91 [06:03:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.85, 3.81, 3.85 [06:07:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.60, 4.00, 3.91 [06:13:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.50, 3.95, 3.95 [06:17:24] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.98, 4.32, 4.08 [06:20:10] RECOVERY - dbbackup2 APT on dbbackup2 is OK: APT OK: 11 packages available for upgrade (0 critical updates). [06:21:47] RECOVERY - dbbackup1 APT on dbbackup1 is OK: APT OK: 11 packages available for upgrade (0 critical updates). [06:40:15] [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqDhZ [06:40:16] [02miraheze/CreateWiki] 07Universal-Omega 03a6c66db - Fix case [06:41:16] miraheze/CreateWiki - Universal-Omega the build passed. [06:41:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.23, 3.54, 3.95 [06:42:39] PROBLEM - ping6 on cp3 is WARNING: PING WARNING - Packet loss = 0%, RTA = 344.10 ms [06:45:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.45, 3.91, 4.01 [06:49:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.41, 3.74, 3.93 [06:53:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.23, 3.73, 3.86 [06:56:56] [02miraheze/IncidentReporting] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/Jqyqp [06:56:58] [02miraheze/IncidentReporting] 07Universal-Omega 03e1a2a61 - Update SpecialIncidentReports.php [06:56:59] [02IncidentReporting] 07Universal-Omega synchronize pull request 03#17: T4420: Introduce stats for IncidentReports (WIP) - 13https://git.io/Jq1rG [06:57:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.99, 3.87, 3.89 [06:58:01] miraheze/IncidentReporting - Universal-Omega the build passed. [06:58:05] [02miraheze/IncidentReporting] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqyYv [06:58:07] [02miraheze/IncidentReporting] 07Universal-Omega 0360b58c3 - Update CHANGELOG [06:58:38] [02miraheze/IncidentReporting] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqyYu [06:58:39] [02miraheze/IncidentReporting] 07Universal-Omega 0340245ba - Wrong branch [06:59:01] miraheze/IncidentReporting - Universal-Omega the build passed. [06:59:13] [02miraheze/IncidentReporting] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JqyOU [06:59:15] [02miraheze/IncidentReporting] 07Universal-Omega 03736f8e0 - Update CHANGELOG [06:59:16] [02IncidentReporting] 07Universal-Omega synchronize pull request 03#17: T4420: Introduce stats for IncidentReports (WIP) - 13https://git.io/Jq1rG [06:59:36] miraheze/IncidentReporting - Universal-Omega the build passed. [07:00:16] miraheze/IncidentReporting - Universal-Omega the build passed. [07:00:41] [02IncidentReporting] 07Universal-Omega deleted branch 03DBConnRef - 13https://git.io/fh5YJ [07:00:43] [02miraheze/IncidentReporting] 07Universal-Omega deleted branch 03DBConnRef [07:02:15] [02miraheze/IncidentReporting] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JqysB [07:02:17] [02miraheze/IncidentReporting] 07Universal-Omega 03847a0fe - Update extension.json [07:02:18] [02IncidentReporting] 07Universal-Omega synchronize pull request 03#17: T4420: Introduce stats for IncidentReports (WIP) - 13https://git.io/Jq1rG [07:03:16] miraheze/IncidentReporting - Universal-Omega the build passed. [07:09:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.03, 3.78, 3.77 [07:11:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.82, 3.76, 3.76 [07:13:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.02, 3.82, 3.78 [07:15:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.92, 3.80, 3.77 [07:19:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.37, 3.79, 3.75 [07:21:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.22, 3.61, 3.69 [07:23:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.20, 3.82, 3.76 [07:25:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.39, 3.67, 3.71 [07:29:24] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.12, 3.84, 3.77 [07:31:23] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.73, 3.86, 3.79 [07:33:23] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.45, 4.05, 3.87 [07:37:23] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.38, 3.97, 3.90 [07:39:23] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.29, 4.10, 3.96 [07:41:23] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.72, 3.92, 3.91 [07:53:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.12, 3.85, 3.83 [07:57:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.66, 3.89, 3.87 [07:59:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.59, 4.07, 3.94 [08:05:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.03, 3.66, 3.83 [08:09:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.33, 3.91, 3.88 [08:15:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.39, 3.98, 3.98 [08:29:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.21, 3.82, 3.84 [08:55:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.20, 3.62, 3.95 [08:59:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.67, 4.20, 4.10 [09:06:08] PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is WARNING: MariaDB replication - both - WARNING - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 137s [09:07:52] PROBLEM - dbbackup2 Current Load on dbbackup2 is WARNING: WARNING - load average: 3.94, 3.31, 1.84 [09:09:52] RECOVERY - dbbackup2 Current Load on dbbackup2 is OK: OK - load average: 2.56, 3.21, 2.00 [09:10:08] RECOVERY - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 0s [09:49:23] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.57, 3.67, 3.97 [09:53:23] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.36, 4.11, 4.08 [10:11:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.47, 3.72, 3.96 [10:13:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.58, 4.02, 4.04 [10:42:42] PROBLEM - dbbackup2 Current Load on dbbackup2 is CRITICAL: CRITICAL - load average: 4.45, 3.01, 1.98 [10:44:40] RECOVERY - dbbackup2 Current Load on dbbackup2 is OK: OK - load average: 1.79, 2.64, 1.98 [11:07:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.40, 3.60, 3.99 [11:11:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.06, 3.80, 3.98 [11:11:44] PROBLEM - ping6 on cp3 is CRITICAL: PING CRITICAL - Packet loss = 0%, RTA = 350.48 ms [11:13:47] PROBLEM - ping6 on cp3 is WARNING: PING WARNING - Packet loss = 0%, RTA = 345.91 ms [11:46:00] PROBLEM - wiki.qadrishattari.xyz - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.qadrishattari.xyz could not be found [11:46:02] PROBLEM - miraheze.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for miraheze.com could not be found [11:46:02] PROBLEM - marinebiodiversitymatrix.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for marinebiodiversitymatrix.org could not be found [11:46:04] PROBLEM - dariawiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for dariawiki.org could not be found [11:46:05] PROBLEM - podpedia.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for podpedia.org could not be found [11:46:06] PROBLEM - hi.gyaanipedia.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for hi.gyaanipedia.com could not be found [11:49:23] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 2.92, 3.58, 3.94 [11:52:53] RECOVERY - wiki.qadrishattari.xyz - reverse DNS on sslhost is OK: rDNS OK - wiki.qadrishattari.xyz reverse DNS resolves to cp11.miraheze.org [11:52:56] RECOVERY - miraheze.com - reverse DNS on sslhost is OK: rDNS OK - miraheze.com reverse DNS resolves to cp10.miraheze.org [11:52:56] RECOVERY - marinebiodiversitymatrix.org - reverse DNS on sslhost is OK: rDNS OK - marinebiodiversitymatrix.org reverse DNS resolves to cp11.miraheze.org [11:52:59] RECOVERY - dariawiki.org - reverse DNS on sslhost is OK: rDNS OK - dariawiki.org reverse DNS resolves to cp11.miraheze.org [11:53:01] RECOVERY - podpedia.org - reverse DNS on sslhost is OK: rDNS OK - podpedia.org reverse DNS resolves to cp11.miraheze.org [11:53:03] RECOVERY - hi.gyaanipedia.com - reverse DNS on sslhost is OK: rDNS OK - hi.gyaanipedia.com reverse DNS resolves to cp10.miraheze.org [11:53:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.01, 3.71, 3.90 [11:55:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 2.76, 3.38, 3.76 [11:59:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 5.23, 4.14, 3.96 [12:21:36] paladox: any clue regarding https://phabricator.miraheze.org/T6970 ? [12:21:37] [ ⚓ T6970 Troubleshooting a 502 Bad Gateway Error ] - phabricator.miraheze.org [12:39:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.25, 3.73, 3.95 [12:41:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.12, 3.94, 4.00 [12:43:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.80, 3.87, 3.97 [12:45:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.03, 3.88, 3.96 [12:47:23] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.46, 3.67, 3.87 [12:49:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.38, 3.85, 3.91 [12:51:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.93, 3.86, 3.90 [12:55:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.74, 4.13, 3.98 [12:56:45] Reception123: what is the wiki? [12:58:30] paladox: tepwiki [12:58:47] it seems like even after I removed the custom domain from ManageWiki it's not working and now saying 404 [13:00:03] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqQZc [13:00:05] [02miraheze/mw-config] 07paladox 03ef7c78a - Revert "Revert "Switch to MemcachedPhpBagOStuff from MemcachedPeclBagOStuff"" [13:01:04] miraheze/mw-config - paladox the build passed. [13:01:39] seems to work now Reception123 since reverting ^ [13:01:52] PROBLEM - dbbackup2 Current Load on dbbackup2 is WARNING: WARNING - load average: 3.63, 2.65, 1.83 [13:03:46] paladox: accessing tep.miraheze.org doesn't work for me :( [13:03:47] RECOVERY - mw8 Puppet on mw8 is OK: OK: Puppet is currently enabled, last run 20 seconds ago with 0 failures [13:03:52] RECOVERY - dbbackup2 Current Load on dbbackup2 is OK: OK - load average: 1.33, 2.19, 1.77 [13:03:57] RECOVERY - jobrunner3 Puppet on jobrunner3 is OK: OK: Puppet is currently enabled, last run 29 seconds ago with 0 failures [13:04:05] RECOVERY - mw10 Puppet on mw10 is OK: OK: Puppet is currently enabled, last run 19 seconds ago with 0 failures [13:04:11] Reception123 https://tep.miraheze.org/wiki/Main works for me [13:04:12] [ Main - TEPwiki ] - tep.miraheze.org [13:04:19] RECOVERY - jobrunner4 Puppet on jobrunner4 is OK: OK: Puppet is currently enabled, last run 28 seconds ago with 0 failures [13:04:27] and works on my phone. (which recieved the 502 before i reverted the aboce) [13:04:58] RECOVERY - mw9 Puppet on mw9 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:05:15] RECOVERY - mw11 Puppet on mw11 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:09:58] If that changes caused a user facing impact, we would need an incident report Paladox please [13:10:23] ok [13:12:40] JohnLewis i cannot seem to create a IR on https://meta.miraheze.org/wiki/Special:IncidentReports. It doesn't say an error but it returns back to https://meta.miraheze.org/wiki/Special:IncidentReportswhen i click create report [13:12:41] [ Incident Reports - Miraheze Meta ] - meta.miraheze.org [13:12:42] [ No such special page - Miraheze Meta ] - meta.miraheze.org [13:14:12] Works for me [13:14:47] oh i need to log out and back in [13:17:20] works now yeah [13:17:32] will restore the custom domain then [13:19:06] RECOVERY - ping6 on cp3 is OK: PING OK - Packet loss = 0%, RTA = 267.82 ms [13:21:51] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqQa8 [13:21:53] [02miraheze/mw-config] 07paladox 032aaf167 - Revert "Switch to nutcracker for memcached" [13:22:53] miraheze/mw-config - paladox the build passed. [13:24:21] paladox: there seem to be login issues now :( [13:24:30] I can't log in and two others have reported too on Discord [13:24:35] I got No active login attempt is in progress for your session. [13:24:46] and before it said my auth token was invalid [13:24:47] oh, logins should work as of https://git.io/JqQa8 [13:24:49] [ Comparing ef7c78adbfe8...2aaf16747cca · miraheze/mw-config · GitHub ] - git.io [13:24:58] ok, thanks [13:24:59] I've been forced logged out when I tried creating 2 new additional templates on the Template Wiki. [13:25:13] Luckily, I was able to log in again. [13:36:41] I can log in now, thanks [13:53:10] @Lake: is cargo still broke? [13:53:43] let me check [13:54:41] RhinosF1: yes, at least this one: https://pokemundo.miraheze.org/wiki/Especial:CargoTables/Moves [13:54:42] [ Erro interno - Pokémundo ] - pokemundo.miraheze.org [13:54:50] (comes from this template: https://pokemundo.miraheze.org/wiki/Predefini%C3%A7%C3%A3o:Movebox) [13:54:51] [ Predefinição:Movebox - Pokémundo ] - pokemundo.miraheze.org [13:55:05] Will look later [13:55:30] I thought initially it might be something wrong with the way I declared the table, but I couldn't find anything that could cause that [14:05:24] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.64, 3.86, 3.99 [14:06:54] paladox: https://phabricator.wikimedia.org/T277433#6913207 [14:06:55] [ ⚓ T277433 InstantCommons can render a wiki completely unavailable during an outage. ] - phabricator.wikimedia.org [14:07:12] I assume that's correct and we have it using curl [14:17:24] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.76, 4.14, 4.03 [14:30:02] PROBLEM - ping6 on cp3 is WARNING: PING WARNING - Packet loss = 0%, RTA = 336.14 ms [14:32:36] Voidwalker hi, around? [14:33:15] paladox, yup [14:33:37] Voidwalker i'm wondering if you could disable InstantCommon on https://login.miraheze.org/wiki/Special:ManageWiki/settings#mw-section-media please? [14:33:39] [ Manage this wiki's additional settings - Miraheze Login Wiki ] - login.miraheze.org [14:34:54] [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/Jq7gn [14:34:55] [02miraheze/dns] 07Reception123 038eba7b4 - add wiki.mlpwiki.net zone [14:35:00] oh? any particular reason? just curious [14:35:37] paladox: hang on. [14:35:43] Because I bet many images use it [14:35:57] Voidwalker we don't need it on loginwiki. [14:36:02] Voidwalker: it caused an outage because loginwiki is used for health checks [14:36:08] paladox: has that been verified? [14:36:12] Like no images use it [14:36:16] oh, no [14:36:35] Then Voidwalker please don't yet [14:36:39] We need to do that first [14:37:48] paladox: it's happened twice in 18 months. I don't think there's need to rush revising how health checks work. [14:38:13] RECOVERY - ping6 on cp3 is OK: PING OK - Packet loss = 0%, RTA = 269.48 ms [14:38:21] well actually wikis would just backup the queue if they are slow [14:38:35] so really there's nothing we can do unless we change how we healthcheck [14:39:21] [02miraheze/ssl] 07Reception123 pushed 033 commits to 03master [+3/-0/±3] 13https://git.io/Jq7wk [14:39:22] [02miraheze/ssl] 07Reception123 03f68be88 - add wiki.yumeka.icu cert [14:39:23] [02miraheze/ssl] 07Reception123 03846360a - add civwiki.org cert [14:39:25] [02miraheze/ssl] 07Reception123 038e389cd - add wiki.mlpwiki.net cert [14:39:31] RhinosF1 so then you can decline the task, i don't see how upstream can resolve that. [14:39:42] paladox: well the other option is to create an inactiveexempt wiki that's got anything that uses IC off [14:39:50] I'm chatting with tgr [14:40:01] I think it's to do with the http timeout [14:40:10] I mean IC brings wikis down is true [14:40:19] looks like many of the files listed at https://login.miraheze.org/wiki/Special:WantedFiles are loaded using one form of commons or another [14:40:21] [ Wanted files - Miraheze Login Wiki ] - login.miraheze.org [14:40:26] The fact non-IC wikis came down is health checks [14:40:36] IC wikis by itself is an upstream issue [14:40:53] paladox: what's our timeout for response from backend in an actual request [14:41:55] what ever the default is i think [14:42:05] paladox: between mw* and cp* [14:42:08] Can you check [14:42:26] I know for MediaWiki [14:42:54] 30s [14:42:54] https://github.com/miraheze/puppet/blob/master/modules/varnish/templates/default.vcl#L27 [14:42:55] [ puppet/default.vcl at master · miraheze/puppet · GitHub ] - github.com [14:43:06] paladox: that's why [14:43:13] So is MediaWiki for http requests [14:43:17] I'm checking graylog [14:44:33] seems like WMF do this https://github.com/wikimedia/puppet/blob/13ef4869f0eaef36af791e2a466d180abe5019ae/modules/profile/files/trafficserver/default.lua#L66 [14:44:34] [ puppet/default.lua at 13ef4869f0eaef36af791e2a466d180abe5019ae · wikimedia/puppet · GitHub ] - github.com [14:44:40] and [14:44:40] https://github.com/wikimedia/puppet/blob/a07549ca2df5b21dcedf066b00e0cba6c614d52a/modules/varnish/templates/wikimedia-frontend.vcl.erb#L87 [14:44:41] [ puppet/wikimedia-frontend.vcl.erb at a07549ca2df5b21dcedf066b00e0cba6c614d52a · wikimedia/puppet · GitHub ] - github.com [14:45:23] paladox: yes but they don't use instant commons apparently and have much better caching [14:46:31] well yes because its not loading a wiki :) [14:46:55] it's just returning a empty content [14:48:36] paladox: well the backend won't have responded because it's only just timed out from IC [14:48:50] I see a bunch of curl error 28 in graylog [14:49:22] But I'd guess we should either lower MediaWiki's http timeout or up Varnish's [14:49:51] upping varnish would do nothing tbh. [14:50:38] if things are queing up then all childs are in use thus slowness will occur. [14:53:50] paladox: then we lower MediaWiki to like 27 seconds [14:54:04] I can't imagine we normally get anywhere close [14:54:31] isn't it 25s for MW or did i read things wrong? So wouldn't that be increasing it. [14:54:43] 5 for connect and 25 for request [14:55:00] So id take it as 30 total [14:55:12] But request needs to be lowered [14:55:43] 30s won't be the limit in Varnish for much longer [14:56:07] JohnLewis what changes are we planning? [14:56:21] 30s is way too high, so as part of plans to improve caching and capacity, the limit will be decreased to something more sane and reasonable [14:56:36] JohnLewis: then we need to drop that even further [14:56:52] As it doesn't make sense to allow MediaWiki to take longer to process than varnish waits [14:57:33] This isn't how long Varnish waits, this is a simple health check [14:59:34] JohnLewis: well whichever way it needs fixing [15:00:12] It does, and the responsibility of how falls upon the MediaWiki to figure out [15:00:22] *MediaWiki team [15:01:19] JohnLewis: well if you advise me what you want to set the health check to in future then I might as well do it in one go [15:01:41] "in future" is the key word there [15:02:09] I do not know what my future plans will do for caching, I do not know what my future research will give as an appropriate number [15:02:16] JohnLewis: should I just use the 30 seconds for now then [15:02:54] That is what it is at now, in the future you're probably looked at 15s or under [15:10:24] Okay [15:21:27] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.47, 3.62, 3.94 [15:25:27] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.78, 4.01, 4.01 [15:39:26] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.50, 3.82, 4.00 [15:41:25] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.17, 4.02, 4.06 [15:49:26] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.57, 3.78, 3.96 [15:51:26] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 5.18, 4.27, 4.11 [15:55:39] [02mediawiki] 07Universal-Omega closed pull request 03#1326: Bump extensions/MediaWikiChat from `eadada7` to `ca65d0d` - 13https://git.io/JqKHq [15:55:42] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/MediaWikiChat-ca65d0d - 13https://git.io/vbL5b [15:55:43] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/MediaWikiChat-ca65d0d [15:55:45] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5wK [15:55:46] [02miraheze/mediawiki] 07dependabot[bot] 033ef6889 - Bump extensions/MediaWikiChat from `eadada7` to `ca65d0d` (#1326) [15:56:14] [02mediawiki] 07Universal-Omega closed pull request 03#1324: Bump extensions/cldr from `02f2895` to `c03d2bb` - 13https://git.io/JqK9N [15:56:16] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5re [15:56:17] [02miraheze/mediawiki] 07dependabot[bot] 0358e40b0 - Bump extensions/cldr from `02f2895` to `c03d2bb` (#1324) [15:56:19] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/cldr-c03d2bb [15:56:20] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/cldr-c03d2bb - 13https://git.io/vbL5b [15:56:48] [02mediawiki] 07Universal-Omega closed pull request 03#1322: Bump extensions/GettingStarted from `32cb951` to `5ec1c75` - 13https://git.io/JqK9P [15:56:50] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5ru [15:56:51] [02miraheze/mediawiki] 07dependabot[bot] 03c7fd72e - Bump extensions/GettingStarted from `32cb951` to `5ec1c75` (#1322) [15:56:53] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/GettingStarted-5ec1c75 - 13https://git.io/vbL5b [15:56:54] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/GettingStarted-5ec1c75 [15:57:11] [02mediawiki] 07Universal-Omega closed pull request 03#1319: Bump extensions/UniversalLanguageSelector from `0b92c67` to `f31b0c9` - 13https://git.io/JqK93 [15:57:13] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5rP [15:57:14] [02miraheze/mediawiki] 07dependabot[bot] 037c490e5 - Bump extensions/UniversalLanguageSelector from `0b92c67` to `f31b0c9` (#1319) [15:57:16] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/UniversalLanguageSelector-f31b0c9 [15:57:17] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/UniversalLanguageSelector-f31b0c9 - 13https://git.io/vbL5b [15:57:50] [02mediawiki] 07Universal-Omega closed pull request 03#1318: Bump extensions/BlogPage from `7951ba8` to `c4dbf5b` - 13https://git.io/JqK9T [15:57:52] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5ok [15:57:53] [02miraheze/mediawiki] 07dependabot[bot] 03cb3d19b - Bump extensions/BlogPage from `7951ba8` to `c4dbf5b` (#1318) [15:57:55] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/BlogPage-c4dbf5b - 13https://git.io/vbL5b [15:57:56] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/BlogPage-c4dbf5b [16:00:06] [02mediawiki] 07Universal-Omega closed pull request 03#1325: Bump extensions/Cargo from `8fc3204` to `3d811e2` - 13https://git.io/JqKHe [16:00:08] [02mediawiki] 07dependabot[bot] commented on pull request 03#1325: Bump extensions/Cargo from `8fc3204` to `3d811e2` - 13https://git.io/Jq5Ku [16:00:11] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/Cargo-3d811e2 [16:00:13] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/Cargo-3d811e2 - 13https://git.io/vbL5b [16:00:39] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/PageForms-08fbb02 - 13https://git.io/vbL5b [16:00:41] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/PageForms-08fbb02 [16:00:43] [02mediawiki] 07Universal-Omega closed pull request 03#1330: Bump extensions/PageForms from `ff424c0` to `08fbb02` - 13https://git.io/JqKHV [16:00:44] [02mediawiki] 07dependabot[bot] commented on pull request 03#1330: Bump extensions/PageForms from `ff424c0` to `08fbb02` - 13https://git.io/Jq5Kb [16:01:05] [02mediawiki] 07Universal-Omega closed pull request 03#1317: Bump extensions/VEForAll from `38290cd` to `8c90b8d` - 13https://git.io/JqKSh [16:01:06] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq56J [16:01:08] [02miraheze/mediawiki] 07dependabot[bot] 036ca0e26 - Bump extensions/VEForAll from `38290cd` to `8c90b8d` (#1317) [16:01:09] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/VEForAll-8c90b8d - 13https://git.io/vbL5b [16:01:11] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/VEForAll-8c90b8d [16:01:29] [02mediawiki] 07Universal-Omega closed pull request 03#1314: Bump extensions/CommentStreams from `22ebac2` to `6fbe4c1` - 13https://git.io/JqKS2 [16:01:31] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq563 [16:01:32] [02miraheze/mediawiki] 07dependabot[bot] 03c85aafa - Bump extensions/CommentStreams from `22ebac2` to `6fbe4c1` (#1314) [16:01:34] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/CommentStreams-6fbe4c1 [16:01:35] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/CommentStreams-6fbe4c1 - 13https://git.io/vbL5b [16:01:59] [02mediawiki] 07Universal-Omega closed pull request 03#1312: Bump extensions/Editcount from `a6599e6` to `b9a6080` - 13https://git.io/JqKSs [16:02:01] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq562 [16:02:02] [02miraheze/mediawiki] 07dependabot[bot] 036403e08 - Bump extensions/Editcount from `a6599e6` to `b9a6080` (#1312) [16:02:04] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/Editcount-b9a6080 - 13https://git.io/vbL5b [16:02:05] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/Editcount-b9a6080 [16:02:29] [02mediawiki] 07Universal-Omega closed pull request 03#1310: Bump extensions/VoteNY from `80c2e22` to `c7a829a` - 13https://git.io/JqKSf [16:02:30] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq567 [16:02:32] [02miraheze/mediawiki] 07dependabot[bot] 03043e44c - Bump extensions/VoteNY from `80c2e22` to `c7a829a` (#1310) [16:02:33] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/VoteNY-c7a829a - 13https://git.io/vbL5b [16:02:35] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/VoteNY-c7a829a [16:04:59] [02mediawiki] 07Universal-Omega closed pull request 03#1286: Bump extensions/AJAXPoll from `8f2cb6a` to `3830cb4` - 13https://git.io/JqK16 [16:05:00] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5P8 [16:05:02] [02miraheze/mediawiki] 07dependabot[bot] 03a630a99 - Bump extensions/AJAXPoll from `8f2cb6a` to `3830cb4` (#1286) [16:05:05] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/AJAXPoll-3830cb4 - 13https://git.io/vbL5b [16:05:07] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/AJAXPoll-3830cb4 [16:05:26] [02mediawiki] 07Universal-Omega closed pull request 03#1308: Bump extensions/ProtectSite from `ba3a7e1` to `76ba0e8` - 13https://git.io/JqKyS [16:05:28] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5P2 [16:05:29] [02miraheze/mediawiki] 07dependabot[bot] 03aeb7904 - Bump extensions/ProtectSite from `ba3a7e1` to `76ba0e8` (#1308) [16:05:31] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/ProtectSite-76ba0e8 - 13https://git.io/vbL5b [16:05:32] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/ProtectSite-76ba0e8 [16:05:50] [02mediawiki] 07Universal-Omega closed pull request 03#1307: Bump extensions/Purge from `218b277` to `47b21bc` - 13https://git.io/JqKyP [16:05:51] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5Pi [16:05:52] [02miraheze/mediawiki] 07dependabot[bot] 03439cad1 - Bump extensions/Purge from `218b277` to `47b21bc` (#1307) [16:05:54] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/Purge-47b21bc [16:05:55] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/Purge-47b21bc - 13https://git.io/vbL5b [16:06:15] [02mediawiki] 07Universal-Omega closed pull request 03#1301: Bump extensions/HAWelcome from `93bc28f` to `3d81ae1` - 13https://git.io/JqKD7 [16:06:16] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5PH [16:06:18] [02miraheze/mediawiki] 07dependabot[bot] 038acc156 - Bump extensions/HAWelcome from `93bc28f` to `3d81ae1` (#1301) [16:06:19] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/HAWelcome-3d81ae1 - 13https://git.io/vbL5b [16:06:21] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/HAWelcome-3d81ae1 [16:08:04] PROBLEM - jobrunner3 Puppet on jobrunner3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [16:08:50] [02mediawiki] 07Universal-Omega closed pull request 03#1305: Bump extensions/RandomSelection from `79518ac` to `9063e58` - 13https://git.io/JqKyW [16:08:51] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq51t [16:08:53] PROBLEM - jobrunner4 Puppet on jobrunner4 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [16:08:53] [02miraheze/mediawiki] 07dependabot[bot] 03c0f514a - Bump extensions/RandomSelection from `79518ac` to `9063e58` (#1305) [16:08:55] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/RandomSelection-9063e58 [16:08:56] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/RandomSelection-9063e58 - 13https://git.io/vbL5b [16:09:15] [02mediawiki] 07Universal-Omega closed pull request 03#1303: Bump extensions/TwitterTag from `56bf26b` to `0c2cee6` - 13https://git.io/JqKyJ [16:09:16] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq513 [16:09:17] [02miraheze/mediawiki] 07dependabot[bot] 036c9ccc3 - Bump extensions/TwitterTag from `56bf26b` to `0c2cee6` (#1303) [16:09:19] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/TwitterTag-0c2cee6 [16:09:21] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/TwitterTag-0c2cee6 - 13https://git.io/vbL5b [16:10:41] PROBLEM - mw8 Puppet on mw8 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [16:10:52] PROBLEM - mw9 Puppet on mw9 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [16:11:25] PROBLEM - mw10 Puppet on mw10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [16:11:39] PROBLEM - mw11 Puppet on mw11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [16:34:02] RECOVERY - jobrunner3 Puppet on jobrunner3 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [16:34:39] RECOVERY - mw8 Puppet on mw8 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [16:34:50] RECOVERY - jobrunner4 Puppet on jobrunner4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [16:34:51] RECOVERY - mw9 Puppet on mw9 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [16:35:08] RECOVERY - test3 Puppet on test3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [16:35:18] [02mediawiki] 07Universal-Omega closed pull request 03#1295: Bump extensions/QuizGame from `c8268e3` to `e5119a2` - 13https://git.io/JqKMx [16:35:19] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5NM [16:35:21] [02miraheze/mediawiki] 07dependabot[bot] 03b003358 - Bump extensions/QuizGame from `c8268e3` to `e5119a2` (#1295) [16:35:22] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/QuizGame-e5119a2 [16:35:23] RECOVERY - mw10 Puppet on mw10 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [16:35:24] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/QuizGame-e5119a2 - 13https://git.io/vbL5b [16:35:38] RECOVERY - mw11 Puppet on mw11 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [16:37:21] [02mediawiki] 07Universal-Omega closed pull request 03#1294: Bump extensions/ApprovedRevs from `916a0ea` to `801399b` - 13https://git.io/JqKMQ [16:37:24] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5AP [16:37:26] [02miraheze/mediawiki] 07dependabot[bot] 035855644 - Bump extensions/ApprovedRevs from `916a0ea` to `801399b` (#1294) [16:37:27] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/ApprovedRevs-801399b - 13https://git.io/vbL5b [16:37:29] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/ApprovedRevs-801399b [16:37:40] [02mediawiki] 07Universal-Omega closed pull request 03#1293: Bump extensions/WikiForum from `218044c` to `4db12ac` - 13https://git.io/JqKMr [16:37:41] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5AF [16:37:43] [02miraheze/mediawiki] 07dependabot[bot] 03906bc6a - Bump extensions/WikiForum from `218044c` to `4db12ac` (#1293) [16:37:44] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/WikiForum-4db12ac [16:37:46] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/WikiForum-4db12ac - 13https://git.io/vbL5b [16:38:22] [02mediawiki] 07Universal-Omega closed pull request 03#1280: Bump extensions/SocialProfile from `e0286cb` to `fbb875b` - 13https://git.io/JqKXx [16:38:24] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5xY [16:38:25] [02miraheze/mediawiki] 07dependabot[bot] 03d65c5a2 - Bump extensions/SocialProfile from `e0286cb` to `fbb875b` (#1280) [16:38:27] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/SocialProfile-fbb875b - 13https://git.io/vbL5b [16:38:28] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/SocialProfile-fbb875b [16:38:59] [02mediawiki] 07Universal-Omega closed pull request 03#1304: Bump extensions/WikiCategoryTagCloud from `8d08994` to `eaa931b` - 13https://git.io/JqKym [16:39:00] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5x2 [16:39:02] [02miraheze/mediawiki] 07dependabot[bot] 03be5d327 - Bump extensions/WikiCategoryTagCloud from `8d08994` to `eaa931b` (#1304) [16:39:03] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/WikiCategoryTagCloud-eaa931b [16:39:05] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/WikiCategoryTagCloud-eaa931b - 13https://git.io/vbL5b [16:39:29] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.21, 3.59, 4.00 [16:43:21] [02mediawiki] 07Universal-Omega closed pull request 03#1302: Bump extensions/MsCatSelect from `8b91b1a` to `6dddf39` - 13https://git.io/JqKDF [16:43:23] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5h5 [16:43:24] [02miraheze/mediawiki] 07dependabot[bot] 033d8020f - Bump extensions/MsCatSelect from `8b91b1a` to `6dddf39` (#1302) [16:43:26] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/MsCatSelect-6dddf39 - 13https://git.io/vbL5b [16:43:27] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/MsCatSelect-6dddf39 [16:43:29] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.10, 3.79, 3.98 [16:44:05] [02mediawiki] 07Universal-Omega closed pull request 03#1297: Bump extensions/NukeDPL from `6858907` to `82a69fc` - 13https://git.io/JqKDW [16:44:06] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5jq [16:44:08] [02miraheze/mediawiki] 07dependabot[bot] 03ae98df9 - Bump extensions/NukeDPL from `6858907` to `82a69fc` (#1297) [16:44:10] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/NukeDPL-82a69fc - 13https://git.io/vbL5b [16:44:11] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/NukeDPL-82a69fc [16:44:46] [02mediawiki] 07Universal-Omega closed pull request 03#1292: Bump extensions/Kartographer from `7272b2b` to `ce6de47` - 13https://git.io/JqKMa [16:44:48] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5j6 [16:44:49] [02miraheze/mediawiki] 07dependabot[bot] 0350cfc3e - Bump extensions/Kartographer from `7272b2b` to `ce6de47` (#1292) [16:44:51] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/Kartographer-ce6de47 - 13https://git.io/vbL5b [16:44:52] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/Kartographer-ce6de47 [16:45:11] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jq5j9 [16:45:12] [02miraheze/mediawiki] 07dependabot[bot] 034c63e70 - Bump extensions/PollNY from `668f186` to `ab834a4` (#1291) [16:45:14] [02mediawiki] 07Universal-Omega closed pull request 03#1291: Bump extensions/PollNY from `668f186` to `ab834a4` - 13https://git.io/JqKMB [16:45:15] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/PollNY-ab834a4 [16:45:17] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/PollNY-ab834a4 - 13https://git.io/vbL5b [16:46:29] [02mediawiki] 07Universal-Omega closed pull request 03#1289: Bump extensions/GeoCrumbs from `20fbdc8` to `ff63e3b` - 13https://git.io/JqKMk [16:46:31] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JqdeG [16:46:32] [02miraheze/mediawiki] 07dependabot[bot] 03dd41ef0 - Bump extensions/GeoCrumbs from `20fbdc8` to `ff63e3b` (#1289) [16:46:34] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/GeoCrumbs-ff63e3b - 13https://git.io/vbL5b [16:46:35] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/GeoCrumbs-ff63e3b [16:46:52] [02mediawiki] 07Universal-Omega closed pull request 03#1288: Bump skins/MinervaNeue from `5141b06` to `97729f6` - 13https://git.io/JqKMe [16:46:53] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JqdeR [16:46:55] [02miraheze/mediawiki] 07dependabot[bot] 034166ef5 - Bump skins/MinervaNeue from `5141b06` to `97729f6` (#1288) [16:46:56] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/skins/MinervaNeue-97729f6 [16:46:58] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/skins/MinervaNeue-97729f6 - 13https://git.io/vbL5b [16:47:09] [02mediawiki] 07Universal-Omega closed pull request 03#1287: Bump extensions/Maps from `79cac63` to `d4f8f90` - 13https://git.io/JqK1Q [16:47:11] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jqder [16:47:12] [02miraheze/mediawiki] 07dependabot[bot] 031c57dcd - Bump extensions/Maps from `79cac63` to `d4f8f90` (#1287) [16:47:14] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/Maps-d4f8f90 [16:47:15] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/Maps-d4f8f90 - 13https://git.io/vbL5b [16:47:30] [02mediawiki] 07Universal-Omega closed pull request 03#1285: Bump extensions/ContributionScores from `0c38613` to `6fac009` - 13https://git.io/JqK1a [16:47:32] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JqdeS [16:47:33] [02miraheze/mediawiki] 07dependabot[bot] 03dbfc787 - Bump extensions/ContributionScores from `0c38613` to `6fac009` (#1285) [16:47:35] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/ContributionScores-6fac009 - 13https://git.io/vbL5b [16:47:36] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/ContributionScores-6fac009 [16:47:53] [02mediawiki] 07Universal-Omega closed pull request 03#1283: Bump extensions/LocalisationUpdate from `4813f8c` to `f1cc54c` - 13https://git.io/JqK1s [16:47:55] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/JqdvU [16:47:56] [02miraheze/mediawiki] 07dependabot[bot] 032c53802 - Bump extensions/LocalisationUpdate from `4813f8c` to `f1cc54c` (#1283) [16:47:58] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/LocalisationUpdate-f1cc54c [16:47:59] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/LocalisationUpdate-f1cc54c - 13https://git.io/vbL5b [16:48:21] [02mediawiki] 07Universal-Omega closed pull request 03#1282: Bump extensions/Graph from `69ec90a` to `09f0338` - 13https://git.io/JqK1t [16:48:23] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jqdvc [16:48:24] [02miraheze/mediawiki] 07dependabot[bot] 03600962e - Bump extensions/Graph from `69ec90a` to `09f0338` (#1282) [16:48:26] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/Graph-09f0338 [16:48:27] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/Graph-09f0338 - 13https://git.io/vbL5b [16:48:42] [02mediawiki] 07Universal-Omega closed pull request 03#1281: Bump extensions/3D from `f29cb2a` to `2549882` - 13https://git.io/JqK1J [16:48:44] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jqdvu [16:48:45] [02miraheze/mediawiki] 07dependabot[bot] 03a47250f - Bump extensions/3D from `f29cb2a` to `2549882` (#1281) [16:48:47] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/3D-2549882 - 13https://git.io/vbL5b [16:48:48] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_35/extensions/3D-2549882 [16:51:38] PROBLEM - cp11 Current Load on cp11 is CRITICAL: CRITICAL - load average: 6.09, 5.00, 2.22 [16:53:37] PROBLEM - cp11 Current Load on cp11 is WARNING: WARNING - load average: 1.39, 3.53, 2.02 [16:54:59] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JqdU0 [16:55:01] [02miraheze/mw-config] 07Universal-Omega 03df25fba - Update LocalSettings.php [16:55:02] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/vbvb3 [16:55:36] RECOVERY - cp11 Current Load on cp11 is OK: OK - load average: 0.97, 2.72, 1.90 [16:56:29] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JqdT8 [16:56:31] [02miraheze/mw-config] 07Universal-Omega 03b03923f - Update LocalExtensions.php [17:00:19] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JqdI3 [17:00:21] [02miraheze/mw-config] 07Universal-Omega 0378a8f55 - Update ManageWikiExtensions.php [17:00:57] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JqdI8 [17:00:58] [02miraheze/mw-config] 07Universal-Omega 039ad4e19 - Update extension-list [17:01:47] [02mw-config] 07Universal-Omega opened pull request 03#3786: Add VariablesLua extension (T6959) - 13https://git.io/JqdI1 [17:02:51] miraheze/mw-config - Universal-Omega the build passed. [17:07:34] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.74, 3.69, 3.96 [17:07:59] !log DROP DATABASE maximusescwiki; [17:08:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:09:11] PROBLEM - test3 Puppet on test3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [17:09:33] PROBLEM - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 246s [17:09:36] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.79, 4.07, 4.07 [17:11:19] [02ssl] 07Universal-Omega opened pull request 03#401: Remove civclassic.civwiki.org (changed per T6953) - 13https://git.io/JqdmE [17:12:26] [02ssl] 07Universal-Omega synchronize pull request 03#401: Remove civclassic.civwiki.org (changed per T6953) - 13https://git.io/JqdmE [17:15:09] PROBLEM - dbbackup2 Current Load on dbbackup2 is WARNING: WARNING - load average: 3.81, 2.47, 1.43 [17:15:40] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.64, 3.91, 4.00 [17:17:08] RECOVERY - dbbackup2 Current Load on dbbackup2 is OK: OK - load average: 2.82, 2.66, 1.64 [17:17:41] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.45, 4.03, 4.03 [17:19:56] [02ssl] 07Reception123 closed pull request 03#401: Remove civclassic.civwiki.org (changed per T6953) - 13https://git.io/JqdmE [17:19:57] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-1/±1] 13https://git.io/Jqd32 [17:19:59] [02miraheze/ssl] 07Universal-Omega 038e58fc8 - Remove civclassic.civwiki.org (changed per T6953) (#401) [17:21:43] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.88, 3.93, 3.99 [17:27:10] i think that we should up the check_log warning/critical for dbbackup* thoughts JohnLewis ? [17:27:17] it's quite spammy [17:27:49] Are they real warnings? [17:27:58] If yes, they're not 'spammy' they're alerting to a problem [17:29:45] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 6.07, 4.51, 4.12 [17:35:37] RECOVERY - dbbackup2 Check MariaDB Replication c3 on dbbackup2 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 1s [17:55:43] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.86, 3.89, 4.00 [18:03:40] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.98, 4.17, 4.02 [18:06:38] [02miraheze/CreateWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/JqdgU [18:06:40] [02miraheze/CreateWiki] 07translatewiki 031841fce - Localisation updates from https://translatewiki.net. [18:06:41] [ Main page - translatewiki.net ] - translatewiki.net [18:06:41] [02miraheze/ManageWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqdgT [18:06:43] [02miraheze/ManageWiki] 07translatewiki 032b48684 - Localisation updates from https://translatewiki.net. [18:06:44] [ Main page - translatewiki.net ] - translatewiki.net [18:06:44] [02miraheze/MirahezeMagic] 07translatewiki pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/Jqdgk [18:06:46] [02miraheze/MirahezeMagic] 07translatewiki 03b6c6f48 - Localisation updates from https://translatewiki.net. [18:06:47] [ Main page - translatewiki.net ] - translatewiki.net [18:07:36] miraheze/CreateWiki - translatewiki the build passed. [18:07:39] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.15, 3.80, 3.91 [18:07:42] miraheze/MirahezeMagic - translatewiki the build passed. [18:07:49] miraheze/ManageWiki - translatewiki the build passed. [18:09:32] Can chanop remove that ban ^^ [18:13:07] Ty Universal_Omega [18:13:22] No problem. [18:13:41] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.31, 3.94, 3.92 [18:18:33] 502 Bad Gateways on `metawiki` (likely cp12 again) [18:19:35] im getting 502 on some wikis but not on others [18:19:38] I was going to say the same thing. [18:19:52] !log reception@jobrunner3:~$ sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=awesomegameswiki --r "[[phab:T6980|Requested]]" /home/reception/awesomegamesdel.txt [18:19:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:20:14] https://grafana.miraheze.org/d/W9MIkA7iz/miraheze-cluster?orgId=1&var-job=node&var-node=cp12.miraheze.org&var-port=9100 [18:20:15] [ Grafana ] - grafana.miraheze.org [18:20:20] cp12 went down [18:20:35] PROBLEM - cp12 HTTPS on cp12 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 344 bytes in 0.353 second response time [18:20:47] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 51.222.25.132/cpweb, 2607:5300:205:200::1c30/cpweb [18:20:55] Mar 15 18:17:59 cp12 kernel: [264917.105644] Out of memory: Kill process 17838 (cache-main) score 757 or sacrifice child [18:21:00] ran out of storage space? [18:21:01] PROBLEM - cp12 Varnish Backends on cp12 is WARNING: No backends detected. If this is an error, see readme.txt [18:21:10] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 51.222.25.132/cpweb, 2607:5300:205:200::1c30/cpweb [18:21:28] PROBLEM - cp12 HTTP 4xx/5xx ERROR Rate on cp12 is CRITICAL: CRITICAL - NGINX Error Rate is 100% [18:21:57] paladox: seems to be like last time [18:22:12] i'm going to depool cp12 [18:22:47] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jqdrt [18:22:48] [02miraheze/dns] 07paladox 03c41cb08 - Depool ca [18:24:07] paladox, ah thanks :) [18:25:37] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.10, 3.75, 3.95 [18:27:37] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.23, 3.88, 3.96 [18:35:34] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.29, 3.76, 3.96 [18:37:04] PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 1.65, 3.67, 2.18 [18:37:34] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.43, 3.99, 4.02 [18:39:04] RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 0.98, 2.75, 2.02 [18:39:20] !log reception@jobrunner3:~$ sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki maximusescwiki /home/reception/delbackups3/maximusescwiki.xml --username-prefix="maximusesc" [18:39:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:39:33] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.93, 3.94, 3.99 [18:41:33] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 5.26, 4.36, 4.13 [18:43:18] !log reception@jobrunner4:~$ sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=pokemundowiki --r "[[phab:T6960|Requested]]" /home/reception/deletebatch.txt [18:43:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:43:32] SRE, I'm getting 502 bad gateways on `maximusescwiki` now. Not sure which CP [18:44:04] works fine for me [18:44:12] yeah must've been a temporary thing [18:44:19] PROBLEM - wiki.conworlds.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.conworlds.org could not be found [18:44:23] but the servers seem to be really struggling today [18:44:24] PROBLEM - innersphere.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for innersphere.tk could not be found [18:44:24] PROBLEM - adadevelopersacademy.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for adadevelopersacademy.wiki could not be found [18:44:27] PROBLEM - wiki.tigcity.tk - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.tigcity.tk could not be found [18:44:31] PROBLEM - wiki.meeusen.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.meeusen.net could not be found [18:44:32] PROBLEM - pt.graalmilitary.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for pt.graalmilitary.com could not be found [18:44:36] PROBLEM - mediawiki.ecfin.eu - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for mediawiki.ecfin.eu could not be found [18:44:37] PROBLEM - tl.awiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for tl.awiki.org could not be found [18:44:47] dmehus: well now with cp12 depooled there would be more pressure on the others [18:45:04] PROBLEM - cp10 Current Load on cp10 is WARNING: WARNING - load average: 3.87, 3.88, 2.75 [18:45:15] Reception123, yeah, but cp12 is back in service now? [18:45:24] so why would there be more pressure on the others? [18:45:24] Reception123: the rDNS down shows so too [18:45:33] ^ [18:45:39] dmehus: it's not in service [18:45:42] oh [18:45:43] wow [18:46:11] restarting CP takes a long time then [18:46:21] it doesn't, not sure what paladox is doing [18:46:25] dmehus: it's depooled because something seems broke [18:46:29] paladox: status? [18:46:30] oh [18:46:37] we're debugging cp12, we've depooled it. [18:46:40] Reception123, ah, okay, that makes sense then [18:46:45] dmehus: things don't randomly run out of memory for no reason [18:46:56] RhinosF1, yeah, definitely [18:47:04] RECOVERY - cp10 Current Load on cp10 is OK: OK - load average: 1.47, 2.91, 2.53 [18:49:01] dmehus: unless there's high traffic which I can't see then it would probably indicate a fault [18:51:10] RECOVERY - wiki.conworlds.org - reverse DNS on sslhost is OK: rDNS OK - wiki.conworlds.org reverse DNS resolves to cp11.miraheze.org [18:51:11] RECOVERY - innersphere.tk - reverse DNS on sslhost is OK: rDNS OK - innersphere.tk reverse DNS resolves to cp11.miraheze.org [18:51:14] RECOVERY - pt.graalmilitary.com - reverse DNS on sslhost is OK: rDNS OK - pt.graalmilitary.com reverse DNS resolves to cp10.miraheze.org [18:51:14] RECOVERY - adadevelopersacademy.wiki - reverse DNS on sslhost is OK: rDNS OK - adadevelopersacademy.wiki reverse DNS resolves to cp11.miraheze.org [18:51:24] RECOVERY - wiki.tigcity.tk - reverse DNS on sslhost is OK: rDNS OK - wiki.tigcity.tk reverse DNS resolves to cp11.miraheze.org [18:51:25] RECOVERY - tl.awiki.org - reverse DNS on sslhost is OK: rDNS OK - tl.awiki.org reverse DNS resolves to cp10.miraheze.org [18:51:27] RECOVERY - mediawiki.ecfin.eu - reverse DNS on sslhost is OK: rDNS OK - mediawiki.ecfin.eu reverse DNS resolves to cp10.miraheze.org [18:51:28] RECOVERY - wiki.meeusen.net - reverse DNS on sslhost is OK: rDNS OK - wiki.meeusen.net reverse DNS resolves to cp11.miraheze.org [18:51:52] ok [18:52:11] RhinosF1: "which I can't see" hm? [18:52:36] JohnLewis: when I skimmed over grafana last time it didn't look like it [18:53:05] Today is a different story though [18:53:10] This time [18:53:28] https://grafana.miraheze.org/d/W9MIkA7iz/miraheze-cluster?viewPanel=289&orgId=1&var-job=node&var-node=cp12.miraheze.org&var-port=9100&from=now-3h&to=now-1m [18:53:29] [ Grafana ] - grafana.miraheze.org [18:53:36] Yeah that's a traffic spike [19:25:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.05, 3.72, 4.00 [19:29:20] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 5.07, 4.11, 4.08 [19:35:19] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.50, 3.87, 3.99 [19:38:44] !log restart varnish on cp12 [19:38:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:38:58] RECOVERY - cp12 Varnish Backends on cp12 is OK: All 7 backends are healthy [19:39:04] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jqdho [19:39:05] [02miraheze/dns] 07paladox 037d842f1 - Revert "Depool ca" [19:39:10] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [19:39:48] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [19:40:07] RECOVERY - cp12 HTTPS on cp12 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 2214 bytes in 0.417 second response time [19:40:09] paladox: things ok now? [19:40:29] well we've decided on nothing we can really do right now so restarted [19:41:00] Ok [19:41:08] RECOVERY - cp12 HTTP 4xx/5xx ERROR Rate on cp12 is OK: OK - NGINX Error Rate is 3% [19:41:18] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.42, 3.75, 3.86 [19:41:45] "nothing we can really do" is not quite right [19:42:01] There are two things we *can* do, one is just bad and the other isn't my decision [19:43:40] Can you expand? Out of interest [19:43:51] Mainly on the one that's not your call [19:44:09] Not my call: adding another cp in Canada [19:44:18] Not a good idea: Decreasing caching efficiency [19:44:42] Another cp has come up before [19:44:56] I think last stats showed it wouldn't be a bad idea [19:45:04] But then you found issues with OVH [19:45:14] And didn't it get moved [19:45:18] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.80, 3.74, 3.83 [19:45:25] But I guess that's SPF|Cloud [19:46:16] We wouldn't be able to afford another cp [19:46:52] Wouldn't we? [19:47:00] We barely have enough to afford what we have now if we loose the same amount as last year in the next 9 months [19:47:08] According to my very quick maths [19:47:42] oh i didn't realise we have 3k [19:47:50] gained 1.5k [19:51:04] The issue is if we're going to do caching PoPs, they need to be done right [19:51:26] That's a very fair point [19:51:33] Caching is useless if done wrong [20:01:16] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.12, 3.84, 3.73 [20:03:17] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.48, 3.64, 3.67 [20:09:08] I'm not sure what another cache proxy costs, but it's not a material part of our infrastructure costs, so yeah, we likely could afford it. The question for me is whether our current budget overall is sustainable. I think we're fine through the remainder of the year, but we did still come up about 400-500 GBP of what we needed, so next year we'll still have fundraise for this year's goal + planned expansion costs + still account for this [20:09:09] year's UK tax liability [20:09:54] I think I've seen something about caching on one of the Stewards' noticeboard's archives. [20:10:38] That's possible, yes, likely referring the user(s) to a Phabricator ticket, if I remember correctly [20:11:18] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.22, 3.68, 3.68 [20:21:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.01, 3.81, 3.88 [20:29:25] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 5.55, 4.45, 4.08 [21:06:05] PROBLEM - debilepedie.gq - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - debilepedie.gq All nameservers failed to answer the query. [21:06:33] .dns debliepedie.gq A [21:06:34] RhinosF1: DNS lookup returned NXDOMAIN for debliepedie.gq. [21:08:23] .dns debliepedie.gq CNAME [21:08:23] RhinosF1: DNS lookup returned NXDOMAIN for debliepedie.gq. [21:08:26] Hmm [21:08:38] Works in browser [21:08:44] Unless my spelling is PII [21:08:46] Poo [21:12:06] PROBLEM - arquivo.ucmg.ml - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - arquivo.ucmg.ml All nameservers failed to answer the query. [21:12:53] RECOVERY - debilepedie.gq - reverse DNS on sslhost is OK: rDNS OK - debilepedie.gq reverse DNS resolves to cp10.miraheze.org [21:15:21] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.78, 3.89, 3.99 [21:15:40] PROBLEM - wiki.ripto.gq - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.ripto.gq All nameservers failed to answer the query. [21:16:09] PROBLEM - factorio-mods.tk - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - factorio-mods.tk All nameservers failed to answer the query. [21:18:36] PROBLEM - kk.uncyc.tk - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - kk.uncyc.tk All nameservers failed to answer the query. [21:21:19] PROBLEM - wiki.wysteria.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.wysteria.ga All nameservers failed to answer the query. [21:21:22] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.08, 3.86, 3.93 [21:23:06] RECOVERY - factorio-mods.tk - reverse DNS on sslhost is OK: rDNS OK - factorio-mods.tk reverse DNS resolves to cp10.miraheze.org [21:25:29] RECOVERY - kk.uncyc.tk - reverse DNS on sslhost is OK: rDNS OK - kk.uncyc.tk reverse DNS resolves to cp11.miraheze.org [21:26:01] RECOVERY - arquivo.ucmg.ml - reverse DNS on sslhost is OK: rDNS OK - arquivo.ucmg.ml reverse DNS resolves to cp10.miraheze.org [21:27:20] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.28, 3.82, 3.93 [21:28:14] RECOVERY - wiki.wysteria.ga - reverse DNS on sslhost is OK: rDNS OK - wiki.wysteria.ga reverse DNS resolves to cp11.miraheze.org [21:29:41] RECOVERY - wiki.ripto.gq - reverse DNS on sslhost is OK: rDNS OK - wiki.ripto.gq reverse DNS resolves to cp11.miraheze.org [21:33:18] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.24, 3.89, 3.92 [21:37:28] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JqFBX [21:37:30] [02miraheze/puppet] 07paladox 037b37014 - cloud[345]: Send logs to centralised logger [21:37:31] [02puppet] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vbiAS [21:37:33] [02puppet] 07paladox opened pull request 03#1712: cloud[345]: Send logs to centralised logger - 13https://git.io/JqFB1 [21:37:46] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JqFBQ [21:37:48] [02miraheze/puppet] 07paladox 0304e84cd - Update cloud4.yaml [21:37:49] [02puppet] 07paladox synchronize pull request 03#1712: cloud[345]: Send logs to centralised logger - 13https://git.io/JqFB1 [21:37:56] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JqFBF [21:37:57] [02miraheze/puppet] 07paladox 03b3c7156 - Update cloud5.yaml [21:37:59] [02puppet] 07paladox synchronize pull request 03#1712: cloud[345]: Send logs to centralised logger - 13https://git.io/JqFB1 [21:38:06] [02puppet] 07paladox closed pull request 03#1712: cloud[345]: Send logs to centralised logger - 13https://git.io/JqFB1 [21:38:07] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/JqFBN [21:38:09] [02miraheze/puppet] 07paladox 030eae836 - cloud[345]: Send logs to centralised logger (#1712) [21:38:10] [02puppet] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vbiAS [21:38:12] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-2 [21:39:18] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.86, 3.94, 3.95 [21:41:18] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.64, 4.16, 4.02 [21:43:18] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.45, 3.85, 3.92 [21:47:19] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.20, 3.89, 3.90 [21:57:38] PROBLEM - graylog2 Current Load on graylog2 is CRITICAL: CRITICAL - load average: 12.80, 5.57, 2.68 [22:01:38] RECOVERY - graylog2 Current Load on graylog2 is OK: OK - load average: 0.66, 2.96, 2.29 [22:51:19] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.14, 3.57, 3.97 [22:53:06] PROBLEM - ping4 on cp3 is CRITICAL: PING CRITICAL - Packet loss = 100% [22:53:17] PROBLEM - cp3 Disk Space on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:53:19] PROBLEM - cp3 APT on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:53:20] PROBLEM - cp3 Stunnel Http for mw11 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:53:29] PROBLEM - cp3 SSH on cp3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:53:33] PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:53:38] PROBLEM - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:53:39] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb [22:53:45] PROBLEM - cp3 PowerDNS Recursor on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:54:06] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb [22:55:06] RECOVERY - ping4 on cp3 is OK: PING OK - Packet loss = 0%, RTA = 249.04 ms [22:55:13] RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 6171 MB (25% inode=92%); [22:55:15] RECOVERY - cp3 APT on cp3 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [22:55:16] RECOVERY - cp3 Stunnel Http for mw11 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15194 bytes in 1.024 second response time [22:55:27] RECOVERY - cp3 SSH on cp3 is OK: SSH OK - OpenSSH_7.9p1 Debian-10+deb10u2 (protocol 2.0) [22:55:33] RECOVERY - cp3 Varnish Backends on cp3 is OK: All 7 backends are healthy [22:55:37] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [22:55:38] RECOVERY - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is OK: OK - NGINX Error Rate is 15% [22:55:43] RECOVERY - cp3 PowerDNS Recursor on cp3 is OK: DNS OK: 0.499 seconds response time. miraheze.org returns 128.199.139.216,2400:6180:0:d0::403:f001 [22:56:08] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [22:57:38] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.33, 3.89, 3.96 [23:02:04] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb [23:02:15] PROBLEM - cp3 SSH on cp3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:02:32] PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:02:33] PROBLEM - cp3 PowerDNS Recursor on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:02:44] PROBLEM - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:02:47] PROBLEM - cp3 HTTPS on cp3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:02:56] PROBLEM - cp3 Stunnel Http for mw10 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:03:01] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb [23:03:02] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:03:08] PROBLEM - cp3 NTP time on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:03:18] PROBLEM - cp3 Stunnel Http for test3 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:03:22] PROBLEM - cp3 Stunnel Http for mw8 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:03:34] PROBLEM - cp3 Current Load on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:03:37] PROBLEM - cp3 Stunnel Http for mon2 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:03:44] PROBLEM - ping6 on cp3 is CRITICAL: PING CRITICAL - Packet loss = 100% [23:03:55] huh [23:03:55] PROBLEM - cp3 Stunnel Http for mw9 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:04:00] PROBLEM - Host cp3 is DOWN: CRITICAL - Time to live exceeded (128.199.139.216) [23:04:13] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.51, 3.85, 3.95 [23:04:43] PROBLEM - dreamsit.com.br - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:05:03] cp3 has gone down [23:05:10] cc JohnLewis [23:05:30] i'm not sure what we do? [23:06:17] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.91, 4.08, 4.01 [23:06:31] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqFXx [23:06:32] [02miraheze/dns] 07paladox 038aaeb05 - Depool sg [23:08:29] Paladox: down down? Can you access the DigitalOcean account? [23:08:41] i haven't been able to access the account in years [23:08:54] and yeh, the networks gone down [23:09:01] pinging not working even from my mac [23:09:03] I lost access when I last left and never regained it [23:09:29] pingings working now [23:10:39] Mar 15 23:01:13 cp3 pdns_recursor[12633]: Failed to update . records, got an exception: Too much time waiting for .|NS, timeouts: 5, throttles: 0, queries: 6, 7507msec [23:10:45] looks like do had a network outage [23:10:56] RECOVERY - Host cp3 is UP: PING OK - Packet loss = 0%, RTA = 349.72 ms [23:11:00] RECOVERY - cp3 Stunnel Http for mw9 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15193 bytes in 1.386 second response time [23:11:00] PROBLEM - ping4 on cp3 is WARNING: PING WARNING - Packet loss = 0%, RTA = 349.42 ms [23:11:00] RECOVERY - cp3 SSH on cp3 is OK: SSH OK - OpenSSH_7.9p1 Debian-10+deb10u2 (protocol 2.0) [23:11:05] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [23:11:09] RECOVERY - cp3 PowerDNS Recursor on cp3 is OK: DNS OK: 0.497 seconds response time. miraheze.org returns 2001:41d0:800:178a::5,2001:41d0:800:1bbd::4,51.195.236.219,51.195.236.250 [23:11:14] RECOVERY - cp3 Varnish Backends on cp3 is OK: All 7 backends are healthy [23:11:16] RECOVERY - cp3 HTTPS on cp3 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 2210 bytes in 1.739 second response time [23:11:22] RECOVERY - cp3 Stunnel Http for mw10 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15194 bytes in 1.437 second response time [23:11:25] RECOVERY - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is OK: OK - NGINX Error Rate is 10% [23:11:30] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 28 minutes ago with 0 failures [23:11:35] RECOVERY - cp3 Stunnel Http for test3 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15203 bytes in 1.384 second response time [23:11:40] RECOVERY - cp3 NTP time on cp3 is OK: NTP OK: Offset -0.002309024334 secs [23:11:42] RECOVERY - dreamsit.com.br - LetsEncrypt on sslhost is OK: OK - Certificate 'dreamsit.com.br' will expire on Fri 11 Jun 2021 15:23:19 GMT +0000. [23:11:45] RECOVERY - cp3 Stunnel Http for mw8 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15193 bytes in 1.384 second response time [23:11:55] RECOVERY - cp3 Stunnel Http for mon2 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 32910 bytes in 1.381 second response time [23:11:55] RECOVERY - cp3 Current Load on cp3 is OK: OK - load average: 0.08, 0.03, 0.05 [23:12:08] RECOVERY - ping6 on cp3 is OK: PING OK - Packet loss = 0%, RTA = 274.19 ms [23:12:12] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [23:12:14] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.71, 3.92, 3.99 [23:12:39] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JqFMq [23:12:40] [02miraheze/dns] 07paladox 03dcfe357 - Revert "Depool sg" [23:14:33] JohnLewis resolved ^ [23:14:35] RECOVERY - ping4 on cp3 is OK: PING OK - Packet loss = 0%, RTA = 249.38 ms [23:26:13] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.04, 3.66, 3.72 [23:28:14] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.92, 3.64, 3.70 [23:30:16] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.11, 3.74, 3.72 [23:32:16] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.89, 3.81, 3.76 [23:36:14] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 4.34, 3.94, 3.81 [23:36:22] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+1/-0/±0] 13https://git.io/JqF9M [23:36:23] [02miraheze/puppet] 07paladox 03a216115 - Introduce cloud::logging [23:36:25] [02puppet] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vbiAS [23:36:26] [02puppet] 07paladox opened pull request 03#1713: Introduce cloud::logging - 13https://git.io/JqF9y [23:37:31] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JqFHv [23:37:32] [02miraheze/puppet] 07paladox 0318cf255 - Update init.pp [23:37:34] [02puppet] 07paladox synchronize pull request 03#1713: Introduce cloud::logging - 13https://git.io/JqF9y [23:38:13] PROBLEM - dbbackup1 Current Load on dbbackup1 is WARNING: WARNING - load average: 3.95, 3.91, 3.81 [23:38:16] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JqFHq [23:38:17] [02miraheze/puppet] 07paladox 03ee38640 - Update init.pp [23:38:19] [02puppet] 07paladox synchronize pull request 03#1713: Introduce cloud::logging - 13https://git.io/JqF9y [23:40:33] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+1/-0/±0] 13https://git.io/JqFHP [23:40:35] [02miraheze/puppet] 07paladox 032c87f1f - Create pve.logrotate.conf [23:40:36] [02puppet] 07paladox synchronize pull request 03#1713: Introduce cloud::logging - 13https://git.io/JqF9y [23:48:09] PROBLEM - dbbackup1 Current Load on dbbackup1 is CRITICAL: CRITICAL - load average: 5.12, 4.06, 3.83 [23:53:44] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JqF5C [23:53:45] [02miraheze/puppet] 07paladox 03a9c6dd1 - Update init.pp [23:53:47] [02puppet] 07paladox synchronize pull request 03#1713: Introduce cloud::logging - 13https://git.io/JqF9y [23:59:24] paladox: can you please close https://github.com/miraheze/puppet/pull/1706 ? [23:59:24] [ What will we do with a drunken sailor? What will we do with a drunken… · Pull Request #1706 · miraheze/puppet · GitHub ] - github.com [23:59:35] [02puppet] 07paladox closed pull request 03#1706: What will we do with a drunken sailor? What will we do with a drunken… - 13https://git.io/JqadM [23:59:35] done [23:59:49] Thanks!