[00:25:07] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvh4e [00:25:09] [02miraheze/services] 07MirahezeSSLBot 03d897d40 - BOT: Updating services config for wikis [08:06:22] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_34 [+1/-0/±1] 13https://git.io/JvhaB [08:06:24] [02miraheze/mediawiki] 07Reception123 0360ec512 - Install GoogleDocs4MW [08:22:35] PROBLEM - mw4 Puppet on mw4 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 4 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [08:22:52] PROBLEM - mw5 Puppet on mw5 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 4 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [08:25:32] RECOVERY - mw4 Puppet on mw4 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [08:25:39] RECOVERY - mw5 Puppet on mw5 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [08:37:13] PROBLEM - cp8 Current Load on cp8 is CRITICAL: CRITICAL - load average: 0.91, 2.12, 1.35 [08:39:56] RECOVERY - cp8 Current Load on cp8 is OK: OK - load average: 0.61, 1.46, 1.22 [09:07:15] [02puppet] 07heroku-miraheze opened pull request 03#1328: Add X-Cache-Hits - 13https://git.io/JvhVF [10:19:22] [02landing] 07hsl0 opened pull request 03#31: Add some Korean translations - 13https://git.io/JvhrP [10:20:37] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03Reception123-patch-1 [+0/-0/±3] 13https://git.io/Jvhr1 [10:20:38] [02miraheze/mw-config] 07Reception123 03921d99d - add GoogleDocs4MW extension [10:20:40] [02mw-config] 07Reception123 created branch 03Reception123-patch-1 - 13https://git.io/vbvb3 [10:24:39] [02mw-config] 07Reception123 opened pull request 03#2994: add GoogleDocs4MW extension T4620 - 13https://git.io/Jvhrd [10:26:36] [02mw-config] 07Reception123 synchronize pull request 03#2994: add GoogleDocs4MW extension T4620 - 13https://git.io/Jvhrd [10:26:38] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03Reception123-patch-1 [+0/-0/±1] 13https://git.io/Jvhrx [10:26:39] [02miraheze/mw-config] 07Reception123 03022bbb8 - typo [10:30:30] [02mw-config] 07Reception123 closed pull request 03#2994: add GoogleDocs4MW extension T4620 - 13https://git.io/Jvhrd [10:30:32] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/JvhoT [10:30:33] [02miraheze/mw-config] 07Reception123 03a5fd7cc - add GoogleDocs4MW extension T4620 (#2994) * add GoogleDocs4MW extension * typo [10:39:49] !log rebuildLocalisationCache.php on mw* [10:39:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [11:35:20] [02mw-config] 07Reception123 deleted branch 03Reception123-patch-1 - 13https://git.io/vbvb3 [11:35:21] [02miraheze/mw-config] 07Reception123 deleted branch 03Reception123-patch-1 [11:41:10] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_34 [+1/-0/±1] 13https://git.io/Jvh6t [11:41:12] [02miraheze/mediawiki] 07Reception123 030de8a3f - Install LanguageSelector extension [11:41:30] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03Reception123-patch-1 [+0/-0/±4] 13https://git.io/Jvh63 [11:41:32] [02miraheze/mw-config] 07Reception123 032de8ef1 - add LanguageSelector T5064 [11:41:33] [02mw-config] 07Reception123 created branch 03Reception123-patch-1 - 13https://git.io/vbvb3 [11:42:17] [02mw-config] 07Reception123 opened pull request 03#2995: add LanguageSelector T5064 - 13https://git.io/Jvh6Z [11:44:18] [02mw-config] 07Reception123 closed pull request 03#2995: add LanguageSelector T5064 - 13https://git.io/Jvh6Z [11:44:20] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/Jvh6B [11:44:21] [02miraheze/mw-config] 07Reception123 036f9eff7 - add LanguageSelector T5064 (#2995) [11:58:39] !log rebuildLocalisationCache.php on mw* [12:30:01] !log rebuildLocalisationCache.php on mw* [13:02:19] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03Reception123-patch-2 [+0/-0/±4] 13https://git.io/JvhPv [13:02:20] [02miraheze/mw-config] 07Reception123 037befd8e - add ForcePreview T5075 [13:02:22] [02mw-config] 07Reception123 created branch 03Reception123-patch-2 - 13https://git.io/vbvb3 [13:05:42] [02mw-config] 07Reception123 deleted branch 03Reception123-patch-1 - 13https://git.io/vbvb3 [13:05:43] [02miraheze/mw-config] 07Reception123 deleted branch 03Reception123-patch-1 [13:05:45] [02mw-config] 07Reception123 deleted branch 03Reception123-patch-2 - 13https://git.io/vbvb3 [13:05:46] [02miraheze/mw-config] 07Reception123 deleted branch 03Reception123-patch-2 [13:07:23] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03Reception123-patch-1 [+0/-0/±4] 13https://git.io/JvhPt [13:07:24] [02miraheze/mw-config] 07Reception123 03926b8e1 - Add ForcePreivew T5075 [13:07:26] [02mw-config] 07Reception123 created branch 03Reception123-patch-1 - 13https://git.io/vbvb3 [13:07:47] [02mw-config] 07Reception123 opened pull request 03#2996: Add ForcePreivew T5075 - 13https://git.io/JvhPq [13:34:48] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03Reception123-patch-1 [+0/-0/±1] 13https://git.io/JvhPQ [13:34:49] [02miraheze/mw-config] 07Reception123 03237dd5c - fix [13:34:51] [02mw-config] 07Reception123 synchronize pull request 03#2996: Add ForcePreivew T5075 - 13https://git.io/JvhPq [13:44:07] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_34 [+1/-0/±1] 13https://git.io/JvhXT [13:44:08] [02miraheze/mediawiki] 07Reception123 03c14f05f - Install ForcePreview [13:47:11] [02mw-config] 07Reception123 closed pull request 03#2996: Add ForcePreivew T5075 - 13https://git.io/JvhPq [13:47:12] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/JvhXq [13:47:14] [02miraheze/mw-config] 07Reception123 03209b04f - Add ForcePreivew T5075 (#2996) * Add ForcePreivew T5075 * fix [13:47:15] [02mw-config] 07Reception123 deleted branch 03Reception123-patch-1 - 13https://git.io/vbvb3 [13:47:17] [02miraheze/mw-config] 07Reception123 deleted branch 03Reception123-patch-1 [13:58:56] !log rebuildLocalisationCache.php on mw* [13:59:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [14:07:37] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_34 [+1/-0/±1] 13https://git.io/JvhXj [14:07:39] [02miraheze/mediawiki] 07Reception123 03771e602 - Install SlackNotifications [16:05:08] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvhyG [16:05:09] [02miraheze/services] 07MirahezeSSLBot 0333c5a8b - BOT: Updating services config for wikis [16:08:37] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03Reception123-patch-1 [+0/-0/±4] 13https://git.io/Jvhy8 [16:08:39] [02miraheze/mw-config] 07Reception123 03585ebd8 - add SlackNotifications T5231 [16:08:40] [02mw-config] 07Reception123 created branch 03Reception123-patch-1 - 13https://git.io/vbvb3 [16:09:14] [02mw-config] 07Reception123 opened pull request 03#2997: add SlackNotifications T5231 - 13https://git.io/Jvhy4 [16:09:18] ^ JohnLewis does this look ok? or is there something wrong? [16:11:34] Reception123: $wgSitename won't be set at that point, $wmgHostname would neither be set as well, and your wmgWikiHooks part needs to be the variable, on the top (no 'default' => array) [16:13:19] JohnLewis: ah ok so just $wgSlackIncomingWebhookUrl = "wmgWikiMirahezeDiscordHooks" like it's done at the top of LS? [16:13:24] and what about sitename and hostname? what do I do with those? [16:14:02] Guess you can't do it in LS then [16:14:34] JohnLewis: hm, guess for now I'll do it the old LocalExtensions way. And instead, should we maybe try to get the "non essential" stuff out of there? [16:14:52] Yes, but end goal is LE is gone [16:14:55] JohnLewis: I'd qualify https://github.com/miraheze/mw-config/blob/master/LocalExtensions.php#L18 as unessential [16:14:55] [ mw-config/LocalExtensions.php at master · miraheze/mw-config · GitHub ] - github.com [16:15:10] well the more we convert now the less work later [16:15:20] JohnLewis: though how could we get it gone if we've got stuff like this that can't be done in LS? [16:15:21] Anything that can be in LS, should be in LS [16:15:30] Alternative methods [16:15:47] yeah, after I install these extensions and clear Phabricator I'll look into moving stuff from LW and LE into LS [16:16:03] some of it was just there because it was more convenient, not because it was needed there [16:16:14] though some settings can't be done in LS as it seems to be the case here [16:17:07] They'll have to be done at top level [16:17:35] yeah guess that's the only way [16:24:56] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03Reception123-patch-1 [+0/-0/±2] 13https://git.io/JvhSt [16:24:58] [02miraheze/mw-config] 07Reception123 03908ae5b - change to LE [16:24:59] [02mw-config] 07Reception123 synchronize pull request 03#2997: add SlackNotifications T5231 - 13https://git.io/Jvhy4 [16:36:53] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03Reception123-patch-1 [+0/-0/±2] 13https://git.io/JvhSl [16:36:54] [02miraheze/puppet] 07Reception123 03cbb4ea0 - add slack webhooks [16:36:56] [02puppet] 07Reception123 created branch 03Reception123-patch-1 - 13https://git.io/vbiAS [16:37:13] [02puppet] 07Reception123 opened pull request 03#1329: add slack webhooks - 13https://git.io/JvhS8 [16:40:08] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvhSE [16:40:10] [02miraheze/services] 07MirahezeSSLBot 0382cff28 - BOT: Updating services config for wikis [16:47:12] ^ paladox mind reviewing please? [16:48:50] [02mw-config] 07paladox synchronize pull request 03#2997: add SlackNotifications T5231 - 13https://git.io/Jvhy4 [16:48:52] [02miraheze/mw-config] 07paladox pushed 031 commit to 03Reception123-patch-1 [+0/-0/±1] 13https://git.io/JvhSy [16:48:53] [02miraheze/mw-config] 07paladox 03be63b89 - Update LocalExtensions.php [16:49:14] paladox: https://github.com/miraheze/mw-config/compare/908ae5b526ee...be63b8943e6c#diff-3a52556e1811b5d47054493e1c7c00fcR877 - discordnotifs had it like I did it [16:49:15] [ Comparing 908ae5b526ee...be63b8943e6c · miraheze/mw-config · GitHub ] - github.com [16:49:31] huh? [16:50:26] paladox: in the DS notification settings it was $wgWikiUrl = $wgServer . '/w/'; [16:50:33] and you removed the w from slack [16:51:08] oh [16:51:13] is this the same author? [16:51:47] [02mw-config] 07paladox synchronize pull request 03#2997: add SlackNotifications T5231 - 13https://git.io/Jvhy4 [16:51:49] [02miraheze/mw-config] 07paladox pushed 031 commit to 03Reception123-patch-1 [+0/-0/±1] 13https://git.io/JvhSF [16:51:50] [02miraheze/mw-config] 07paladox 03468fea1 - Update LocalExtensions.php [16:52:19] paladox: I don't know, but I can't imagine it would change our site URL :P [16:53:17] Reception123 did you set mediawiki::wiki_slack_hooks_url: [] in the private repo? [16:54:05] paladox: I'll do that now :) [16:54:10] ok [16:57:18] # mw slack hook url (value is a array) [16:57:18] mediawiki::wiki_discord_hooks_url: [16:57:25] ^ paladox what else do I put? do we have a default? [16:57:35] no [16:57:41] so make it an array [16:57:46] mediawiki::wiki_discord_hooks_url: [] [16:57:49] i think that'll work [16:58:55] paladox: ok will do [16:59:28] !log added mediawiki::wiki_discord_hooks_url: [] to private hiera [16:59:30] paladox: ^ [16:59:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [16:59:43] [02puppet] 07paladox closed pull request 03#1329: add slack webhooks - 13https://git.io/JvhS8 [16:59:45] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/Jvh9T [16:59:46] [02miraheze/puppet] 07Reception123 0308fbcc7 - add slack webhooks (#1329) [16:59:48] [02puppet] 07paladox deleted branch 03Reception123-patch-1 - 13https://git.io/vbiAS [16:59:49] [02miraheze/puppet] 07paladox deleted branch 03Reception123-patch-1 [17:00:16] paladox: let's hope I don't win the Break Something Every Week award again :D [17:00:21] heh [17:00:27] paladox: is mw good to go? [17:00:40] no [17:00:51] you'll break something if you don't let the puppet part go first... [17:01:01] ok [17:03:51] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:04:13] PROBLEM - dbt1 Puppet on dbt1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:04:14] PROBLEM - mw5 Puppet on mw5 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:04:17] PROBLEM - mail1 Puppet on mail1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:04:17] PROBLEM - rdb2 Puppet on rdb2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:04:18] PROBLEM - mon1 Puppet on mon1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:04:23] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:04:28] PROBLEM - cloud1 Puppet on cloud1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:04:41] PROBLEM - gluster1 Puppet on gluster1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:04:46] PROBLEM - puppet2 Puppet on puppet2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:04:55] PROBLEM - mw6 Puppet on mw6 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:04:55] PROBLEM - cp6 Puppet on cp6 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:05:00] Reception123 you broke things [17:05:03] PROBLEM - cloud2 Puppet on cloud2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:05:06] PROBLEM - db7 Puppet on db7 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:05:14] PROBLEM - cp8 Puppet on cp8 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:05:19] PROBLEM - phab1 Puppet on phab1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:05:34] paladox: my feud with Puppet never ends [17:05:34] PROBLEM - bacula2 Puppet on bacula2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:05:39] PROBLEM - services2 Puppet on services2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:05:39] PROBLEM - rdb1 Puppet on rdb1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:05:41] paladox: though what could be wrong? I just copied the config [17:05:49] PROBLEM - services1 Puppet on services1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:06:30] you didn't add a space! [17:06:34] yaml is quite sensitive [17:06:57] fixed [17:07:00] yaml is a touchy [17:07:16] Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Function lookup() did not find a value for the name 'mediawiki::wiki_slack_hooks_url' on node mw4.miraheze.org [17:07:33] [02miraheze/mw-config] 07Reception123 pushed 032 commits to 03Reception123-patch-1 [+0/-0/±2] 13https://git.io/Jvh93 [17:07:35] [02miraheze/mw-config] 07Reception123 03d65eaa3 - disable "show IP" and email for slack [17:07:36] [02miraheze/mw-config] 07Reception123 034818d1c - space [17:07:38] [02mw-config] 07Reception123 synchronize pull request 03#2997: add SlackNotifications T5231 - 13https://git.io/Jvhy4 [17:07:55] you also typo'd [17:07:55] paladox: wow didn't think a space would annoy it that much [17:08:11] paladox: what did I write? [17:08:21] discord was in the name [17:08:25] should have been mediawiki::wiki_slack_hooks_url [17:08:38] ok new error [17:08:40] oh, I thought I changed that.. [17:08:40] * paladox fixes [17:09:12] should've picked the extension in need of SQL config first :P [17:10:08] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvh9s [17:10:10] [02miraheze/services] 07MirahezeSSLBot 03ac1692c - BOT: Updating services config for wikis [17:10:34] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://git.io/Jvh9G [17:10:35] [02miraheze/puppet] 07paladox 03cb6172b - PrivateSettings: Fix wiki_slack_hooks_url [17:10:37] [02puppet] 07paladox created branch 03paladox-patch-3 - 13https://git.io/vbiAS [17:10:38] [02puppet] 07paladox opened pull request 03#1330: PrivateSettings: Fix wiki_slack_hooks_url - 13https://git.io/Jvh9Z [17:10:48] actually [17:10:55] i could simplify that [17:11:55] !log rebuildlc on jobrunner1 [17:15:43] hope icinga-miraheze doesn't rat me out again this time xD [17:16:16] RECOVERY - mw6 Puppet on mw6 is OK: OK: Puppet is currently enabled, last run 8 seconds ago with 0 failures [17:17:12] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [17:17:27] RECOVERY - mw7 Puppet on mw7 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [17:17:29] [02puppet] 07paladox synchronize pull request 03#1330: PrivateSettings: Fix wiki_slack_hooks_url - 13https://git.io/Jvh9Z [17:17:30] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://git.io/Jvh9z [17:17:32] [02miraheze/puppet] 07paladox 03bfedb4a - Update PrivateSettings.php [17:17:35] [02puppet] 07paladox closed pull request 03#1330: PrivateSettings: Fix wiki_slack_hooks_url - 13https://git.io/Jvh9Z [17:17:37] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvh9g [17:17:38] [02miraheze/puppet] 07paladox 033c79408 - PrivateSettings: Fix wiki_slack_hooks_url (#1330) * PrivateSettings: Fix wiki_slack_hooks_url * Update PrivateSettings.php [17:17:40] [02puppet] 07paladox deleted branch 03paladox-patch-3 - 13https://git.io/vbiAS [17:17:41] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-3 [17:18:29] RECOVERY - mw5 Puppet on mw5 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [17:18:49] PROBLEM - cp7 Varnish Backends on cp7 is CRITICAL: 3 backends are down. mw5 mw6 mw7 [17:18:53] PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 3 backends are down. mw5 mw6 mw7 [17:19:05] PROBLEM - cp8 Varnish Backends on cp8 is CRITICAL: 3 backends are down. mw5 mw6 mw7 [17:20:22] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 3 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb [17:21:26] PROBLEM - cp6 Varnish Backends on cp6 is CRITICAL: 1 backends are down. mw7 [17:22:56] PROBLEM - cp6 Stunnel Http for mw4 on cp6 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [17:23:17] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 4 datacenters are down: 128.199.139.216/cpweb, 51.77.107.210/cpweb, 2001:41d0:800:1056::2/cpweb, 2607:5300:205:200::17f6/cpweb [17:23:41] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [17:24:34] RECOVERY - cp6 Varnish Backends on cp6 is OK: All 7 backends are healthy [17:25:02] RECOVERY - cp7 Varnish Backends on cp7 is OK: All 7 backends are healthy [17:25:12] RECOVERY - cp3 Varnish Backends on cp3 is OK: All 7 backends are healthy [17:25:26] RECOVERY - cp8 Varnish Backends on cp8 is OK: All 7 backends are healthy [17:25:49] RECOVERY - cp6 Stunnel Http for mw4 on cp6 is OK: HTTP OK: HTTP/1.1 200 OK - 15316 bytes in 0.005 second response time [17:26:01] [02mw-config] 07Reception123 closed pull request 03#2997: add SlackNotifications T5231 - 13https://git.io/Jvhy4 [17:26:02] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [17:26:03] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/Jvh9X [17:26:04] [02miraheze/mw-config] 07Reception123 0373a842d - add SlackNotifications T5231 (#2997) * add SlackNotifications T5231 * change to LE * Update LocalExtensions.php * Update LocalExtensions.php * disable "show IP" and email for slack * space Co-authored-by: paladox [17:30:23] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03Reception123-patch-1 [+0/-0/±1] 13https://git.io/Jvh99 [17:30:24] [02miraheze/mw-config] 07Reception123 0397ecd5e - mw-restricted for slacknotifications Needs settings [17:31:47] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvh97 [17:31:49] [02miraheze/mw-config] 07Reception123 03b7547db - restrict slacknotifications, needs settings [17:32:27] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_34 [+1/-0/±1] 13https://git.io/Jvh9d [17:32:28] [02miraheze/mediawiki] 07Reception123 03b8a9e64 - Install RegexFunctions [17:32:50] ^ Zppix have fun [17:32:53] and don't follow my lead [17:33:03] * Zppix hides [17:33:38] [02miraheze/mw-config] 07Pix1234 created branch 03RegexFunctions 13https://git.io/Jvh9N [17:33:40] [02mw-config] 07Pix1234 created branch 03RegexFunctions - 13https://git.io/vbvb3 [17:35:12] miraheze/mw-config/RegexFunctions/b7547db - Reception123 The build passed. https://travis-ci.org/miraheze/mw-config/builds/673841449 [17:35:12] [02mw-config] 07Reception123 deleted branch 03Reception123-patch-1 - 13https://git.io/vbvb3 [17:35:14] [02miraheze/mw-config] 07Reception123 deleted branch 03Reception123-patch-1 [17:36:35] [02miraheze/mw-config] 07Pix1234 pushed 031 commit to 03RegexFunctions [+0/-0/±1] 13https://git.io/Jvh9h [17:36:36] [02miraheze/mw-config] 07Pix1234 0337aaf41 - + RegexFunctions to ext-list [17:38:06] [02miraheze/mw-config] 07Pix1234 pushed 031 commit to 03RegexFunctions [+0/-0/±1] 13https://git.io/JvhHe [17:38:07] [02miraheze/mw-config] 07Pix1234 03fb5373e - + RegexFunctions to LocalExt.php [17:38:57] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvhHf [17:38:58] [02miraheze/puppet] 07paladox 03a33b5fb - Reduce command_processing_threads to 1 [17:40:08] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvhHU [17:40:10] [02miraheze/services] 07MirahezeSSLBot 032c80b46 - BOT: Updating services config for wikis [17:40:19] [02miraheze/mw-config] 07Pix1234 pushed 031 commit to 03RegexFunctions [+0/-0/±1] 13https://git.io/JvhHk [17:40:20] [02miraheze/mw-config] 07Pix1234 030469d5e - + regexfunctions [17:43:11] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 88 failures. Last run 5 minutes ago with 88 failures. Failed resources (up to 3 shown): File[spiral.wiki_private],File[permanentfuturelab.wiki],File[permanentfuturelab.wiki_private],File[private.revi.wiki] [17:43:30] [02miraheze/mw-config] 07Pix1234 pushed 031 commit to 03RegexFunctions [+0/-0/±1] 13https://git.io/JvhHm [17:43:32] [02miraheze/mw-config] 07Pix1234 039fa85b1 - + regexfunctions [17:43:54] [02mw-config] 07Pix1234 opened pull request 03#2998: +Regex functions ext - 13https://git.io/JvhHY [17:44:05] PROBLEM - mw5 Puppet on mw5 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 1 minute ago with 3 failures. Failed resources (up to 3 shown): File[wiki.valentinaproject.org],File[wiki.valentinaproject.org_private],File[wiki.dwplive.com],File[wiki.dwplive.com_private] [17:44:08] Zppix: why restricted? [17:44:20] Reception123: just until i test [17:44:30] ok fair enough [17:44:33] Zppix: looking at the rest [17:45:18] Zppix: LGTM if you've made sure there's no SQL changes and no mandatory settings [17:45:23] PROBLEM - mw6 Puppet on mw6 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 2 minutes ago with 10 failures. Failed resources (up to 3 shown): File[wiki.lbcomms.co.za],File[wiki.lbcomms.co.za_private],File[www.openonderwijs.org],File[www.openonderwijs.org_private] [17:45:41] Reception123: installation section just says to add wfLoad [17:46:08] Reception123: so its good? [17:46:54] PROBLEM - puppet2 Puppet on puppet2 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 5 minutes ago with 0 failures [17:47:04] Zppix: don't trust MW.org it's mislead me quite a few times, always check if there's an extension.json [17:47:04] Reception123: im good to merge then? [17:47:08] RECOVERY - mw5 Puppet on mw5 is OK: OK: Puppet is currently enabled, last run 44 seconds ago with 0 failures [17:47:15] there is [17:47:18] but f yes [17:47:19] *if [17:48:08] [02mw-config] 07Pix1234 closed pull request 03#2998: +Regex functions ext - 13https://git.io/JvhHY [17:48:10] [02miraheze/mw-config] 07Pix1234 pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/JvhHC [17:48:11] [02miraheze/mw-config] 07Pix1234 03c45382a - + Regex functions ext (#2998) * + RegexFunctions to ext-list * + RegexFunctions to LocalExt.php * + regexfunctions * + regexfunctions [17:48:20] RECOVERY - mw6 Puppet on mw6 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [17:50:01] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://git.io/JvhH0 [17:50:02] [02miraheze/puppet] 07paladox 03b86fefe - puppetdb: Increase command_processing_threads to 6 [17:50:04] [02puppet] 07paladox created branch 03paladox-patch-3 - 13https://git.io/vbiAS [17:50:05] [02puppet] 07paladox opened pull request 03#1331: puppetdb: Increase command_processing_threads to 6 - 13https://git.io/JvhHE [18:08:24] paladox: testwiki is broken [18:14:19] Examknow: known [18:14:49] ok [18:20:04] !log sudo -u www-data php ext*/ManageWiki/maint*/toggleExtension.php --wiki testwiki --disable regexfunctions [18:20:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [18:24:36] paladox: since when have we got a toggleExtensions.php? [18:24:47] since JohnLewis build it? [18:25:28] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://git.io/JvhQi [18:25:30] [02miraheze/puppet] 07paladox 03c3c39f5 - Update init.pp [18:25:31] [02puppet] 07paladox synchronize pull request 03#1331: puppetdb: Increase command_processing_threads to 6 - 13https://git.io/JvhHE [18:25:33] [02puppet] 07paladox edited pull request 03#1331: puppetdb: Increase command_processing_threads to 4 - 13https://git.io/JvhHE [18:25:38] [02puppet] 07paladox closed pull request 03#1331: puppetdb: Increase command_processing_threads to 4 - 13https://git.io/JvhHE [18:25:40] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvhQP [18:25:41] [02miraheze/puppet] 07paladox 035a90141 - puppetdb: Increase command_processing_threads to 4 (#1331) * puppetdb: Increase command_processing_threads to 6 * Update init.pp [18:25:43] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-3 [18:25:44] [02puppet] 07paladox deleted branch 03paladox-patch-3 - 13https://git.io/vbiAS [18:30:24] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/JvhQy [18:30:25] [02miraheze/mw-config] 07Reception123 037e3f7b0 - rm slacknotifications bug [18:32:00] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JvhQQ [18:32:01] [02miraheze/puppet] 07Reception123 03dbf2388 - rm slacknotifications [18:32:06] ^ paladox do I remove from private now or after it runs? [18:32:33] after it runs [18:32:37] ok [18:40:47] !log decrease puppet2 core by 1 (so 3 cores and 3 vcpu) [18:40:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [18:52:08] [02miraheze/mw-config] 07Pix1234 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvh78 [18:52:10] [02miraheze/mw-config] 07Pix1234 03d6c2a4c - replace wfLoad with require_once [18:53:24] [02miraheze/mw-config] 07Pix1234 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvh7R [18:53:26] [02miraheze/mw-config] 07Pix1234 0362c2d80 - change to regexfunctions.php [18:53:43] miraheze/mw-config/master/d6c2a4c - Devin/Zppix The build was broken. https://travis-ci.org/miraheze/mw-config/builds/673861887 [18:54:58] [02miraheze/mw-config] 07Pix1234 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvh7z [18:54:58] fixed [18:54:59] [02miraheze/mw-config] 07Pix1234 03d58a51e - fix [18:55:04] miraheze/mw-config/master/62c2d80 - Devin/Zppix The build was broken. https://travis-ci.org/miraheze/mw-config/builds/673862201 [18:56:05] Zppix: you broke AddThis! [18:56:33] And yes I know how you did it :p it's happened to me too [18:56:37] Ctrl+f ;) [18:56:47] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://git.io/Jvh72 [18:56:49] [02miraheze/puppet] 07paladox 036381d9f - Nutcracker: Add nice value to systemd unit [18:56:50] I assume the 503s are a known issue? [18:56:50] [02puppet] 07paladox created branch 03paladox-patch-3 - 13https://git.io/vbiAS [18:56:52] [02puppet] 07paladox opened pull request 03#1332: Nutcracker: Add nice value to systemd unit - 13https://git.io/Jvh7V [18:57:08] Reception123: can you fix [18:58:02] Reception123: you mean i broke addFist [18:58:26] [02miraheze/mw-config] 07Pix1234 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvh76 [18:58:28] [02miraheze/mw-config] 07Pix1234 0358167cb - Update LocalExtensions.php [18:59:04] Zppix: :p [18:59:18] Ctrl+f is usually your friend but sometimes it can turn into an enemy [19:00:08] Reception123: once again the software should be like, meh close enough i got you dude [19:02:22] Zppix: well my two greatest enemies here are Puppet and icinga-miraheze [19:02:27] They both want to see me fail [19:06:25] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-3 [+1/-0/±0] 13https://git.io/Jvh75 [19:06:26] [02miraheze/puppet] 07paladox 0379e864d - Create nutcracker-systemd-override.conf.erb [19:06:28] [02puppet] 07paladox synchronize pull request 03#1332: Nutcracker: Add nice value to systemd unit - 13https://git.io/Jvh7V [19:08:51] [02puppet] 07paladox closed pull request 03#1332: Nutcracker: Add nice value to systemd unit - 13https://git.io/Jvh7V [19:08:53] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/Jvh7b [19:08:54] [02miraheze/puppet] 07paladox 0313782d3 - Nutcracker: Add nice value to systemd unit (#1332) [19:08:56] [02puppet] 07paladox deleted branch 03paladox-patch-3 - 13https://git.io/vbiAS [19:08:57] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-3 [19:14:57] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvh5I [19:14:58] [02miraheze/puppet] 07paladox 0330a58e9 - Update init.pp [19:17:21] [02miraheze/mw-config] 07Pix1234 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvh5L [19:17:23] [02miraheze/mw-config] 07Pix1234 03333c168 - unrestrict RegexFunctions [19:50:05] [02miraheze/mw-config] 07Pix1234 deleted branch 03RegexFunctions [19:50:06] [02mw-config] 07Pix1234 deleted branch 03RegexFunctions - 13https://git.io/vbvb3 [20:13:08] [02miraheze/MirahezeDebug] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/Jvhdq [20:13:09] [02miraheze/MirahezeDebug] 07paladox 03c45c391 - Add new hosts, remove old hosts [20:13:11] [02MirahezeDebug] 07paladox created branch 03paladox-patch-1 - 13https://git.io/fpk7C [20:13:12] [02MirahezeDebug] 07paladox opened pull request 03#2: Add new hosts, remove old hosts - 13https://git.io/Jvhdm [20:14:16] [02MirahezeDebug] 07paladox synchronize pull request 03#2: Add new hosts, remove old hosts - 13https://git.io/Jvhdm [20:14:18] [02miraheze/MirahezeDebug] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvhdO [20:14:19] [02miraheze/MirahezeDebug] 07paladox 03a9fff15 - Update popup.html [20:14:25] [02MirahezeDebug] 07paladox closed pull request 03#2: Add new hosts, remove old hosts - 13https://git.io/Jvhdm [20:14:27] [02miraheze/MirahezeDebug] 07paladox pushed 033 commits to 03master [+0/-0/±3] 13https://git.io/Jvhd3 [20:14:28] [02miraheze/MirahezeDebug] 07paladox 03a88822d - Merge pull request #2 from miraheze/paladox-patch-1 Add new hosts, remove old hosts [20:14:30] [02miraheze/MirahezeDebug] 07paladox deleted branch 03paladox-patch-1 [20:14:31] [02MirahezeDebug] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/fpk7C [20:16:41] !log root@test2:/srv/mediawiki/w/cache# rm metawiki.json [20:16:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [21:19:01] [02miraheze/services] 07Pix1234 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvhFh [21:19:02] [02miraheze/services] 07Pix1234 03f8846e1 - update README to reflect current practices [21:31:03] [02dns] 07Pix1234 opened pull request 03#144: delete athenapedia.org as it no longer exists - 13https://git.io/Jvhb3 [21:31:27] [02miraheze/ssl] 07Pix1234 created branch 03athenapedia-removal 13https://git.io/JvhbZ [21:31:28] [02ssl] 07Pix1234 created branch 03athenapedia-removal - 13https://git.io/vxP9L [21:31:37] [02dns] 07paladox closed pull request 03#144: delete athenapedia.org as it no longer exists - 13https://git.io/Jvhb3 [21:31:39] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/Jvhbn [21:31:40] [02miraheze/dns] 07Pix1234 034910cbb - delete athenapedia.org as it no longer exists (#144) [21:31:47] [02miraheze/ssl] 07Pix1234 pushed 031 commit to 03athenapedia-removal [+0/-0/±1] 13https://git.io/Jvhbc [21:31:49] [02miraheze/ssl] 07Pix1234 03aa0fb33 - rm athenapedia.org [21:32:07] [02miraheze/ssl] 07Pix1234 pushed 031 commit to 03athenapedia-removal [+0/-1/±0] 13https://git.io/JvhbC [21:32:09] [02miraheze/ssl] 07Pix1234 03d0f991d - Delete athenapedia.org.crt [21:32:31] [02ssl] 07Pix1234 opened pull request 03#285: Athenapedia.org removal as wiki no longer exist - 13https://git.io/Jvhbl [21:32:39] [02ssl] 07Pix1234 closed pull request 03#285: Athenapedia.org removal as wiki no longer exist - 13https://git.io/Jvhbl [21:32:41] [02miraheze/ssl] 07Pix1234 pushed 031 commit to 03master [+0/-1/±1] 13https://git.io/Jvhb8 [21:32:42] [02miraheze/ssl] 07Pix1234 03ad0ca49 - Athenapedia.org removal as wiki no longer exist (#285) * rm athenapedia.org * Delete athenapedia.org.crt [21:32:51] [02ssl] 07Pix1234 deleted branch 03athenapedia-removal - 13https://git.io/vxP9L [21:32:53] [02miraheze/ssl] 07Pix1234 deleted branch 03athenapedia-removal [21:47:48] [02miraheze/mw-config] 07JohnFLewis pushed 031 commit to 03new-cache [+0/-0/±7] 13https://git.io/JvhbS [21:47:49] [02miraheze/mw-config] 07John Lewis 037a8de96 - new-cachev2 [21:47:51] [02mw-config] 07JohnFLewis created branch 03new-cache - 13https://git.io/vbvb3 [21:48:52] JohnLewis: i think you spelt 503 wrong when you created that branch :P [21:49:11] :P [21:56:21] [02miraheze/CreateWiki] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvhNL [21:56:23] [02miraheze/CreateWiki] 07JohnFLewis 033cecc8d - modify onSetupAfterCache for safety catches + last minute changes [21:59:07] [02miraheze/ManageWiki] 07JohnFLewis pushed 032 commits to 03master [+0/-0/±25] 13https://git.io/JvhNs [21:59:09] [02miraheze/ManageWiki] 07JohnFLewis 03349e48d - long standing fixes for perms and NS [21:59:10] [02miraheze/ManageWiki] 07JohnFLewis 038fa4f0d - Merge branch 'master' of github.com:/miraheze/ManageWiki [22:00:59] !log depool mw4 [22:01:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [22:13:24] [02miraheze/mw-config] 07JohnFLewis pushed 031 commit to 03new-cache [+0/-0/±1] 13https://git.io/JvhN1 [22:13:26] [02miraheze/mw-config] 07JohnFLewis 032a2e29e - fixes [22:13:46] ohhh [22:16:00] [02miraheze/mw-config] 07JohnFLewis pushed 031 commit to 03new-cache [+0/-0/±2] 13https://git.io/JvhNA [22:16:01] [02miraheze/mw-config] 07JohnFLewis 0397c5b13 - fixes [22:19:59] [02miraheze/mw-config] 07JohnFLewis pushed 031 commit to 03new-cache [+0/-0/±2] 13https://git.io/JvhAL [22:20:00] [02miraheze/mw-config] 07JohnFLewis 03fd816ba - fixes [22:26:03] !log MariaDB [phabricator_user]> update user_email set address = '' where address = 'John@miraheze.org'; [22:26:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [22:47:46] [02miraheze/ManageWiki] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvhAM [22:47:47] [02miraheze/ManageWiki] 07JohnFLewis 033c4186e - add servername i18n [22:50:02] !log repool mw4 temp [22:50:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [23:10:44] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JvhxK [23:10:45] [02miraheze/mw-config] 07paladox 03f407892 - Switch off cdb in wgManageWiki [23:10:47] [02mw-config] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vbvb3 [23:10:48] [02mw-config] 07paladox opened pull request 03#2999: Switch off cdb in wgManageWiki - 13https://git.io/Jvhx6 [23:11:56] [02mw-config] 07paladox closed pull request 03#2999: Switch off cdb in wgManageWiki - 13https://git.io/Jvhx6 [23:11:58] [02miraheze/mw-config] 07paladox pushed 031 commit to 03new-cache [+0/-0/±1] 13https://git.io/Jvhx1 [23:11:59] [02miraheze/mw-config] 07paladox 03b5d32c8 - Switch off cdb in wgManageWiki (#2999) [23:12:01] [02mw-config] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vbvb3 [23:12:02] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-2 [23:12:24] miraheze/mw-config/paladox-patch-2/f407892 - paladox The build has errored. https://travis-ci.org/miraheze/mw-config/builds/673920468 [23:30:08] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvhxj [23:30:10] [02miraheze/services] 07MirahezeSSLBot 0382e0689 - BOT: Updating services config for wikis [23:45:04]