[00:02:46] PROBLEM - mon1 Current Load on mon1 is WARNING: WARNING - load average: 3.69, 2.83, 2.33 [00:02:48] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 34.12, 29.28, 21.27 [00:03:37] PROBLEM - cloud1 Current Load on cloud1 is WARNING: WARNING - load average: 20.07, 21.68, 15.59 [00:04:47] RECOVERY - mon1 Current Load on mon1 is OK: OK - load average: 2.07, 2.50, 2.27 [00:05:37] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 10.75, 17.58, 14.82 [00:06:40] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.85, 21.80, 19.97 [00:08:35] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 12.62, 18.88, 19.13 [01:35:11] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkMuf [01:35:13] [02miraheze/services] 07MirahezeSSLBot 03c04f573 - BOT: Updating services config for wikis [01:59:14] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JkMgE [01:59:16] [02miraheze/mw-config] 07Universal-Omega 03c56a8f6 - Migrate Minerva skin to MWE [01:59:17] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-2 - 13https://git.io/vbvb3 [01:59:29] [02mw-config] 07Universal-Omega opened pull request 03#3478: Migrate Minerva skin to ManageWikiExtensions - 13https://git.io/JkMgg [02:02:35] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JkMgN [02:02:36] [02miraheze/mw-config] 07Universal-Omega 037595f00 - add vars [02:02:38] [02mw-config] 07Universal-Omega synchronize pull request 03#3478: Migrate Minerva skin to ManageWikiExtensions - 13https://git.io/JkMgg [02:06:36] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JkM20 [02:06:37] [02miraheze/mw-config] 07Universal-Omega 036d96201 - add [02:06:39] [02mw-config] 07Universal-Omega synchronize pull request 03#3478: Migrate Minerva skin to ManageWikiExtensions - 13https://git.io/JkMgg [03:18:50] miraheze/mw-config/Universal-Omega-patch-2/6d96201 - Universal Omega The build passed. https://travis-ci.org/miraheze/mw-config/builds/745784524 [03:50:20] RECOVERY - aman.awiki.org - reverse DNS on sslhost is OK: rDNS OK - aman.awiki.org reverse DNS resolves to cp6.miraheze.org [03:56:14] RECOVERY - tl.awiki.org - reverse DNS on sslhost is OK: rDNS OK - tl.awiki.org reverse DNS resolves to cp6.miraheze.org [08:22:47] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+1/-1/±1] 13https://git.io/JkDGc [08:22:49] [02miraheze/ssl] 07Reception123 031464c59 - replace battlecats.ml with battlecats.f5.si [08:25:29] [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/JkDGr [08:25:31] [02miraheze/dns] 07Reception123 0380a5455 - add lukaibrisimovic.org zone [08:34:38] PROBLEM - cp7 Puppet on cp7 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[battlecats.ml] [08:34:49] PROBLEM - jobrunner1 Puppet on jobrunner1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[battlecats.ml] [08:34:51] PROBLEM - cp6 Puppet on cp6 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[battlecats.ml] [08:35:13] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkDZ5 [08:35:14] [02miraheze/services] 07MirahezeSSLBot 03e15a7d6 - BOT: Updating services config for wikis [08:35:17] PROBLEM - mw5 Puppet on mw5 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[battlecats.ml] [08:35:31] PROBLEM - mw4 Puppet on mw4 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[battlecats.ml] [08:35:31] PROBLEM - cp9 Puppet on cp9 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[battlecats.ml] [08:35:37] PROBLEM - mw7 Puppet on mw7 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[battlecats.ml] [08:35:47] PROBLEM - mw6 Puppet on mw6 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[battlecats.ml] [08:35:55] PROBLEM - jobrunner2 Puppet on jobrunner2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[battlecats.ml] [08:38:19] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[battlecats.ml] [09:08:21] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkDlX [09:08:23] [02miraheze/ssl] 07Reception123 03f182dac - rm battlecats.ml redirect [09:12:45] RECOVERY - cp6 Puppet on cp6 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [09:12:49] RECOVERY - cp7 Puppet on cp7 is OK: OK: Puppet is currently enabled, last run 42 seconds ago with 0 failures [09:13:08] RECOVERY - mw4 Puppet on mw4 is OK: OK: Puppet is currently enabled, last run 33 seconds ago with 0 failures [09:13:13] RECOVERY - mw5 Puppet on mw5 is OK: OK: Puppet is currently enabled, last run 38 seconds ago with 0 failures [09:13:15] RECOVERY - mw7 Puppet on mw7 is OK: OK: Puppet is currently enabled, last run 54 seconds ago with 0 failures [09:13:19] RECOVERY - cp9 Puppet on cp9 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [09:13:38] RECOVERY - jobrunner2 Puppet on jobrunner2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:13:53] RECOVERY - mw6 Puppet on mw6 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:14:17] RECOVERY - jobrunner1 Puppet on jobrunner1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:16:33] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:36:09] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+1/-0/±2] 13https://git.io/JkDRQ [09:36:11] [02miraheze/ssl] 07Reception123 031cfb93a - add www.lukaibrisimovic.org cert [09:42:39] PROBLEM - jobrunner2 Puppet on jobrunner2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:43:15] PROBLEM - jobrunner1 Puppet on jobrunner1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:43:39] PROBLEM - mw6 Puppet on mw6 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:44:02] PROBLEM - cp7 Puppet on cp7 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:44:02] PROBLEM - cp6 Puppet on cp6 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:44:14] PROBLEM - cp9 Puppet on cp9 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:44:19] PROBLEM - mw5 Puppet on mw5 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:44:20] PROBLEM - mw4 Puppet on mw4 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:44:26] PROBLEM - mon1 Puppet on mon1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:44:28] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:44:44] PROBLEM - mw7 Puppet on mw7 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:47:26] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkDEC [09:47:27] [02miraheze/ssl] 07Reception123 032fcf972 - fix [09:54:36] PROBLEM - mon1 Check correctness of the icinga configuration on mon1 is CRITICAL: Icinga configuration contains errors [10:02:18] RECOVERY - mon1 Puppet on mon1 is OK: OK: Puppet is currently enabled, last run 14 seconds ago with 0 failures [11:07:40] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkD6a [11:07:41] [02miraheze/ssl] 07Reception123 03b19fe12 - fix [11:12:22] RECOVERY - jobrunner2 Puppet on jobrunner2 is OK: OK: Puppet is currently enabled, last run 2 seconds ago with 0 failures [11:12:31] RECOVERY - cp7 Puppet on cp7 is OK: OK: Puppet is currently enabled, last run 17 seconds ago with 0 failures [11:12:36] RECOVERY - mw5 Puppet on mw5 is OK: OK: Puppet is currently enabled, last run 11 seconds ago with 0 failures [11:12:39] RECOVERY - jobrunner1 Puppet on jobrunner1 is OK: OK: Puppet is currently enabled, last run 14 seconds ago with 0 failures [11:12:42] RECOVERY - mon1 Check correctness of the icinga configuration on mon1 is OK: Icinga configuration is correct [11:12:50] RECOVERY - cp6 Puppet on cp6 is OK: OK: Puppet is currently enabled, last run 38 seconds ago with 0 failures [11:13:15] RECOVERY - mw4 Puppet on mw4 is OK: OK: Puppet is currently enabled, last run 44 seconds ago with 0 failures [11:13:29] RECOVERY - mw6 Puppet on mw6 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [11:13:44] RECOVERY - cp9 Puppet on cp9 is OK: OK: Puppet is currently enabled, last run 57 seconds ago with 0 failures [11:14:14] RECOVERY - mw7 Puppet on mw7 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [11:26:06] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 57 seconds ago with 0 failures [12:24:49] PROBLEM - ns1 Current Load on ns1 is WARNING: WARNING - load average: 0.42, 1.87, 1.10 [12:26:49] RECOVERY - ns1 Current Load on ns1 is OK: OK - load average: 0.01, 0.84, 0.85 [14:02:12] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkyIf [14:02:13] [02miraheze/ssl] 07paladox 03367972c - Update www.lukaibrisimovic.org.crt [14:37:41] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.41, 19.45, 16.27 [14:39:39] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 17.95, 18.88, 16.44 [15:21:09] [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/JkyCI [15:21:10] [02miraheze/dns] 07Reception123 038d53bbc - add wooriwiki.com zone [15:23:02] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.54, 21.91, 17.61 [15:24:58] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.56, 20.93, 17.76 [15:25:19] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/JkyCX [15:25:21] [02miraheze/ssl] 07Reception123 038d508ea - add wooriwiki.com cert [15:26:55] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 29.46, 23.50, 19.02 [15:35:14] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkylZ [15:35:15] [02miraheze/services] 07MirahezeSSLBot 03250e0cf - BOT: Updating services config for wikis [15:42:42] PROBLEM - cloud1 Current Load on cloud1 is CRITICAL: CRITICAL - load average: 32.47, 22.31, 16.51 [15:44:25] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.92, 23.20, 23.17 [15:44:37] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 12.87, 18.84, 15.97 [15:56:25] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 11.23, 15.90, 19.52 [16:04:30] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.89, 21.03, 19.97 [16:06:25] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.44, 18.23, 19.07 [16:07:41] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jky0f [16:07:43] [02miraheze/mediawiki] 07paladox 035ea3b7f - Update DynamicPageList3 [16:10:32] PROBLEM - cloud1 Current Load on cloud1 is CRITICAL: CRITICAL - load average: 24.97, 18.62, 15.43 [16:14:25] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.20, 23.08, 20.55 [16:14:33] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 14.27, 18.01, 16.06 [16:16:25] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.64, 22.73, 20.71 [16:20:30] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.43, 23.18, 21.35 [16:24:26] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.73, 22.35, 21.60 [16:28:26] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 20.12, 19.29, 20.39 [17:00:28] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.94, 20.29, 17.58 [17:04:31] PROBLEM - cloud1 Current Load on cloud1 is CRITICAL: CRITICAL - load average: 24.86, 20.62, 15.49 [17:06:31] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 13.68, 18.24, 15.28 [17:08:26] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 17.70, 21.20, 19.77 [17:10:26] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.66, 21.99, 20.18 [17:14:25] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.62, 23.51, 21.38 [17:20:25] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.76, 22.19, 21.34 [17:21:17] PROBLEM - cloud1 Current Load on cloud1 is CRITICAL: CRITICAL - load average: 24.22, 17.21, 15.68 [17:23:11] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 19.42, 18.69, 16.46 [17:29:44] PROBLEM - test2 APT on test2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [17:33:58] RECOVERY - test2 APT on test2 is OK: APT OK: 14 packages available for upgrade (0 critical updates). [17:38:00] PROBLEM - test2 Puppet on test2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [17:39:57] PROBLEM - test2 Puppet on test2 is WARNING: WARNING: Puppet is currently disabled, message: Universal Omega, last run 2 days ago with 0 failures [17:40:26] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.25, 22.26, 23.20 [17:42:27] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.52, 23.25, 23.38 [17:43:10] [02miraheze/MirahezeMagic] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/Jky6X [17:43:11] [02miraheze/MirahezeMagic] 07Universal-Omega 038763774 - finish [17:43:13] [02MirahezeMagic] 07Universal-Omega synchronize pull request 03#151: Fix assignImportedEdits, createUsers, and fixUserIdRevision scripts to use actor; rewrite assignImportedEdits script so it will update edit count and revision links - 13https://git.io/JkXev [17:44:26] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.16, 21.50, 22.69 [17:50:25] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.36, 18.72, 20.69 [17:52:25] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.71, 19.45, 20.72 [17:52:49] [02miraheze/MirahezeMagic] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/Jkyio [17:52:50] [02miraheze/MirahezeMagic] 07Universal-Omega 03fefab1d - a little cleanup [17:52:52] [02MirahezeMagic] 07Universal-Omega synchronize pull request 03#151: Fix assignImportedEdits, createUsers, and fixUserIdRevision scripts to use actor; rewrite assignImportedEdits script so it will update edit count and revision links - 13https://git.io/JkXev [17:56:25] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 11.71, 16.81, 19.50 [17:57:49] PROBLEM - storytime.jdstroy.cf - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - storytime.jdstroy.cf All nameservers failed to answer the query. [18:03:06] PROBLEM - cloud1 Current Load on cloud1 is CRITICAL: CRITICAL - load average: 26.09, 19.85, 16.63 [18:05:00] RECOVERY - cloud1 Current Load on cloud1 is OK: OK - load average: 13.05, 17.52, 16.20 [18:10:00] [02MirahezeMagic] 07paladox closed pull request 03#151: Fix assignImportedEdits, createUsers, and fixUserIdRevision scripts to use actor; rewrite assignImportedEdits script so it will update edit count and revision links - 13https://git.io/JkXev [18:10:02] [02miraheze/MirahezeMagic] 07paladox pushed 0314 commits to 03master [+0/-0/±16] 13https://git.io/JkyXt [18:10:03] [02miraheze/MirahezeMagic] 07paladox 03d65b725 - Merge pull request #151 from miraheze/Universal-Omega-patch-1 Fix assignImportedEdits, createUsers, and fixUserIdRevision scripts to use actor; rewrite assignImportedEdits script so it will update edit count and revision links [18:10:05] [02miraheze/MirahezeMagic] 07paladox deleted branch 03Universal-Omega-patch-1 [18:10:06] [02MirahezeMagic] 07paladox deleted branch 03Universal-Omega-patch-1 - 13https://git.io/fQRGX [18:11:26] RECOVERY - storytime.jdstroy.cf - reverse DNS on sslhost is OK: rDNS OK - storytime.jdstroy.cf reverse DNS resolves to cp6.miraheze.org [18:11:34] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkyXc [18:11:36] [02miraheze/mediawiki] 07Universal-Omega 034067a98 - Updating DynamicPageList3 and Cosmos [18:11:37] [02mediawiki] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/vbL5b [18:13:29] [02mediawiki] 07Universal-Omega opened pull request 03#576: Updating Cosmos - 13https://git.io/JkyXg [18:15:50] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkyXD [18:15:51] [02miraheze/mediawiki] 07Universal-Omega 039530ba8 - Cosmos use master [18:15:53] [02mediawiki] 07Universal-Omega synchronize pull request 03#576: Updating Cosmos - 13https://git.io/JkyXg [18:18:21] [02mediawiki] 07Universal-Omega closed pull request 03#576: Updating Cosmos - 13https://git.io/JkyXg [18:18:22] [02miraheze/mediawiki] 07Universal-Omega pushed 033 commits to 03REL1_35 [+0/-0/±4] 13https://git.io/JkyXx [18:18:24] [02miraheze/mediawiki] 07Universal-Omega 03a5aabae - Merge pull request #576 from miraheze/Universal-Omega-patch-1 Updating Cosmos [18:18:28] [02mediawiki] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://git.io/vbL5b [18:18:29] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [18:29:40] PROBLEM - mw6 Puppet on mw6 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [18:30:47] PROBLEM - mw4 Puppet on mw4 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [18:30:50] PROBLEM - jobrunner1 Puppet on jobrunner1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [18:31:03] PROBLEM - jobrunner2 Puppet on jobrunner2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [18:31:20] Universal_Omega: lol, puppet didn't like that :D [18:31:25] I really wish we could fix that issue [18:31:35] PROBLEM - mw7 Puppet on mw7 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [18:31:39] PROBLEM - mw5 Puppet on mw5 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [18:31:43] weirdly enough, whenever someone updates a repo it fails, but once I tried deleting a repo and recloning it with Puppet and it worked [18:32:03] oh. [18:32:30] @Universal Omega yeah, it often happens with updates [18:32:33] and I don't get why [18:32:45] RECOVERY - mw4 Puppet on mw4 is OK: OK: Puppet is currently enabled, last run 10 seconds ago with 0 failures [18:32:49] RECOVERY - jobrunner1 Puppet on jobrunner1 is OK: OK: Puppet is currently enabled, last run 18 seconds ago with 0 failures [18:32:55] Should I revert then? [18:33:04] @Universal Omega no, not at all [18:33:05] RECOVERY - jobrunner2 Puppet on jobrunner2 is OK: OK: Puppet is currently enabled, last run 36 seconds ago with 0 failures [18:33:09] it will get fixed when it runs next [18:33:09] see ^ [18:33:20] I was just commenting on the general issue, it's not your fault, it almost always does that [18:33:31] Oh. I see what you mean now. [18:33:35] RECOVERY - mw7 Puppet on mw7 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:33:53] RECOVERY - mw5 Puppet on mw5 is OK: OK: Puppet is currently enabled, last run 59 seconds ago with 0 failures [18:33:54] RECOVERY - mw6 Puppet on mw6 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:36:29] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.68, 22.11, 20.20 [18:37:48] PROBLEM - miraheze.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - miraheze.ga All nameservers failed to answer the query. [18:38:16] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JkyMF [18:38:17] [02miraheze/mediawiki] 07Universal-Omega 032176f9e - Updating Cosmos [18:38:19] [02mediawiki] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/vbL5b [18:39:47] [02mediawiki] 07Universal-Omega opened pull request 03#577: Updating Cosmos - 13https://git.io/JkyDI [18:40:01] [02mediawiki] 07Universal-Omega closed pull request 03#577: Updating Cosmos - 13https://git.io/JkyDI [18:40:02] [02miraheze/mediawiki] 07Universal-Omega pushed 032 commits to 03REL1_35 [+0/-0/±2] 13https://git.io/JkyDt [18:40:04] [02miraheze/mediawiki] 07Universal-Omega 032bb685d - Merge pull request #577 from miraheze/Universal-Omega-patch-1 Updating Cosmos [18:40:05] [02mediawiki] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://git.io/vbL5b [18:40:07] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [18:44:25] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 17.24, 19.58, 19.97 [18:44:40] RECOVERY - miraheze.ga - reverse DNS on sslhost is OK: rDNS OK - miraheze.ga reverse DNS resolves to cp6.miraheze.org [18:46:13] [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/JkyDP [18:46:14] [02miraheze/dns] 07Reception123 032b8fe91 - add pornwiki.org zone [18:50:37] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.08, 25.38, 22.12 [18:52:31] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.91, 23.89, 21.91 [18:55:37] PROBLEM - test2 Puppet on test2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 13 seconds ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki config] [18:58:25] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.81, 18.15, 20.11 [19:00:35] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+1/-0/±2] 13https://git.io/JkySw [19:00:37] [02miraheze/ssl] 07Reception123 032287cf2 - add www.pornwiki.org cert [19:10:07] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.12, 22.10, 20.80 [19:15:54] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.87, 21.55, 21.25 [19:19:48] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.05, 18.31, 20.01 [19:25:02] miraheze/MirahezeMagic/Universal-Omega-patch-1/fefab1d - Universal Omega The build has errored. https://travis-ci.org/miraheze/MirahezeMagic/builds/745929335 [19:25:21] PROBLEM - test2 APT on test2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:25:40] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.48, 21.92, 20.86 [19:26:26] PROBLEM - test2 Current Load on test2 is CRITICAL: CRITICAL - load average: 4.17, 2.98, 1.52 [19:27:37] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.91, 22.54, 21.22 [19:29:27] RECOVERY - test2 APT on test2 is OK: APT OK: 14 packages available for upgrade (0 critical updates). [19:30:28] RECOVERY - test2 Current Load on test2 is OK: OK - load average: 1.00, 2.53, 1.73 [19:31:31] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.33, 22.94, 21.69 [19:33:27] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.67, 22.17, 21.59 [19:37:20] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.52, 23.53, 22.22 [19:39:08] PROBLEM - test2 APT on test2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:39:16] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.70, 21.83, 21.78 [19:39:32] PROBLEM - test2 MediaWiki Rendering on test2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:41:07] RECOVERY - test2 APT on test2 is OK: APT OK: 14 packages available for upgrade (0 critical updates). [19:41:13] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.36, 23.02, 22.21 [19:41:27] RECOVERY - test2 MediaWiki Rendering on test2 is OK: HTTP OK: HTTP/1.1 200 OK - 19613 bytes in 0.187 second response time [19:45:08] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.04, 21.56, 21.81 [19:54:52] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.78, 22.96, 21.98 [19:56:48] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.01, 20.48, 21.16 [19:58:45] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 12.16, 17.68, 20.06 [20:13:25] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.36, 24.16, 21.30 [20:15:22] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.29, 21.16, 20.57 [20:17:19] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 19.91, 20.17, 20.22 [20:23:18] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.79, 19.94, 19.91 [20:23:47] PROBLEM - es.publictestwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'publictestwiki.com' expires in 15 day(s) (Fri 11 Dec 2020 20:20:15 GMT +0000). [20:24:25] PROBLEM - www.publictestwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'publictestwiki.com' expires in 15 day(s) (Fri 11 Dec 2020 20:20:15 GMT +0000). [20:25:16] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 17.43, 18.58, 19.40 [20:26:16] PROBLEM - factorio-mods.tk - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - factorio-mods.tk All nameservers failed to answer the query. [20:27:42] PROBLEM - publictestwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'publictestwiki.com' expires in 15 day(s) (Fri 11 Dec 2020 20:20:15 GMT +0000). [20:31:57] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkyNO [20:31:58] [02miraheze/ssl] 07MirahezeSSLBot 035de1bf6 - Bot: Update SSL cert for publictestwiki.com [20:32:58] RECOVERY - factorio-mods.tk - reverse DNS on sslhost is OK: rDNS OK - factorio-mods.tk reverse DNS resolves to cp6.miraheze.org [20:35:15] !log sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --report=1 --wiki minelandwiki /home/reception/mineland.xml [20:35:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:44:17] RECOVERY - es.publictestwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'publictestwiki.com' will expire on Tue 23 Feb 2021 19:31:50 GMT +0000. [20:44:43] RECOVERY - www.publictestwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'publictestwiki.com' will expire on Tue 23 Feb 2021 19:31:50 GMT +0000. [20:47:55] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.45, 21.39, 19.60 [20:47:55] RECOVERY - publictestwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'publictestwiki.com' will expire on Tue 23 Feb 2021 19:31:50 GMT +0000. [20:49:51] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.72, 21.10, 19.71 [20:51:47] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 29.90, 24.93, 21.32 [20:55:41] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.18, 21.28, 20.77 [20:57:37] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.71, 19.20, 20.08 [21:07:54] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 14.60, 8.24, 4.95 [21:09:23] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 29.00, 26.34, 22.65 [21:09:51] PROBLEM - cp7 Current Load on cp7 is WARNING: WARNING - load average: 6.68, 7.55, 5.10 [21:11:47] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 1.89, 5.45, 4.63 [21:13:15] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.74, 23.98, 22.43 [21:19:12] PROBLEM - graylog1 Current Load on graylog1 is WARNING: WARNING - load average: 3.69, 2.89, 1.97 [21:21:09] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.12, 21.80, 21.70 [21:21:16] PROBLEM - graylog1 Current Load on graylog1 is CRITICAL: CRITICAL - load average: 4.31, 3.32, 2.25 [21:22:16] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_35 [+0/-0/±1] 13https://git.io/Jkyhp [21:22:17] [02miraheze/mediawiki] 07paladox 032b44cd4 - Update MinervaNeue [21:23:07] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.93, 21.27, 21.54 [21:25:05] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 35.06, 26.47, 23.38 [21:25:16] RECOVERY - graylog1 Current Load on graylog1 is OK: OK - load average: 2.98, 3.15, 2.42 [21:27:03] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 17.51, 23.21, 22.59 [21:30:16] PROBLEM - mw4 Puppet on mw4 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [21:30:59] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 34.40, 23.70, 22.55 [21:31:07] PROBLEM - mw6 Puppet on mw6 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [21:31:11] PROBLEM - mw7 Puppet on mw7 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [21:31:47] PROBLEM - mw5 Puppet on mw5 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [21:33:06] RECOVERY - mw6 Puppet on mw6 is OK: OK: Puppet is currently enabled, last run 32 seconds ago with 0 failures [21:33:08] RECOVERY - mw7 Puppet on mw7 is OK: OK: Puppet is currently enabled, last run 37 seconds ago with 0 failures [21:33:46] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 8.27, 6.19, 4.39 [21:33:46] RECOVERY - mw5 Puppet on mw5 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [21:34:14] RECOVERY - mw4 Puppet on mw4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [21:34:52] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 16.12, 22.09, 22.45 [21:35:45] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 2.54, 4.55, 4.00 [21:42:40] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.58, 17.48, 19.79 [22:07:28] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 8 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.77.107.210/cpweb, 51.89.160.142/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [22:07:46] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 10.30, 6.26, 3.64 [22:08:33] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.38, 21.74, 18.81 [22:09:27] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [22:09:45] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 3.50, 5.06, 3.52 [22:10:33] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.94, 22.94, 19.57 [22:12:33] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.16, 22.27, 19.68 [22:16:33] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 13.48, 19.30, 19.21 [22:25:13] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkSTm [22:25:14] [02miraheze/services] 07MirahezeSSLBot 031bb313d - BOT: Updating services config for wikis [23:06:22] https://meta.miraheze.org/w/index.php?title=Meta:Administrators%27_noticeboard&oldid=prev&diff=146614&diffmode=source [23:06:23] [ Difference between revisions of "Meta:Administrators' noticeboard" - Miraheze Meta ] - meta.miraheze.org [23:06:31] Is it just me or did this mention me [23:06:43] I think because he transcluded the interface admins list [23:15:02] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[ufw-logging-low],Exec[ufw-allow-tcp-from-any-to-any-port-22] [23:23:02] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:25:25] PROBLEM - test2 Current Load on test2 is CRITICAL: CRITICAL - load average: 7.95, 5.44, 2.44 [23:25:55] PROBLEM - test2 APT on test2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:29:58] RECOVERY - test2 APT on test2 is OK: APT OK: 14 packages available for upgrade (0 critical updates). [23:31:25] PROBLEM - test2 Current Load on test2 is WARNING: WARNING - load average: 1.72, 3.99, 2.93 [23:33:24] RECOVERY - test2 Current Load on test2 is OK: OK - load average: 0.25, 2.68, 2.57 [23:44:11] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.64, 19.51, 17.40 [23:46:08] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.79, 17.84, 17.04