[00:03:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 38.77, 32.89, 25.85 [00:11:03] PROBLEM - wiki.xfbs.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.xfbs.net could not be found [00:18:02] RECOVERY - wiki.xfbs.net - reverse DNS on sslhost is OK: rDNS OK - wiki.xfbs.net reverse DNS resolves to cp7.miraheze.org [00:37:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 14.32, 19.64, 22.97 [00:49:45] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.38, 17.57, 20.10 [00:53:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.64, 20.61, 20.77 [00:55:44] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 18.52, 19.26, 20.23 [01:01:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.92, 23.86, 21.73 [01:03:46] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.59, 22.79, 21.61 [01:11:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 42.31, 26.60, 22.99 [01:13:46] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.94, 23.92, 22.47 [01:19:47] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 33.93, 25.11, 22.90 [01:27:48] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.27, 22.46, 22.96 [01:31:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 30.46, 23.51, 23.05 [01:33:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 16.31, 20.94, 22.21 [01:41:47] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 32.32, 23.20, 22.02 [01:43:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.22, 22.46, 21.92 [01:44:06] PROBLEM - ns1 APT on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [01:44:43] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [01:45:13] PROBLEM - ns1 Current Load on ns1 is CRITICAL: CRITICAL - load average: 20.62, 12.20, 5.48 [01:46:04] RECOVERY - ns1 APT on ns1 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [01:46:39] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 15 minutes ago with 0 failures [01:49:44] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.74, 18.07, 20.23 [01:55:14] RECOVERY - ns1 Current Load on ns1 is OK: OK - load average: 0.00, 0.29, 1.69 [02:19:49] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.54, 19.86, 18.72 [02:21:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.76, 22.05, 19.63 [02:23:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.00, 21.89, 19.89 [02:35:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.26, 24.47, 21.72 [02:37:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.26, 22.77, 21.44 [02:41:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.47, 22.14, 21.35 [02:47:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.53, 23.24, 22.50 [02:51:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.92, 23.18, 22.49 [02:53:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.47, 21.77, 22.07 [02:59:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.43, 21.11, 21.38 [03:01:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.72, 21.05, 21.32 [03:15:44] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 17.19, 19.25, 20.25 [03:19:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.08, 19.97, 20.42 [03:23:44] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 18.65, 20.09, 20.39 [03:33:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.66, 20.83, 20.01 [03:35:46] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 20.02, 20.34, 19.93 [03:51:30] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.99, 21.42, 20.27 [03:55:23] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.92, 19.28, 19.81 [04:01:13] PROBLEM - cp9 NTP time on cp9 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [04:03:16] RECOVERY - cp9 NTP time on cp9 is OK: NTP OK: Offset -0.006172150373 secs [04:11:41] PROBLEM - cp9 NTP time on cp9 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [04:13:37] RECOVERY - cp9 NTP time on cp9 is OK: NTP OK: Offset -0.004600822926 secs [04:22:36] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.25, 20.85, 20.12 [04:28:24] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.57, 18.52, 19.47 [04:30:19] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJASQ [04:30:20] [02miraheze/services] 07MirahezeSSLBot 0309d4299 - BOT: Updating services config for wikis [04:34:14] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.02, 21.98, 20.52 [04:36:11] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.52, 21.09, 20.37 [04:38:07] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 18.46, 20.13, 20.10 [04:44:15] PROBLEM - wiki.valentinaproject.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.valentinaproject.org could not be found [04:51:04] RECOVERY - wiki.valentinaproject.org - reverse DNS on sslhost is OK: rDNS OK - wiki.valentinaproject.org reverse DNS resolves to cp6.miraheze.org [04:53:46] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.14, 24.78, 22.26 [04:55:45] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.97, 22.75, 21.80 [04:59:46] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 13.52, 17.69, 19.93 [05:14:24] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.39, 23.23, 21.35 [05:16:20] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 17.27, 20.99, 20.75 [05:18:16] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 18.59, 20.00, 20.39 [05:42:41] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.86, 20.41, 19.66 [05:44:37] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.62, 19.01, 19.24 [06:02:04] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.26, 20.63, 19.14 [06:04:00] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 20.14, 20.14, 19.13 [06:05:36] PROBLEM - wiki.xysspon.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.xysspon.com could not be found [06:12:33] RECOVERY - wiki.xysspon.com - reverse DNS on sslhost is OK: rDNS OK - wiki.xysspon.com reverse DNS resolves to cp6.miraheze.org [06:13:52] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 29.70, 23.24, 20.44 [06:15:48] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.17, 22.07, 20.34 [06:19:44] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 19.33, 20.35, 20.01 [06:23:45] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.29, 20.87, 20.41 [06:25:47] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.64, 22.62, 21.04 [06:27:45] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.78, 21.26, 20.72 [06:29:45] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.51, 19.59, 20.18 [06:33:46] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.82, 22.89, 21.60 [06:41:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 29.69, 24.61, 22.43 [06:45:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.25, 22.81, 22.23 [06:49:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.98, 23.04, 22.27 [06:55:46] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 16.60, 21.62, 22.17 [07:01:46] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 32.62, 23.67, 22.48 [07:05:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.60, 22.70, 22.41 [07:11:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.31, 24.60, 23.15 [07:13:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.24, 22.89, 22.70 [07:21:42] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 9.78, 8.38, 4.90 [07:21:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 28.43, 25.00, 23.47 [07:23:42] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 3.62, 6.30, 4.55 [07:23:45] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.25, 23.46, 23.07 [07:27:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.10, 21.20, 22.09 [07:29:46] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.50, 22.69, 22.62 [07:31:47] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.57, 23.47, 22.90 [07:33:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 17.87, 21.27, 22.17 [07:49:45] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.22, 17.69, 19.94 [08:25:57] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.55, 20.68, 19.87 [08:27:54] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.28, 22.37, 20.57 [08:29:52] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.36, 20.98, 20.30 [08:37:44] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 19.63, 20.36, 20.25 [09:02:12] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 30.41, 22.04, 20.14 [09:04:08] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.16, 21.20, 20.04 [09:06:06] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.81, 21.79, 20.29 [09:06:09] icinga-miraheze: I ain't got time for you. I got non stop emails and texts today. [09:09:42] PROBLEM - cp7 Current Load on cp7 is WARNING: WARNING - load average: 5.79, 7.43, 4.51 [09:11:42] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 3.66, 6.05, 4.35 [09:13:50] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 17.60, 22.85, 22.29 [09:21:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.68, 22.61, 22.06 [09:23:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.82, 23.44, 22.53 [09:31:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.29, 21.86, 21.74 [09:33:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.30, 22.49, 22.02 [09:43:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.73, 23.57, 22.48 [09:45:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 16.59, 20.74, 21.57 [09:51:47] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.32, 22.76, 21.95 [09:53:45] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.17, 22.33, 21.95 [09:57:44] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.30, 18.06, 20.22 [10:13:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.30, 20.79, 20.32 [10:15:44] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 19.06, 20.07, 20.11 [10:19:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.36, 21.09, 20.54 [10:23:44] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.47, 19.21, 20.04 [10:33:48] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.10, 22.94, 21.34 [10:37:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.21, 23.53, 21.81 [10:39:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.49, 22.25, 21.53 [10:49:46] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 14.87, 18.17, 19.90 [10:53:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.66, 21.74, 20.96 [11:01:46] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.83, 22.03, 21.30 [11:09:45] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.57, 23.83, 22.68 [11:11:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.09, 24.67, 23.10 [11:13:46] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.29, 23.46, 22.84 [11:15:46] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 32.06, 25.56, 23.59 [11:17:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 17.77, 22.22, 22.59 [11:25:44] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 18.94, 17.85, 20.23 [11:31:20] Huh, GSs don't have unblock self? :D [11:31:42] Can you please unblock me on TestWiki? RhinosF1 [11:31:51] Ye [11:32:57] Thanks [11:33:13] Sorted [11:33:22] Gave you admin as admin has it afaik [11:33:33] Yes, have [11:33:45] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 16.03, 19.79, 20.48 [11:35:45] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.94, 18.78, 20.01 [11:39:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 31.33, 24.15, 21.84 [11:45:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.51, 23.04, 22.22 [11:51:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.19, 23.48, 22.47 [11:53:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.80, 22.55, 22.22 [11:55:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 28.15, 24.55, 22.99 [12:01:08] .op [12:01:08] Attempting to OP... [12:03:09] ^ testing something [12:03:45] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.07, 23.48, 23.34 [12:05:46] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 34.11, 25.71, 24.05 [12:09:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.02, 23.54, 23.57 [12:11:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.28, 25.65, 24.37 [12:19:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.85, 23.24, 23.88 [12:21:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.96, 24.55, 24.27 [12:23:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.13, 23.04, 23.74 [12:31:53] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 36.67, 26.40, 24.39 [12:33:49] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.24, 23.93, 23.78 [12:34:06] PROBLEM - bn.gyaanipedia.co.in - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for bn.gyaanipedia.co.in could not be found [12:34:08] PROBLEM - fa.gyaanipedia.co.in - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for fa.gyaanipedia.co.in could not be found [12:41:00] RECOVERY - bn.gyaanipedia.co.in - reverse DNS on sslhost is OK: rDNS OK - bn.gyaanipedia.co.in reverse DNS resolves to cp6.miraheze.org [12:41:02] RECOVERY - fa.gyaanipedia.co.in - reverse DNS on sslhost is OK: rDNS OK - fa.gyaanipedia.co.in reverse DNS resolves to cp6.miraheze.org [12:41:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.11, 23.48, 23.32 [12:42:53] RhinosF1 [12:42:55] https://horriblevyonderswiki.miraheze.org/wiki/Special:Log?type=block&user=&page=&wpdate=&tagfilter= [12:42:57] [ Block log - Horrible Vyonders Wiki ] - horriblevyonderswiki.miraheze.org [12:43:05] Do you getting error? [12:43:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.58, 22.03, 22.82 [12:46:23] No @MrJaroslavik: what you seeing? [12:46:43] [6ee2ccba5077f6b8ea630aeb] 2020-08-20 12:46:32: Fatal exception of type "MWException" [12:47:33] Incognito okay [12:48:06] Hmm, must be a user setting then [12:48:16] File a task and someone'll look [12:51:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.98, 21.31, 21.72 [12:53:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.30, 18.94, 20.82 [12:59:46] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.53, 18.93, 20.31 [13:03:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.56, 20.69, 20.77 [13:07:45] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 17.65, 18.85, 20.03 [13:13:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 28.34, 24.19, 21.86 [13:15:45] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.82, 23.00, 21.70 [13:21:46] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.23, 22.11, 21.48 [13:22:11] Someone set a block expiry to 31 December 9999, and if your time zone pushes that past midnight, MediaWiki throws a fit [13:23:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.52, 22.14, 21.55 [13:27:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.91, 22.60, 21.75 [13:29:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.76, 21.95, 21.57 [13:41:47] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.76, 21.23, 21.51 [13:45:47] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.11, 23.62, 22.55 [13:47:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.44, 23.86, 22.76 [13:51:43] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 18.94, 11.38, 6.21 [13:55:42] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 2.01, 6.31, 5.32 [13:59:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.22, 20.84, 23.16 [14:01:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 30.46, 24.29, 24.12 [14:09:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.10, 23.42, 23.87 [14:11:46] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 30.15, 26.05, 24.78 [14:36:03] [02miraheze/dns] 07Southparkfan pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJxVt [14:36:05] [02miraheze/dns] 07Southparkfan 0392b9cd0 - Depool cp9 for investigation on T6056 [14:40:26] PROBLEM - cp7 Current Load on cp7 is WARNING: WARNING - load average: 6.49, 7.32, 5.47 [14:41:21] PROBLEM - www.dariawiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.dariawiki.org could not be found [14:42:39] hm [14:44:13] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 8.45, 7.73, 6.02 [14:46:07] PROBLEM - cp7 Current Load on cp7 is WARNING: WARNING - load average: 7.81, 7.48, 6.11 [14:46:58] !log disable puppet on cp9 [14:47:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:48:00] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 6.37, 6.80, 6.01 [14:48:04] RECOVERY - www.dariawiki.org - reverse DNS on sslhost is OK: rDNS OK - www.dariawiki.org reverse DNS resolves to cp7.miraheze.org [14:50:42] PROBLEM - cp9 Puppet on cp9 is WARNING: WARNING: Puppet is currently disabled, message: SPF, last run 8 minutes ago with 0 failures [14:53:48] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 26.90, 11.80, 7.87 [14:58:21] !log install wrk on cp6 [14:58:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:59:42] PROBLEM - cp7 Current Load on cp7 is WARNING: WARNING - load average: 5.49, 7.51, 7.24 [15:01:42] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 10.85, 9.57, 8.05 [15:04:56] PROBLEM - ns1 Current Load on ns1 is WARNING: WARNING - load average: 0.80, 1.81, 1.19 [15:05:28] Hello sorcerer! If you have any questions, feel free to ask and someone should answer soon. [15:06:56] RECOVERY - ns1 Current Load on ns1 is OK: OK - load average: 0.40, 0.95, 0.97 [15:07:42] PROBLEM - cp7 Current Load on cp7 is WARNING: WARNING - load average: 5.53, 7.47, 7.60 [15:14:07] PROBLEM - meregos.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for meregos.com could not be found [15:14:08] PROBLEM - mai.gyaanipedia.co.in - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for mai.gyaanipedia.co.in could not be found [15:14:16] PROBLEM - cp9 HTTP 4xx/5xx ERROR Rate on cp9 is CRITICAL: CRITICAL - NGINX Error Rate is 94% [15:16:16] RECOVERY - cp9 HTTP 4xx/5xx ERROR Rate on cp9 is OK: OK - NGINX Error Rate is 23% [15:17:42] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 14.62, 10.32, 8.44 [15:20:17] PROBLEM - cp9 HTTP 4xx/5xx ERROR Rate on cp9 is WARNING: WARNING - NGINX Error Rate is 41% [15:20:49] RECOVERY - meregos.com - reverse DNS on sslhost is OK: rDNS OK - meregos.com reverse DNS resolves to cp6.miraheze.org [15:20:51] RECOVERY - mai.gyaanipedia.co.in - reverse DNS on sslhost is OK: rDNS OK - mai.gyaanipedia.co.in reverse DNS resolves to cp6.miraheze.org [15:22:17] RECOVERY - cp9 HTTP 4xx/5xx ERROR Rate on cp9 is OK: OK - NGINX Error Rate is 18% [15:29:48] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJp72 [15:29:49] [02miraheze/puppet] 07Southparkfan 03ed36a20 - nginx: rate limit requests per IP per second at cache proxy level [15:30:43] RECOVERY - cp9 Puppet on cp9 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:30:48] !log enable puppet on cp9 [15:31:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:33:04] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 4 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 51.222.27.129/cpweb, 2607:5300:205:200::2ac4/cpweb [15:33:22] PROBLEM - gluster1 Current Load on gluster1 is CRITICAL: CRITICAL - load average: 8.39, 6.82, 5.68 [15:33:23] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJpbz [15:33:24] [02miraheze/puppet] 07Southparkfan 031f99d00 - nginx: remove delay=8 Shouldn't be in the file. [15:33:27] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 4 datacenters are down: 51.77.107.210/cpweb, 2001:41d0:800:1056::2/cpweb, 2001:41d0:800:105a::10/cpweb, 2607:5300:205:200::2ac4/cpweb [15:34:44] PROBLEM - cp9 Puppet on cp9 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[nginx-syntax] [15:35:03] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [15:35:23] PROBLEM - gluster1 Current Load on gluster1 is WARNING: WARNING - load average: 7.14, 6.76, 5.79 [15:35:25] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [15:35:34] PROBLEM - cp6 Puppet on cp6 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[nginx-syntax] [15:35:44] PROBLEM - cp7 Puppet on cp7 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[nginx-syntax] [15:35:45] SPF|Cloud: only thing I can think is Parsoid. Anyone tested a VE edit. Also ^ [15:36:13] We seem super slow [15:36:47] RECOVERY - cp9 Puppet on cp9 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:36:52] [02miraheze/dns] 07Southparkfan pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJppP [15:36:54] [02miraheze/dns] 07Southparkfan 03ade2da9 - Repool sg [15:37:01] Hmm I got a thought, I might know why we slow [15:37:14] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[nginx-syntax] [15:37:21] RECOVERY - gluster1 Current Load on gluster1 is OK: OK - load average: 6.41, 6.68, 5.88 [15:37:44] !log remove wrk from cp6, repool cp9 [15:37:45] I wonder if how many requests with logos and all are made [15:38:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:43:32] RECOVERY - cp6 Puppet on cp6 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:43:43] RECOVERY - cp7 Puppet on cp7 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:45:09] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 48 seconds ago with 0 failures [15:47:43] PROBLEM - cp7 Current Load on cp7 is WARNING: WARNING - load average: 3.27, 4.91, 7.68 [15:47:53] PROBLEM - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is WARNING: WARNING - NGINX Error Rate is 45% [15:49:53] RECOVERY - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is OK: OK - NGINX Error Rate is 30% [15:53:42] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 3.47, 4.25, 6.49 [15:53:53] PROBLEM - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is CRITICAL: CRITICAL - NGINX Error Rate is 89% [15:54:04] SPF|Cloud: ^ [15:54:20] could it be...? [15:54:30] SPF|Cloud: logs? [15:54:43] likely someone who is being rate limited [15:54:45] PROBLEM - cp7 HTTP 4xx/5xx ERROR Rate on cp7 is WARNING: WARNING - NGINX Error Rate is 41% [15:55:05] I'm even seeing that on cp9 [15:55:53] RECOVERY - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is OK: OK - NGINX Error Rate is 14% [15:56:44] RECOVERY - cp7 HTTP 4xx/5xx ERROR Rate on cp7 is OK: OK - NGINX Error Rate is 19% [15:56:45] SPF|Cloud: who? [15:56:56] idk [15:57:02] somebody, either a person or bot [15:57:04] somebody, either a person or bot [15:57:24] SPF|Cloud: can you paste privately an example blocked request? [16:10:20] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJhRM [16:10:21] [02miraheze/puppet] 07Southparkfan 03e2cefb0 - Increase rate limits [16:26:20] PROBLEM - db7 Check MariaDB Replication on db7 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 181s [16:30:22] RECOVERY - db7 Check MariaDB Replication on db7 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 1s [16:30:30] afk for an hour or so [17:10:19] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJhXl [17:10:21] [02miraheze/services] 07MirahezeSSLBot 0311b9e43 - BOT: Updating services config for wikis [17:19:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.76, 22.10, 23.77 [17:21:46] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 36.87, 27.09, 25.35 [17:32:27] looking into cloud2 alerts [17:35:10] Ty [17:38:21] it's gluster1 being the culprit [17:39:21] https://grafana.miraheze.org/d/W9MIkA7iz/miraheze-cluster?viewPanel=287&orgId=1&from=1597685942508&to=1597945082508&var-job=node&var-node=cloud2.miraheze.org&var-port=9100 [17:39:22] [ Grafana ] - grafana.miraheze.org [17:39:38] aligns with https://grafana.miraheze.org/d/W9MIkA7iz/miraheze-cluster?viewPanel=287&orgId=1&from=1597685977548&to=1597945117548&var-job=node&var-node=gluster1.miraheze.org&var-port=9100 [17:39:39] [ Grafana ] - grafana.miraheze.org [17:40:03] Ack [17:43:15] ...and that looks like to be caused by bacula-fd [17:45:09] SPF|Cloud: I think paladox set that up because gluster was ooming during backups to ensure we had them [17:45:30] And will probably know / have something to say [17:45:31] but bacula is backing up right now [17:45:36] just like it did before [17:45:44] Ask paladox [17:45:54] he's probably not around [17:46:22] No he's on holiday [17:46:29] But stilll got access [17:49:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.13, 22.03, 23.98 [17:51:46] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.12, 24.43, 24.66 [17:55:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.13, 21.45, 23.39 [17:59:46] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.45, 21.97, 23.03 [18:00:05] [02miraheze/puppet] 07Southparkfan pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJhDO [18:00:07] [02miraheze/puppet] 07Southparkfan 034b793bd - Update gen_fingerprints From https://raw.githubusercontent.com/wikimedia/puppet/4d54f89dfae3e24ad2afd6a2dea26988f5ec5d1d/modules/base/files/environment/gen_fingerprints. Not updating to the latest version, since the latest version doesn't show the md5 fingerprint anymore. However, PuTTY only supports the md5 fingerprint: [18:00:07] https://www.chiark.greenend.org.uk/~sgtatham/putty/wishlist/ssh-fingerprint-formats.html [18:00:08] [ PuTTY wish ssh-fingerprint-formats ] - www.chiark.greenend.org.uk [18:03:45] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.26, 21.67, 22.74 [18:15:32] PROBLEM - cp9 HTTP 4xx/5xx ERROR Rate on cp9 is WARNING: WARNING - NGINX Error Rate is 46% [18:17:32] RECOVERY - cp9 HTTP 4xx/5xx ERROR Rate on cp9 is OK: OK - NGINX Error Rate is 6% [18:21:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.14, 22.55, 22.23 [18:25:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.94, 21.52, 21.96 [18:27:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.38, 22.96, 22.44 [18:29:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.66, 20.34, 21.56 [18:31:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 31.23, 23.75, 22.59 [18:35:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 15.87, 20.69, 21.73 [18:47:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.85, 23.84, 22.66 [18:49:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.14, 22.95, 22.52 [19:05:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.43, 22.18, 22.11 [19:07:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.39, 22.64, 22.32 [19:11:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.61, 22.54, 22.22 [19:15:46] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.92, 22.15, 22.32 [19:21:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 29.29, 24.24, 22.97 [19:26:32] curiosity, some script related to Wikibase has been updated? [19:27:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.80, 22.67, 22.86 [19:30:40] hispano76: no, why? [19:31:45] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.84, 24.00, 23.26 [19:33:45] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.05, 23.03, 23.01 [19:37:46] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.23, 23.23, 23.02 [19:39:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.53, 23.16, 23.06 [19:41:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.20, 24.47, 23.54 [19:43:44] RhinosF1: For this. I wanted to check if it was because of an update of Wikibase or for having put HispanoWiki as a private wiki, which solved the problem of linking pages with items. https://phabricator.miraheze.org/T6020 [19:43:44] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.30, 23.97, 23.47 [19:43:44] [ ⚓ T6020 The site link does not work on interucroniaswiki ] - phabricator.miraheze.org [19:44:34] https://ucronias.miraheze.org/wiki/Ucron%C3%ADas:Portada check working RhinosF1 but I still have to check if it's really resolved or partial [19:44:37] [ Ucronías ] - ucronias.miraheze.org [19:44:41] hispano76: oh, yeah I guess that was the too big dbname bug [19:45:11] :) [19:49:27] hispano76: the script was getting stuck and not updating [19:49:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.23, 22.48, 22.78 [19:49:46] I would update the task but my phone is having a fit [19:50:03] ok [19:51:34] Trying to get someone who used to work at google to have a look over the bug [19:57:45] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.18, 23.48, 23.65 [20:01:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.15, 23.33, 23.46 [20:07:46] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.55, 23.49, 23.48 [20:09:44] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.83, 23.93, 23.63 [20:25:11] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JJhQX [20:25:13] [02miraheze/services] 07MirahezeSSLBot 03e2fc341 - BOT: Updating services config for wikis [20:36:37] [02miraheze/ssl] 07Pix1234 pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/JJh7n [20:36:38] [02miraheze/ssl] 07Pix1234 036a2636c - Add beaconspace.unrestrictedlorefare.com (T6066) [20:39:46] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.33, 23.09, 24.00 [20:41:48] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 34.28, 25.60, 24.70 [20:49:56] !log Run importDump.php for ficreationwiki (T6067) [20:50:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:19:00] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 18.39, 22.00, 23.80 [21:23:00] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 28.05, 22.89, 23.52 [21:25:00] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.62, 22.01, 23.14 [21:45:00] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.41, 18.33, 20.14 [21:53:00] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 27.76, 24.30, 22.01 [21:55:00] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.83, 22.74, 21.73 [22:01:00] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.35, 18.36, 20.10 [22:11:00] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.77, 22.31, 21.17 [22:12:45] probably me im using both jobrunners for imports rn [22:15:00] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 28.45, 24.63, 22.34 [22:17:00] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.47, 22.55, 21.86 [22:18:18] Hello culi! If you have any questions, feel free to ask and someone should answer soon. [22:18:34] hello [22:18:52] hi @culi [22:19:09] I posted a question on the discord, but it doesn't seem as active as the irc so I'm gonna repost if that's ok [22:19:22] oh wait nvm someone j responded lol [22:20:00] @culi Yeah, we're all volunteers, so someone should reply in not too long of time. If ever you don't receive a reply within the hour or so, feel free to repost. [22:20:36] gotchu, thanks so much [22:20:49] np [22:23:00] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.94, 22.76, 22.06 [22:27:00] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.74, 22.88, 22.32 [22:31:03] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 34.43, 24.79, 22.93 [22:35:00] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.49, 23.57, 23.01 [22:39:00] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 26.27, 23.63, 23.09 [22:41:00] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.73, 22.56, 22.75 [22:43:55] i've been considerably busy today, sorry :P [22:49:00] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 32.67, 25.31, 23.34 [22:55:00] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.04, 23.54, 23.32 [23:01:00] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.68, 21.23, 22.16 [23:05:00] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.03, 22.07, 22.40 [23:09:00] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.27, 23.45, 22.90 [23:17:00] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.88, 23.49, 23.48 [23:21:00] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 23.81, 24.06, 23.69 [23:22:57] PROBLEM - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is WARNING: WARNING - NGINX Error Rate is 57% [23:24:52] RECOVERY - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is OK: OK - NGINX Error Rate is 9% [23:25:00] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.69, 23.15, 23.48 [23:39:00] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 16.19, 17.99, 20.19 [23:57:18] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.82, 20.79, 19.69 [23:59:13] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 17.44, 19.69, 19.42