[00:02:54] PROBLEM - espiral.org - Gandi on sslhost is WARNING: WARNING - Certificate 'espiral.org' expires in 30 day(s) (Fri 23 Aug 2019 11:59:59 PM GMT +0000). [00:13:19] PROBLEM - es1 Current Load on es1 is CRITICAL: CRITICAL - load average: 5.07, 3.56, 2.64 [00:15:19] RECOVERY - es1 Current Load on es1 is OK: OK - load average: 1.80, 2.78, 2.46 [00:20:36] PROBLEM - es3 Current Load on es3 is CRITICAL: CRITICAL - load average: 7.95, 4.23, 2.80 [00:22:56] !log reboot misc4 [00:23:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [00:23:19] PROBLEM - es1 Current Load on es1 is WARNING: WARNING - load average: 3.72, 3.46, 2.76 [00:25:19] RECOVERY - es1 Current Load on es1 is OK: OK - load average: 2.00, 2.89, 2.63 [00:27:05] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 3.63, 2.90, 1.80 [00:29:04] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 2.54, 2.77, 1.89 [00:31:57] PROBLEM - misc4 Current Load on misc4 is CRITICAL: CRITICAL - load average: 6.47, 3.15, 1.19 [00:33:54] RECOVERY - misc4 Current Load on misc4 is OK: OK - load average: 1.53, 2.34, 1.12 [00:36:23] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 5.28, 3.20, 1.83 [00:40:12] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjD4C [00:40:13] [02miraheze/services] 07MirahezeSSLBot 030c60e2d - BOT: Updating services config for wikis [00:48:25] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 3.35, 3.84, 3.05 [00:50:23] RECOVERY - es4 Current Load on es4 is OK: OK - load average: 1.36, 2.91, 2.80 [01:04:26] PROBLEM - es3 Current Load on es3 is WARNING: WARNING - load average: 3.49, 2.75, 3.96 [01:05:04] RECOVERY - test1 Disk Space on test1 is OK: DISK OK - free space: / 5935 MB (14% inode=98%); [01:08:28] PROBLEM - es3 Current Load on es3 is CRITICAL: CRITICAL - load average: 5.28, 3.15, 3.78 [01:46:26] PROBLEM - es3 Current Load on es3 is WARNING: WARNING - load average: 0.72, 2.03, 3.64 [01:50:26] PROBLEM - es3 Current Load on es3 is CRITICAL: CRITICAL - load average: 6.30, 3.83, 3.93 [02:04:00] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 4.23, 3.62, 2.44 [02:07:57] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 2.71, 3.46, 2.68 [02:09:57] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 4.36, 3.82, 2.91 [02:13:55] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 3.27, 3.63, 3.04 [02:14:23] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 4.80, 3.25, 1.88 [02:16:23] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 3.73, 3.43, 2.12 [02:16:26] PROBLEM - es3 Current Load on es3 is WARNING: WARNING - load average: 0.68, 2.51, 3.96 [02:17:51] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 0.93, 2.65, 2.81 [02:20:23] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 4.52, 3.74, 2.52 [02:22:25] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 3.96, 3.92, 2.74 [02:24:26] RECOVERY - es3 Current Load on es3 is OK: OK - load average: 0.88, 1.97, 3.16 [02:26:28] RECOVERY - es4 Current Load on es4 is OK: OK - load average: 2.60, 3.17, 2.72 [02:28:33] PROBLEM - es3 Current Load on es3 is CRITICAL: CRITICAL - load average: 4.80, 3.43, 3.45 [02:53:15] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 3.78, 3.38, 2.47 [02:55:13] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 2.91, 3.22, 2.52 [02:56:26] PROBLEM - es3 Current Load on es3 is WARNING: WARNING - load average: 1.49, 2.79, 3.83 [03:00:28] PROBLEM - es3 Current Load on es3 is CRITICAL: CRITICAL - load average: 5.00, 3.25, 3.72 [03:21:19] PROBLEM - es1 Current Load on es1 is WARNING: WARNING - load average: 3.43, 2.89, 2.31 [03:23:19] RECOVERY - es1 Current Load on es1 is OK: OK - load average: 1.41, 2.43, 2.22 [03:26:26] PROBLEM - es3 Current Load on es3 is WARNING: WARNING - load average: 1.38, 2.77, 3.80 [03:27:11] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 3.84, 3.06, 2.02 [03:29:09] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 4.23, 3.35, 2.24 [03:30:26] RECOVERY - es3 Current Load on es3 is OK: OK - load average: 2.03, 2.01, 3.23 [03:31:09] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 3.81, 3.38, 2.38 [03:33:08] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 6.54, 4.59, 2.95 [03:34:35] PROBLEM - es3 Current Load on es3 is CRITICAL: CRITICAL - load average: 5.37, 3.74, 3.61 [03:43:06] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 1.44, 3.94, 3.73 [03:45:05] RECOVERY - es4 Current Load on es4 is OK: OK - load average: 0.38, 2.75, 3.31 [04:00:26] PROBLEM - es3 Current Load on es3 is WARNING: WARNING - load average: 0.84, 2.17, 3.77 [04:02:57] PROBLEM - misc4 Current Load on misc4 is WARNING: WARNING - load average: 3.47, 3.31, 2.42 [04:04:32] PROBLEM - es3 Current Load on es3 is CRITICAL: CRITICAL - load average: 4.60, 2.71, 3.56 [04:04:57] RECOVERY - misc4 Current Load on misc4 is OK: OK - load average: 2.60, 3.04, 2.42 [04:08:29] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 4.10, 2.80, 1.87 [04:10:34] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 3.25, 2.80, 1.98 [04:14:28] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 3.45, 3.41, 2.43 [04:18:25] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 2.49, 3.18, 2.58 [04:38:26] PROBLEM - es3 Current Load on es3 is WARNING: WARNING - load average: 1.24, 2.10, 3.81 [04:44:28] PROBLEM - es3 Current Load on es3 is CRITICAL: CRITICAL - load average: 5.31, 3.13, 3.59 [04:48:13] PROBLEM - es1 Current Load on es1 is CRITICAL: CRITICAL - load average: 6.32, 3.53, 2.57 [04:50:12] RECOVERY - es1 Current Load on es1 is OK: OK - load average: 2.66, 2.95, 2.47 [04:57:50] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 4.01, 4.36, 2.95 [04:59:48] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 2.57, 3.80, 2.92 [05:01:46] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 0.96, 2.77, 2.65 [05:14:32] PROBLEM - es3 Current Load on es3 is WARNING: WARNING - load average: 2.33, 2.54, 3.86 [05:18:26] RECOVERY - es3 Current Load on es3 is OK: OK - load average: 0.70, 1.72, 3.24 [05:21:18] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 5.45, 3.61, 2.23 [05:24:36] PROBLEM - es3 Current Load on es3 is CRITICAL: CRITICAL - load average: 6.84, 4.84, 4.06 [05:25:17] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 3.42, 3.82, 2.67 [05:27:16] RECOVERY - es4 Current Load on es4 is OK: OK - load average: 1.52, 3.16, 2.57 [05:33:14] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 4.02, 3.02, 2.24 [05:35:12] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 2.71, 3.01, 2.34 [05:39:06] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 3.54, 3.39, 2.67 [05:41:05] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 3.01, 3.23, 2.69 [06:12:27] PROBLEM - es1 Current Load on es1 is CRITICAL: CRITICAL - load average: 5.05, 4.21, 3.00 [06:14:22] RECOVERY - es1 Current Load on es1 is OK: OK - load average: 1.25, 3.16, 2.77 [06:27:14] RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 3116 MB (12% inode=94%); [07:02:28] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 7.19, 4.07, 2.15 [07:16:26] PROBLEM - es3 Current Load on es3 is WARNING: WARNING - load average: 1.61, 2.87, 3.98 [07:20:26] PROBLEM - es3 Current Load on es3 is CRITICAL: CRITICAL - load average: 4.34, 2.80, 3.62 [07:22:23] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 2.37, 3.67, 3.99 [07:26:23] RECOVERY - es4 Current Load on es4 is OK: OK - load average: 1.48, 2.23, 3.32 [07:42:21] PROBLEM - cp3 Current Load on cp3 is CRITICAL: CRITICAL - load average: 2.71, 1.99, 1.30 [07:44:21] PROBLEM - cp3 Current Load on cp3 is WARNING: WARNING - load average: 1.34, 1.71, 1.28 [07:46:21] RECOVERY - cp3 Current Load on cp3 is OK: OK - load average: 1.03, 1.57, 1.28 [07:48:26] PROBLEM - es3 Current Load on es3 is WARNING: WARNING - load average: 0.91, 2.47, 3.68 [07:52:25] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 5.36, 3.30, 2.42 [07:52:26] RECOVERY - es3 Current Load on es3 is OK: OK - load average: 2.13, 2.30, 3.32 [07:54:41] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 3.47, 3.43, 2.59 [07:56:23] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 3.61, 3.83, 2.87 [07:56:39] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 2.00, 3.03, 2.56 [07:58:23] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 6.07, 4.30, 3.14 [08:00:28] PROBLEM - es3 Current Load on es3 is CRITICAL: CRITICAL - load average: 8.03, 5.80, 4.41 [08:03:09] PROBLEM - misc2 Current Load on misc2 is WARNING: WARNING - load average: 3.49, 2.49, 1.85 [08:05:09] RECOVERY - misc2 Current Load on misc2 is OK: OK - load average: 2.40, 2.37, 1.88 [08:10:23] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 1.84, 3.44, 3.58 [08:12:24] RECOVERY - es4 Current Load on es4 is OK: OK - load average: 1.98, 2.84, 3.34 [08:22:23] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 7.09, 4.76, 3.67 [08:24:23] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 1.62, 3.41, 3.30 [08:24:43] what's going on with load [08:26:23] RECOVERY - es4 Current Load on es4 is OK: OK - load average: 0.70, 2.44, 2.96 [08:34:04] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 4.77, 2.86, 1.65 [08:34:32] Reception123: around? what's up with the load [08:36:03] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 3.53, 2.88, 1.80 [08:36:23] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 4.39, 3.24, 2.84 [08:38:01] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 3.11, 2.95, 1.96 [08:41:20] PROBLEM - es1 Current Load on es1 is CRITICAL: CRITICAL - load average: 4.85, 2.75, 2.00 [08:41:56] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 3.43, 3.33, 2.34 [08:43:19] RECOVERY - es1 Current Load on es1 is OK: OK - load average: 1.86, 2.40, 1.97 [08:43:54] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 2.93, 3.19, 2.41 [09:00:23] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 2.00, 2.76, 3.95 [09:04:23] RECOVERY - es4 Current Load on es4 is OK: OK - load average: 0.69, 1.73, 3.25 [09:11:45] PROBLEM - es1 Current Load on es1 is CRITICAL: CRITICAL - load average: 6.15, 3.92, 2.79 [09:13:40] PROBLEM - es1 Current Load on es1 is WARNING: WARNING - load average: 3.44, 3.62, 2.81 [09:15:13] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjDud [09:15:14] [02miraheze/services] 07MirahezeSSLBot 030165a2d - BOT: Updating services config for wikis [09:15:35] RECOVERY - es1 Current Load on es1 is OK: OK - load average: 1.55, 2.97, 2.68 [09:26:23] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 3.56, 2.93, 2.36 [09:30:23] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 4.49, 3.58, 2.75 [09:31:19] PROBLEM - es1 Current Load on es1 is WARNING: WARNING - load average: 3.71, 3.03, 2.61 [09:33:19] RECOVERY - es1 Current Load on es1 is OK: OK - load average: 1.36, 2.49, 2.47 [09:35:12] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 6.13, 4.77, 2.76 [09:38:21] PROBLEM - cp3 Current Load on cp3 is CRITICAL: CRITICAL - load average: 2.09, 1.85, 1.52 [09:40:21] PROBLEM - cp3 Current Load on cp3 is WARNING: WARNING - load average: 2.00, 1.85, 1.56 [09:40:25] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 2.98, 3.89, 3.52 [09:42:26] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 5.72, 4.47, 3.77 [09:44:26] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 1.61, 3.43, 3.47 [09:45:32] PROBLEM - es1 Current Load on es1 is CRITICAL: CRITICAL - load average: 9.20, 5.33, 3.34 [09:46:25] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 9.01, 6.76, 4.75 [09:48:21] PROBLEM - cp3 Current Load on cp3 is CRITICAL: CRITICAL - load average: 1.63, 2.03, 1.78 [09:49:26] RECOVERY - es1 Current Load on es1 is OK: OK - load average: 1.48, 3.19, 2.90 [09:50:21] PROBLEM - cp3 Current Load on cp3 is WARNING: WARNING - load average: 1.78, 1.90, 1.76 [09:50:23] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 1.62, 3.78, 3.97 [09:51:09] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 1.40, 3.54, 3.77 [09:52:21] PROBLEM - cp3 Current Load on cp3 is CRITICAL: CRITICAL - load average: 2.52, 2.04, 1.81 [09:54:21] PROBLEM - cp3 Current Load on cp3 is WARNING: WARNING - load average: 1.85, 1.98, 1.82 [09:55:05] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 0.55, 2.13, 3.16 [09:56:21] RECOVERY - cp3 Current Load on cp3 is OK: OK - load average: 0.56, 1.53, 1.68 [09:58:56] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 6 failures. Last run 3 minutes ago with 6 failures. Failed resources (up to 3 shown): Exec[ufw-allow-tcp-from-2a00:d880:6:786::2-to-any-port-3210],Exec[ufw-allow-tcp-from-185.52.2.113-to-any-port-3210],Exec[ufw-allow-tcp-from-2a00:d880:5:799::2-to-any-port-3210],Exec[ufw-allow-tcp-from-81.4.121.113-to-any-port-3210] [10:00:23] RECOVERY - es4 Current Load on es4 is OK: OK - load average: 1.35, 2.74, 3.37 [10:04:55] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 20 seconds ago with 0 failures [10:21:06] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 5.19, 2.78, 2.04 [10:22:16] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 4.31, 3.34, 2.82 [10:23:08] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 3.91, 3.02, 2.22 [10:27:05] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 4.18, 3.39, 2.53 [10:29:05] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 3.10, 3.26, 2.59 [10:33:04] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 4.82, 3.81, 2.92 [10:35:04] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 3.54, 3.59, 2.94 [10:38:18] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 2.03, 3.83, 3.84 [10:39:05] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 2.96, 3.33, 2.99 [10:42:15] RECOVERY - es4 Current Load on es4 is OK: OK - load average: 1.97, 2.76, 3.38 [11:11:59] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 4.42, 3.66, 2.74 [11:19:32] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 3.72, 3.31, 2.52 [11:21:31] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 9.04, 5.12, 3.25 [11:25:56] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 1.30, 3.19, 3.42 [11:27:55] RECOVERY - es4 Current Load on es4 is OK: OK - load average: 1.10, 2.53, 3.14 [11:33:20] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 0.70, 3.06, 3.36 [11:54:37] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 6.09, 3.20, 2.38 [12:05:09] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 5.17, 3.35, 2.15 [12:09:06] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 3.02, 3.31, 2.42 [12:15:05] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 5.23, 5.52, 3.72 [12:18:26] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 0.93, 3.33, 3.95 [12:20:25] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 4.25, 3.25, 3.82 [12:25:05] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 0.76, 3.43, 3.76 [12:26:23] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 1.92, 3.42, 3.90 [12:29:04] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 0.68, 1.92, 3.06 [12:30:23] RECOVERY - es4 Current Load on es4 is OK: OK - load average: 1.47, 1.98, 3.17 [12:51:15] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 3.82, 3.50, 3.02 [12:53:14] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 4.98, 4.12, 3.31 [13:03:20] PROBLEM - es1 Current Load on es1 is WARNING: WARNING - load average: 3.93, 3.46, 2.57 [13:05:05] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 4.69, 3.20, 2.05 [13:05:19] RECOVERY - es1 Current Load on es1 is OK: OK - load average: 1.28, 2.64, 2.38 [13:07:05] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 2.73, 3.05, 2.14 [13:13:04] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 4.39, 3.83, 2.79 [13:13:09] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 1.41, 3.27, 3.85 [13:15:04] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 3.21, 3.58, 2.82 [13:17:06] RECOVERY - es4 Current Load on es4 is OK: OK - load average: 0.36, 1.67, 3.05 [13:17:07] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 4.15, 3.74, 2.97 [13:19:05] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 2.22, 3.30, 2.91 [13:37:19] PROBLEM - es1 Current Load on es1 is WARNING: WARNING - load average: 3.43, 3.50, 2.68 [13:39:19] RECOVERY - es1 Current Load on es1 is OK: OK - load average: 1.99, 2.81, 2.52 [13:44:04] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 7.73, 4.60, 3.20 [13:47:09] PROBLEM - misc2 Current Load on misc2 is WARNING: WARNING - load average: 3.80, 3.11, 2.42 [13:49:09] RECOVERY - misc2 Current Load on misc2 is OK: OK - load average: 3.25, 3.07, 2.48 [13:50:07] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 3.06, 3.80, 3.37 [13:52:07] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 4.62, 4.09, 3.52 [14:02:02] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 1.68, 3.54, 3.82 [14:06:00] RECOVERY - es4 Current Load on es4 is OK: OK - load average: 0.84, 2.30, 3.26 [14:09:47] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 3.51, 3.48, 2.81 [14:11:45] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 2.82, 3.36, 2.85 [14:33:38] Can a meta admin pls get in touch, Thank you1 [14:33:49] s/1/! [14:33:50] RhinosF1 meant to say: Reception!23: around? what's up with the load [14:34:42] PROBLEM - es4 Current Load on es4 is WARNING: WARNING - load average: 3.56, 2.67, 2.30 [14:36:43] RECOVERY - es4 Current Load on es4 is OK: OK - load average: 3.18, 2.98, 2.46 [14:42:39] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 5.93, 4.07, 3.03 [14:47:10] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 3.49, 3.11, 1.98 [14:47:53] paladox: can u check the cause ^^ it's been bad all day [14:48:12] i already know the cause. [14:48:17] You can ignore it. [14:48:46] paladox: k, you're . not a meta admin are u? [14:49:08] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 4.42, 3.58, 2.29 [14:49:18] nope [14:49:58] paladox: if u seen one give me a shout pls [14:50:05] ok [14:50:15] Reception123 is a meta admin. [14:50:32] and has been idle for 18 hrs [14:50:41] Reception123: around? [14:51:00] @Reception123 ping discord as well ^ [15:05:06] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 1.82, 3.25, 3.40 [15:09:41] PROBLEM - es1 Current Load on es1 is CRITICAL: CRITICAL - load average: 6.62, 4.40, 2.98 [15:11:36] RECOVERY - es1 Current Load on es1 is OK: OK - load average: 2.13, 3.38, 2.77 [15:25:44] PROBLEM - misc4 Prometheus on misc4 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:26:33] PROBLEM - es3 Current Load on es3 is CRITICAL: CRITICAL - load average: 4.22, 7.88, 8.19 [15:27:09] !log rebooted misc4 - oom [15:27:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [15:27:39] RECOVERY - misc4 Prometheus on misc4 is OK: TCP OK - 0.001 second response time on 185.52.3.121 port 9090 [15:30:52] !log stop salt-master on misc4 temporarily [15:30:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [15:32:47] PROBLEM - misc4 Puppet on misc4 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 51 seconds ago with 0 failures [15:43:17] !log shut down es3 temporarily [15:43:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [15:46:42] PROBLEM - es3 Disk Space on es3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [15:46:57] PROBLEM - es3 SSH on es3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:47:04] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 5.41, 2.99, 1.92 [15:47:07] PROBLEM - es3 Puppet on es3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [15:48:24] !log switched es3 back on [15:48:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [15:48:37] RECOVERY - es3 Disk Space on es3 is OK: DISK OK - free space: / 23907 MB (61% inode=93%); [15:48:55] RECOVERY - es3 SSH on es3 is OK: SSH OK - OpenSSH_7.4p1 Debian-10+deb9u6 (protocol 2.0) [15:49:04] RECOVERY - es3 Puppet on es3 is OK: OK: Puppet is currently enabled, last run 13 minutes ago with 0 failures [15:49:12] RECOVERY - es3 Current Load on es3 is OK: OK - load average: 2.66, 0.72, 0.24 [15:51:46] Hello Red_1! If you have any questions feel free to ask and someone should answer soon. [15:53:14] Took the time to log on; been a while; sadly I didn't config my router to identd properly and can't set that up from here... [15:54:10] Also have to remember you can't edit your text here like you can on Discord :p [15:55:49] Red_1: we just have s/wrong spelling/correct spelling [15:56:05] and a bot posts the corrected message [15:58:10] That's a nifty bot feature... [15:58:32] All kinds of nostalgia comin' up [15:59:03] I actually tried mIRC again... wtf you have to pay for it again? [15:59:25] irssi'n instead [16:01:33] Do you guys do PHP+SQL accesses without complicating things? eg: "Hi I'd like to just have this small DB with like 10 tables (less than 1mb total) and be able to make REST requests to it through PHP (or any other platform really) as part of the wiki site through a common.js extention I [would] code"? [16:02:04] After the needless chitchat, straight to the point... [16:03:05] PROBLEM - es1 Current Load on es1 is CRITICAL: CRITICAL - load average: 4.41, 3.82, 3.07 [16:06:59] PROBLEM - es1 Current Load on es1 is WARNING: WARNING - load average: 2.94, 3.48, 3.12 [16:07:06] So how does it look from the other side... [sorry for spam, but you guys are quiet ;] [16:08:23] < icinga-miraheze> PROBLEM - es1 Current Load on es1 is CRITICAL: CRITICAL - [16:08:45] Wow, that sucked, sorry meant to paste in another window. [16:08:54] RECOVERY - es1 Current Load on es1 is OK: OK - load average: 2.13, 3.08, 3.02 [16:11:04] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 1.90, 2.62, 3.86 [16:12:03] depending on what you are using it for, it sounds like you could just use Cargo [16:12:42] https://www.mediawiki.org/wiki/Extension:Cargo [16:12:42] [ Extension:Cargo - MediaWiki ] - www.mediawiki.org [16:16:37] PROBLEM - test1 Current Load on test1 is CRITICAL: CRITICAL - load average: 3.87, 2.82, 1.67 [16:16:50] Looking into it, thanks for info! [16:17:04] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 1.46, 1.99, 3.19 [16:28:02] PROBLEM - test1 Current Load on test1 is WARNING: WARNING - load average: 1.00, 1.94, 1.87 [16:29:25] PROBLEM - misc4 Prometheus on misc4 is CRITICAL: connect to address 185.52.3.121 and port 9090: Connection refused [16:29:56] PROBLEM - test1 Current Load on test1 is CRITICAL: CRITICAL - load average: 3.60, 2.46, 2.05 [16:31:25] RECOVERY - misc4 Prometheus on misc4 is OK: TCP OK - 0.001 second response time on 185.52.3.121 port 9090 [16:33:48] PROBLEM - test1 Current Load on test1 is WARNING: WARNING - load average: 0.74, 1.67, 1.83 [16:35:12] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjDK2 [16:35:13] [02miraheze/services] 07MirahezeSSLBot 03e8ffe16 - BOT: Updating services config for wikis [16:35:25] PROBLEM - misc4 Prometheus on misc4 is CRITICAL: connect to address 185.52.3.121 and port 9090: Connection refused [16:37:48] PROBLEM - test1 Current Load on test1 is CRITICAL: CRITICAL - load average: 2.13, 2.24, 2.04 [16:41:04] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 8.45, 4.71, 3.01 [16:41:48] PROBLEM - test1 Current Load on test1 is WARNING: WARNING - load average: 1.14, 1.83, 1.93 [16:43:48] PROBLEM - test1 Current Load on test1 is CRITICAL: CRITICAL - load average: 1.65, 2.21, 2.09 [16:45:04] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 1.81, 3.49, 2.95 [16:45:05] PROBLEM - test1 Disk Space on test1 is WARNING: DISK WARNING - free space: / 4504 MB (10% inode=98%); [16:45:13] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjDK9 [16:45:14] [02miraheze/services] 07MirahezeSSLBot 032371862 - BOT: Updating services config for wikis [16:47:04] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 1.32, 2.74, 2.74 [16:54:27] PROBLEM - es3 Current Load on es3 is CRITICAL: CRITICAL - load average: 6.77, 5.09, 2.64 [16:56:23] RhinosF1: hi. I am available now [16:57:48] PROBLEM - test1 Current Load on test1 is WARNING: WARNING - load average: 1.03, 1.69, 1.99 [16:58:26] PROBLEM - es3 Current Load on es3 is WARNING: WARNING - load average: 1.50, 3.63, 2.66 [17:00:16] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjDKp [17:00:18] [02miraheze/services] 07MirahezeSSLBot 039007a7c - BOT: Updating services config for wikis [17:00:26] RECOVERY - es3 Current Load on es3 is OK: OK - load average: 1.41, 2.88, 2.50 [17:00:47] Voidwalker: Checked them out, and we need something the user can select, not static wiki data... [17:03:37] Reception: handled by voidwalker [17:03:48] PROBLEM - test1 Current Load on test1 is CRITICAL: CRITICAL - load average: 2.97, 1.92, 1.93 [17:03:49] ok [17:03:55] sorry I wasn't available earlier to resolve it :( [17:04:05] hmm, then I'm not sure [17:04:38] Reception123, more of an eyes on this, than something needed doing [17:07:48] PROBLEM - test1 Current Load on test1 is WARNING: WARNING - load average: 0.60, 1.52, 1.80 [17:08:35] ah [17:11:04] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 6.96, 5.24, 3.18 [17:11:48] RECOVERY - test1 Current Load on test1 is OK: OK - load average: 1.58, 1.44, 1.68 [17:15:05] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 1.38, 3.19, 2.82 [17:15:48] PROBLEM - test1 Current Load on test1 is WARNING: WARNING - load average: 1.35, 1.75, 1.78 [17:17:48] RECOVERY - test1 Current Load on test1 is OK: OK - load average: 0.78, 1.45, 1.67 [17:27:14] PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2649 MB (10% inode=94%); [17:27:19] Lost connection 😦 My net connection is flimsy... [17:33:48] PROBLEM - test1 Current Load on test1 is WARNING: WARNING - load average: 1.08, 1.69, 1.73 [17:35:48] PROBLEM - test1 Current Load on test1 is CRITICAL: CRITICAL - load average: 2.37, 2.33, 1.98 [17:37:48] PROBLEM - test1 Current Load on test1 is WARNING: WARNING - load average: 1.59, 1.95, 1.87 [17:39:48] PROBLEM - test1 Current Load on test1 is CRITICAL: CRITICAL - load average: 2.31, 2.13, 1.94 [17:46:27] PROBLEM - es3 Current Load on es3 is WARNING: WARNING - load average: 3.24, 3.67, 2.58 [17:48:26] RECOVERY - es3 Current Load on es3 is OK: OK - load average: 1.44, 2.83, 2.40 [17:49:02] !log reception@mw1:/srv/mediawiki/w/extensions/CentralAuth/maintenance$ sudo -u www-data php createLocalAccount.php --wiki thecommonwealthwiki Monatch [17:49:04] ^ RhinosF1 [17:49:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:49:10] Voidwalker: please assign the rights to the above user [17:49:18] (see https://meta.miraheze.org/wiki/Special:RequestWikiQueue/8727#mw-section-request) [17:49:19] [ Wiki requests queue - Miraheze Meta ] - meta.miraheze.org [17:49:38] Thanks Reception123 [17:50:04] done [17:51:10] Super voidwalker [17:52:27] now only the German/English wiki request left in the queue - A) I don’t understand German and can’t find a creator that does and is around B) He’s got an English wiki requested but a German reason so not user what to do and it’s been there days [17:52:47] I know a bit of German so I can take a look [17:54:16] Reception123: it’s the other one in the queue [17:55:12] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjD6H [17:55:14] [02miraheze/services] 07MirahezeSSLBot 034e59377 - BOT: Updating services config for wikis [17:55:48] PROBLEM - test1 Current Load on test1 is WARNING: WARNING - load average: 1.51, 1.86, 2.00 [17:56:58] !log UPDATE cw_requests SET cw_status = 'approved' WHERE cw_id = '8724'; (cannot be done on-wiki) [17:56:59] ^ RhinosF1 [17:57:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:57:33] heh for some reason that didn't work [17:57:48] PROBLEM - test1 Current Load on test1 is CRITICAL: CRITICAL - load average: 4.98, 2.79, 2.29 [17:58:58] Reception: yeah, that failed [18:00:14] RhinosF1: was this also done with the db last time this happened? [18:00:37] Reception123: yeah John did it [18:00:41] yeah found it [18:00:42] 18:52 John: UPDATE cw_requests SET cw_status = 'approved' WHERE cw_id = 8681; [18:00:54] oh I see why it didn't work [18:02:16] nope, wasn't that, thought it was because the ID shouldn't be in '' but it's not that [18:02:18] paladox: any ideas? [18:02:46] nevermind found it, made a typo wrote 8724 [18:02:54] good thing that request was approved already so I basically did nothing [18:03:58] RhinosF1: if you can, please explain that their wiki would be in English if it's created. (description wise it is ok) [18:04:19] Reception123: Yeah, will do [18:06:33] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 6.95, 6.21, 5.07 [18:06:43] Reception123: see that’s comments as well, didn’t realise that was a rights issue as well [18:06:56] ah [18:07:55] well perhaps the creator should've asked about the English/German issue [18:08:24] Reception123: yeah, It was The Pioneer [18:08:33] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 5.67, 6.16, 5.20 [18:10:18] well, tbh I don't know if I would've noticed that [18:10:22] but anyway it can easily be changed [18:10:25] I'll create the user [18:10:27] PROBLEM - es3 Current Load on es3 is CRITICAL: CRITICAL - load average: 5.40, 4.01, 2.68 [18:10:49] RhinosF1: is this rights not being assigned issue a known issue? [18:11:21] !log reception@mw1:/srv/mediawiki/w/extensions/CentralAuth/maintenance$ sudo -u www-data php createLocalAccount.php --wiki topbikewiki TomK [18:11:22] ^ Voidwalker [18:11:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [18:12:14] done [18:12:26] RECOVERY - es3 Current Load on es3 is OK: OK - load average: 2.48, 3.39, 2.62 [18:12:53] Reception123: ask John but yes and it has been for a while but you normally get an MWException when it’s happens [18:16:32] Reception123: this is an example documented on phab https://phabricator.miraheze.org/T4543 but I normally just ask John to fix them [18:16:33] [ ⚓ T4543 CreateWiki JobQueueError ] - phabricator.miraheze.org [18:19:04] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 4.64, 4.62, 2.88 [18:21:04] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 1.99, 3.59, 2.71 [18:23:04] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 1.39, 2.84, 2.54 [18:25:47] thanks [18:25:51] and ok [18:28:45] !log UPDATE cw_requests SET cw_status = 'approved' WHERE cw_id = 8739; [18:28:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [18:32:05] Reception123 ? [18:32:16] paladox: figured it out :) [18:32:19] ok :) [18:34:26] PROBLEM - es3 Current Load on es3 is WARNING: WARNING - load average: 3.82, 3.22, 2.40 [18:36:26] RECOVERY - es3 Current Load on es3 is OK: OK - load average: 1.85, 2.60, 2.27 [18:38:54] Thx Reception123 [18:41:48] PROBLEM - test1 Current Load on test1 is WARNING: WARNING - load average: 0.26, 0.41, 1.87 [18:43:48] RECOVERY - test1 Current Load on test1 is OK: OK - load average: 0.50, 0.44, 1.70 [19:02:23] PROBLEM - es4 Current Load on es4 is CRITICAL: CRITICAL - load average: 4.13, 4.04, 2.81 [19:04:23] RECOVERY - es4 Current Load on es4 is OK: OK - load average: 1.82, 3.16, 2.63 [19:11:04] PROBLEM - es2 Current Load on es2 is CRITICAL: CRITICAL - load average: 5.16, 5.16, 3.15 [19:13:04] PROBLEM - es2 Current Load on es2 is WARNING: WARNING - load average: 1.75, 3.86, 2.92 [19:15:04] RECOVERY - es2 Current Load on es2 is OK: OK - load average: 1.55, 3.04, 2.72 [19:27:18] !log shutting es[1-4] [19:27:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [19:29:50] PROBLEM - es2 Disk Space on es2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:30:30] PROBLEM - es4 Disk Space on es4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:30:36] PROBLEM - es3 SSH on es3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:30:38] PROBLEM - es1 SSH on es1 is CRITICAL: connect to address 168.235.81.167 and port 22: Connection refused [19:30:39] PROBLEM - es2 SSH on es2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:30:39] PROBLEM - es4 Puppet on es4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:30:41] PROBLEM - es4 Current Load on es4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:30:42] PROBLEM - es3 Disk Space on es3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:30:43] PROBLEM - es3 Current Load on es3 is CRITICAL: connect to address 168.235.110.25 port 5666: Connection refusedconnect to host 168.235.110.25 port 5666: Connection refused [19:31:47] RECOVERY - es2 Disk Space on es2 is OK: DISK OK - free space: / 26147 MB (67% inode=93%); [19:32:25] RECOVERY - es4 Disk Space on es4 is OK: DISK OK - free space: / 29011 MB (75% inode=93%); [19:32:34] RECOVERY - es3 SSH on es3 is OK: SSH OK - OpenSSH_7.4p1 Debian-10+deb9u6 (protocol 2.0) [19:32:34] RECOVERY - es1 SSH on es1 is OK: SSH OK - OpenSSH_7.4p1 Debian-10+deb9u6 (protocol 2.0) [19:32:36] PROBLEM - es4 Puppet on es4 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 10 minutes ago with 0 failures [19:32:37] RECOVERY - es3 Disk Space on es3 is OK: DISK OK - free space: / 23928 MB (61% inode=92%); [19:32:38] RECOVERY - es2 SSH on es2 is OK: SSH OK - OpenSSH_7.4p1 Debian-10+deb9u6 (protocol 2.0) [19:32:39] RECOVERY - es3 Current Load on es3 is OK: OK - load average: 1.12, 0.65, 0.26 [19:32:39] RECOVERY - es4 Current Load on es4 is OK: OK - load average: 1.25, 0.69, 0.27 [19:32:45] RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [19:34:41] PROBLEM - es1 Puppet on es1 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 10 minutes ago with 0 failures [19:34:53]