[00:08:40] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JegWQ [00:08:42] [02miraheze/puppet] 07paladox 0302ec57a - Update packages.pp [00:08:43] [02puppet] 07paladox synchronize pull request 03#1132: Add support for mariadb 10.3 - 13https://git.io/JegWw [00:09:18] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JegW7 [00:09:19] [02miraheze/puppet] 07paladox 03b1b0f3b - Update mw.cnf.erb [00:09:21] [02puppet] 07paladox synchronize pull request 03#1132: Add support for mariadb 10.3 - 13https://git.io/JegWw [00:10:51] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JegWF [00:10:53] [02miraheze/puppet] 07paladox 034028f21 - Update db4.yaml [00:10:54] [02puppet] 07paladox synchronize pull request 03#1132: Add support for mariadb 10.3 - 13https://git.io/JegWw [00:11:28] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JegWb [00:11:30] [02miraheze/puppet] 07paladox 0312da964 - Update config.pp [00:11:31] [02puppet] 07paladox synchronize pull request 03#1132: Add support for mariadb 10.3 - 13https://git.io/JegWw [00:12:41] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JegWp [00:12:43] [02miraheze/puppet] 07paladox 03feb0c52 - Update packages.pp [00:12:44] [02puppet] 07paladox synchronize pull request 03#1132: Add support for mariadb 10.3 - 13https://git.io/JegWw [00:26:42] [02puppet] 07paladox synchronize pull request 03#1132: Add support for mariadb 10.3 - 13https://git.io/JegWw [00:26:43] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JeglY [00:26:45] [02miraheze/puppet] 07paladox 03a231d68 - Update packages.pp [00:47:34] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/Jeglg [00:47:36] [02miraheze/puppet] 07paladox 03dd52291 - Update lizardfs6.yaml [00:47:37] [02puppet] 07paladox synchronize pull request 03#1132: Add support for mariadb 10.3 - 13https://git.io/JegWw [00:48:24] [02puppet] 07paladox synchronize pull request 03#1132: Add support for mariadb 10.3 - 13https://git.io/JegWw [00:48:25] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/Jegla [00:48:27] [02miraheze/puppet] 07paladox 0316b9df0 - Update config.pp [00:49:40] [02puppet] 07paladox closed pull request 03#1132: Add support for mariadb 10.3 - 13https://git.io/JegWw [00:49:41] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±5] 13https://git.io/Jeglr [00:49:43] [02miraheze/puppet] 07paladox 03dea4d37 - Add support for mariadb 10.3 (#1132) * Add support for mariadb 10.3 * Update packages.pp * Update mw.cnf.erb * Update db4.yaml * Update config.pp * Update packages.pp * Update packages.pp * Update lizardfs6.yaml * Update config.pp [00:49:44] [02puppet] 07paladox deleted branch 03paladox-patch-6 - 13https://git.io/vbiAS [00:49:46] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-6 [00:50:42] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeglK [00:50:44] [02miraheze/puppet] 07paladox 037149647 - Fix [00:53:27] PROBLEM - db4 Puppet on db4 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 11 minutes ago with 0 failures [00:55:30] RECOVERY - db4 Puppet on db4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:57:23] RECOVERY - lizardfs6 Puppet on lizardfs6 is OK: OK: Puppet is currently enabled, last run 6 seconds ago with 0 failures [01:18:27] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JeglN [01:18:29] [02miraheze/puppet] 07paladox 032510da8 - mariadb: Pin to ams2.mirrors.digitalocean.com [01:21:14] PROBLEM - lizardfs6 Puppet on lizardfs6 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): Package[nagios-plugins],Package[mariadb-server-10.3] [01:25:15] RECOVERY - lizardfs6 Puppet on lizardfs6 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:28:35] PROBLEM - lizardfs5 Current Load on lizardfs5 is CRITICAL: CRITICAL - load average: 4.87, 4.26, 2.80 [01:36:23] PROBLEM - lizardfs5 Current Load on lizardfs5 is WARNING: WARNING - load average: 0.88, 3.75, 3.41 [01:38:20] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 0.14, 2.56, 3.02 [01:46:01] !log scp'ing metawiki, test1wiki and mhglobal to lizardfs6 [01:46:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [02:05:23] !log restart lizardfs-chunkserver on lizardfs6 [02:05:31] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jeg8M [02:05:33] [02miraheze/puppet] 07paladox 03a6e85fc - Update mfschunkserver.cfg.erb [02:05:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [02:13:44] PROBLEM - lizardfs5 Current Load on lizardfs5 is CRITICAL: CRITICAL - load average: 5.96, 3.85, 2.61 [02:15:43] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 1.18, 2.91, 2.43 [03:29:28] PROBLEM - lizardfs5 Current Load on lizardfs5 is WARNING: WARNING - load average: 3.93, 3.06, 2.03 [03:31:27] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 0.91, 2.22, 1.85 [03:31:39] PROBLEM - test1 Puppet on test1 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 9 minutes ago with 0 failures [04:24:28] !log scp'ing loginwiki to lizardfs6 [04:24:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [04:28:15] PROBLEM - lizardfs4 Current Load on lizardfs4 is CRITICAL: CRITICAL - load average: 4.97, 3.69, 2.39 [04:30:15] RECOVERY - lizardfs4 Current Load on lizardfs4 is OK: OK - load average: 1.24, 2.63, 2.15 [04:43:27] PROBLEM - lizardfs5 Current Load on lizardfs5 is CRITICAL: CRITICAL - load average: 5.70, 3.25, 1.88 [04:45:27] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 1.10, 2.40, 1.74 [04:55:38] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [06:25:48] RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 2810 MB (11% inode=94%); [06:27:30] PROBLEM - lizardfs4 Current Load on lizardfs4 is WARNING: WARNING - load average: 3.44, 2.62, 1.73 [06:29:29] RECOVERY - lizardfs4 Current Load on lizardfs4 is OK: OK - load average: 2.02, 2.53, 1.82 [07:04:13] PROBLEM - lizardfs5 Current Load on lizardfs5 is CRITICAL: CRITICAL - load average: 4.30, 2.85, 1.91 [07:06:09] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 2.64, 2.96, 2.08 [08:09:39] @Zppix unless you’re script hasn’t run fully yet then there is no DarkMode on meta so it can’t have worked [08:11:30] PROBLEM - lizardfs5 Current Load on lizardfs5 is CRITICAL: CRITICAL - load average: 4.56, 3.55, 2.33 [08:13:27] PROBLEM - lizardfs5 Current Load on lizardfs5 is WARNING: WARNING - load average: 2.58, 3.44, 2.45 [08:15:27] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 0.52, 2.34, 2.16 [08:32:48] PROBLEM - lizardfs4 Current Load on lizardfs4 is WARNING: WARNING - load average: 3.52, 2.75, 1.87 [08:34:03] PROBLEM - misc2 Puppet on misc2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [08:34:18] PROBLEM - misc3 Puppet on misc3 is CRITICAL: CRITICAL: Puppet has 20 failures. Last run 2 minutes ago with 20 failures. Failed resources (up to 3 shown) [08:34:21] PROBLEM - db4 Puppet on db4 is CRITICAL: CRITICAL: Puppet has 16 failures. Last run 2 minutes ago with 16 failures. Failed resources (up to 3 shown) [08:34:46] PROBLEM - cp4 Puppet on cp4 is CRITICAL: CRITICAL: Puppet has 210 failures. Last run 3 minutes ago with 210 failures. Failed resources (up to 3 shown) [08:34:47] PROBLEM - lizardfs4 Puppet on lizardfs4 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [08:34:51] PROBLEM - lizardfs5 Puppet on lizardfs5 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [08:34:52] PROBLEM - bacula1 Puppet on bacula1 is CRITICAL: CRITICAL: Puppet has 13 failures. Last run 3 minutes ago with 13 failures. Failed resources (up to 3 shown) [08:34:53] PROBLEM - lizardfs4 Current Load on lizardfs4 is CRITICAL: CRITICAL - load average: 4.88, 3.27, 2.15 [08:35:09] PROBLEM - cp2 Puppet on cp2 is CRITICAL: CRITICAL: Puppet has 208 failures. Last run 2 minutes ago with 208 failures. Failed resources (up to 3 shown): File[/etc/rsyslog.d],File[/etc/rsyslog.conf],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades] [08:35:13] PROBLEM - db5 Puppet on db5 is CRITICAL: CRITICAL: Puppet has 13 failures. Last run 3 minutes ago with 13 failures. Failed resources (up to 3 shown) [08:35:15] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 15 failures. Last run 3 minutes ago with 15 failures. Failed resources (up to 3 shown) [08:35:18] PROBLEM - lizardfs6 Puppet on lizardfs6 is CRITICAL: CRITICAL: Puppet has 17 failures. Last run 3 minutes ago with 17 failures. Failed resources (up to 3 shown) [08:35:26] PROBLEM - mw1 Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 227 failures. Last run 3 minutes ago with 227 failures. Failed resources (up to 3 shown) [08:35:40] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 20 failures. Last run 4 minutes ago with 20 failures. Failed resources (up to 3 shown) [08:35:50] PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Puppet has 220 failures. Last run 3 minutes ago with 220 failures. Failed resources (up to 3 shown) [08:35:51] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 51 failures. Last run 4 minutes ago with 51 failures. Failed resources (up to 3 shown) [08:35:53] PROBLEM - misc4 Puppet on misc4 is CRITICAL: CRITICAL: Puppet has 32 failures. Last run 4 minutes ago with 32 failures. Failed resources (up to 3 shown) [08:36:10] PROBLEM - mw3 Puppet on mw3 is CRITICAL: CRITICAL: Puppet has 223 failures. Last run 4 minutes ago with 223 failures. Failed resources (up to 3 shown) [08:36:39] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 149 failures. Last run 3 minutes ago with 149 failures. Failed resources (up to 3 shown): File[/etc/rsyslog.d],File[/etc/rsyslog.conf],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades] [08:36:52] RECOVERY - lizardfs4 Current Load on lizardfs4 is OK: OK - load average: 1.44, 2.76, 2.12 [08:37:49] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Puppet has 218 failures. Last run 3 minutes ago with 218 failures. Failed resources (up to 3 shown): File[/etc/rsyslog.d],File[/etc/rsyslog.conf],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades] [08:42:45] PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 1 backends are down. mw2 [08:43:02] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb [08:43:09] RECOVERY - bacula1 Puppet on bacula1 is OK: OK: Puppet is currently enabled, last run 15 seconds ago with 0 failures [08:43:09] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 2400:6180:0:d0::403:f001/cpweb [08:43:10] PROBLEM - lizardfs4 Current Load on lizardfs4 is CRITICAL: CRITICAL - load average: 4.65, 5.73, 3.63 [08:43:18] RECOVERY - db5 Puppet on db5 is OK: OK: Puppet is currently enabled, last run 23 seconds ago with 0 failures [08:43:22] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 25 seconds ago with 0 failures [08:43:30] RECOVERY - lizardfs6 Puppet on lizardfs6 is OK: OK: Puppet is currently enabled, last run 28 seconds ago with 0 failures [08:43:42] PROBLEM - lizardfs5 Current Load on lizardfs5 is CRITICAL: CRITICAL - load average: 6.52, 5.40, 3.14 [08:44:00] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 59 seconds ago with 0 failures [08:44:01] RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 53 seconds ago with 0 failures [08:44:03] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 39 seconds ago with 0 failures [08:44:11] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 9 seconds ago with 0 failures [08:44:14] RECOVERY - misc2 Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:44:20] RECOVERY - misc3 Puppet on misc3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:44:27] RECOVERY - db4 Puppet on db4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:44:27] RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 27 seconds ago with 0 failures [08:44:40] RECOVERY - cp3 Varnish Backends on cp3 is OK: All 5 backends are healthy [08:44:57] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [08:45:01] RECOVERY - cp4 Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:45:09] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [08:45:15] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [08:45:15] RECOVERY - lizardfs5 Puppet on lizardfs5 is OK: OK: Puppet is currently enabled, last run 41 seconds ago with 0 failures [08:45:40] RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:46:31] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:46:37] RECOVERY - lizardfs4 Puppet on lizardfs4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:47:13] PROBLEM - lizardfs4 Current Load on lizardfs4 is WARNING: WARNING - load average: 2.44, 3.86, 3.32 [08:47:56] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:49:17] RECOVERY - lizardfs4 Current Load on lizardfs4 is OK: OK - load average: 1.96, 3.30, 3.19 [09:03:34] PROBLEM - lizardfs5 Current Load on lizardfs5 is WARNING: WARNING - load average: 2.10, 3.01, 3.86 [09:09:37] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 1.20, 2.12, 3.19 [09:45:51] PROBLEM - lizardfs4 Current Load on lizardfs4 is CRITICAL: CRITICAL - load average: 7.34, 5.58, 3.51 [09:49:51] RECOVERY - lizardfs4 Current Load on lizardfs4 is OK: OK - load average: 1.12, 3.14, 2.99 [09:51:51] PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2635 MB (10% inode=94%); [10:07:15] PROBLEM - lizardfs5 Current Load on lizardfs5 is CRITICAL: CRITICAL - load average: 5.95, 4.27, 2.86 [10:09:14] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 2.34, 3.27, 2.65 [10:16:35] PuppyKun: gimme your email so I can forward some emails [10:32:05] revi: it’ll be in puppet somewhere for staff [10:33:36] I obviously know the location but the point is I don't want to :-p [10:35:09] revi: ok [10:35:17] PROBLEM - lizardfs5 Current Load on lizardfs5 is CRITICAL: CRITICAL - load average: 4.95, 3.57, 2.61 [10:37:17] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 1.78, 3.04, 2.54 [11:40:10] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jeggq [11:40:11] [02miraheze/services] 07MirahezeSSLBot 03d49ca7f - BOT: Updating services config for wikis [12:02:49] PROBLEM - lizardfs4 Current Load on lizardfs4 is CRITICAL: CRITICAL - load average: 4.74, 4.01, 2.69 [12:04:52] RECOVERY - lizardfs4 Current Load on lizardfs4 is OK: OK - load average: 1.33, 3.00, 2.49 [12:08:05] PROBLEM - lizardfs5 Current Load on lizardfs5 is CRITICAL: CRITICAL - load average: 6.64, 3.85, 2.50 [12:11:59] PROBLEM - lizardfs5 Current Load on lizardfs5 is WARNING: WARNING - load average: 2.20, 3.55, 2.74 [12:13:56] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 0.97, 2.75, 2.55 [12:42:07] PROBLEM - lizardfs4 Current Load on lizardfs4 is CRITICAL: CRITICAL - load average: 6.17, 4.44, 3.01 [12:42:42] PROBLEM - lizardfs5 Current Load on lizardfs5 is CRITICAL: CRITICAL - load average: 5.42, 3.48, 2.62 [12:44:41] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 3.28, 3.40, 2.70 [12:52:08] PROBLEM - lizardfs4 Current Load on lizardfs4 is WARNING: WARNING - load average: 1.64, 3.67, 3.64 [12:54:07] RECOVERY - lizardfs4 Current Load on lizardfs4 is OK: OK - load average: 0.70, 2.64, 3.27 [13:21:09] PROBLEM - lizardfs5 Current Load on lizardfs5 is CRITICAL: CRITICAL - load average: 5.28, 3.71, 2.72 [13:27:06] PROBLEM - lizardfs5 Current Load on lizardfs5 is WARNING: WARNING - load average: 1.90, 3.86, 3.26 [13:29:06] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 1.23, 3.04, 3.03 [13:56:52] .in 3 hours ask void about CN and split sentences [13:56:53] RhinosF1: Okay, I will set the reminder for: 2019-11-02 - 16:56:53GMT [14:10:00] [02ssl] 07Pix1234 opened pull request 03#235: Create cert for wiki.newfil.com.au.crt - 13https://git.io/JegVc [14:11:18] [02ssl] 07paladox closed pull request 03#235: Create cert for wiki.newfil.com.au.crt - 13https://git.io/JegVc [14:11:20] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/JegVl [14:11:21] [02miraheze/ssl] 07Pix1234 035eb606b - Create cert for wiki.newfil.com.au.crt (#235) * Create wiki.newfil.com.au.crt * add wiki.newfil.com.au [14:11:45] .in 10m change managewiki settings for new SSL cert [14:11:45] Zppix: Okay, I will set the reminder for: 2019-11-02 - 09:21:45CDT [14:21:45] Zppix: change managewiki settings for new SSL cert [14:28:05] revi: you around? [14:28:10] yes [14:28:16] doing something else tho [14:28:19] revi: mind a pm? [14:28:24] goahdead [14:32:11] PROBLEM - lizardfs5 Current Load on lizardfs5 is CRITICAL: CRITICAL - load average: 4.92, 3.14, 2.17 [14:34:06] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 2.12, 2.97, 2.24 [14:39:47] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JegVy [14:39:48] [02miraheze/puppet] 07paladox 0380ce928 - Add support for mariadb 10.4 [14:39:50] [02puppet] 07paladox created branch 03paladox-patch-6 - 13https://git.io/vbiAS [14:39:51] [02puppet] 07paladox opened pull request 03#1133: Add support for mariadb 10.4 - 13https://git.io/JegVS [14:40:06] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-6 [+0/-0/±1] 13https://git.io/JegVH [14:40:08] [02miraheze/puppet] 07paladox 0357ce638 - Update packages.pp [14:40:09] [02puppet] 07paladox synchronize pull request 03#1133: Add support for mariadb 10.4 - 13https://git.io/JegVS [14:40:28] [02puppet] 07paladox closed pull request 03#1133: Add support for mariadb 10.4 - 13https://git.io/JegVS [14:40:29] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JegVQ [14:40:31] [02miraheze/puppet] 07paladox 0316f5930 - Add support for mariadb 10.4 (#1133) * Add support for mariadb 10.4 * Update packages.pp [14:43:48] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JegVF [14:43:49] [02miraheze/puppet] 07paladox 03149a828 - Update mfschunkserver.cfg.erb [14:44:26] !log restart lizardfs-chunkserver on lizardfs6 [14:51:17] !log [14:44:26] <+paladox> !log restart lizardfs-chunkserver on lizardfs6 [14:51:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [14:55:29] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jegws [14:55:30] [02miraheze/puppet] 07paladox 035d96907 - Update mfschunkserver.cfg.erb [14:55:59] !log restart lizardfs-chunkserver on lizardfs6 [14:56:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [15:02:00] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jegw4 [15:02:02] [02miraheze/puppet] 07paladox 0341a1dda - Update mfschunkserver.cfg.erb [15:02:31] !log restart lizardfs-chunkserver on lizardfs6 [15:02:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [15:16:18] PROBLEM - mw3 Current Load on mw3 is CRITICAL: CRITICAL - load average: 6.30, 8.43, 6.74 [15:18:19] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 5.26, 7.19, 6.48 [15:19:27] PROBLEM - lizardfs4 Current Load on lizardfs4 is CRITICAL: CRITICAL - load average: 5.13, 4.48, 2.78 [15:21:22] RECOVERY - lizardfs4 Current Load on lizardfs4 is OK: OK - load average: 1.30, 3.22, 2.52 [15:22:14] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 5.39, 6.35, 6.30 [15:36:43] PROBLEM - lizardfs5 Current Load on lizardfs5 is CRITICAL: CRITICAL - load average: 7.06, 3.68, 2.19 [15:38:45] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 2.27, 3.33, 2.27 [15:46:00] PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 1 backends are down. mw1 [15:47:39] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 2604:180:0:33b::2/cpweb, 2400:6180:0:d0::403:f001/cpweb [15:48:29] test [15:48:34] PROBLEM - cp4 Varnish Backends on cp4 is CRITICAL: 1 backends are down. mw3 [15:48:50] PROBLEM - cp2 Varnish Backends on cp2 is CRITICAL: 1 backends are down. mw3 [15:49:22] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 5 datacenters are down: 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb [15:49:29] miraheze is bipolar again :slight_smile: [15:50:05] RECOVERY - cp3 Varnish Backends on cp3 is OK: All 5 backends are healthy [15:50:30] RECOVERY - cp4 Varnish Backends on cp4 is OK: All 5 backends are healthy [15:50:47] RECOVERY - cp2 Varnish Backends on cp2 is OK: All 5 backends are healthy [15:51:16] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [15:51:34] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [16:13:09] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_34 [+0/-0/±1] 13https://git.io/Jegrl [16:13:11] [02miraheze/mediawiki] 07paladox 033f88ebd - Update OOUIHTMLForm.php [16:16:59] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_34 [+0/-0/±1] 13https://git.io/Jegru [16:17:01] [02miraheze/mediawiki] 07paladox 031d83ab8 - Update HTMLForm.php [16:18:50] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [16:20:33] PROBLEM - cp2 Varnish Backends on cp2 is CRITICAL: 1 backends are down. mw3 [16:22:13] PROBLEM - lizardfs4 Current Load on lizardfs4 is CRITICAL: CRITICAL - load average: 6.25, 4.95, 3.06 [16:22:33] RECOVERY - cp2 Varnish Backends on cp2 is OK: All 5 backends are healthy [16:23:57] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jegrz [16:23:59] [02miraheze/puppet] 07paladox 0310da043 - Update mfschunkserver.cfg.erb [16:24:37] !log restart lizardfs-chunkserver on lizardfs6 [16:24:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [16:26:43] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [16:27:59] PROBLEM - lizardfs4 Current Load on lizardfs4 is WARNING: WARNING - load average: 0.77, 3.64, 3.25 [16:29:54] RECOVERY - lizardfs4 Current Load on lizardfs4 is OK: OK - load average: 0.89, 2.77, 2.98 [16:42:46] PROBLEM - lizardfs5 Current Load on lizardfs5 is CRITICAL: CRITICAL - load average: 6.57, 4.40, 2.85 [16:48:43] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 1.19, 3.40, 3.05 [16:56:54] RhinosF1: ask void about CN and split sentences [17:18:28] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb [17:20:26] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [17:26:19] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JegKU [17:26:20] [02miraheze/puppet] 07paladox 0388aca8b - Update mfschunkserver.cfg.erb [17:26:56] !log restart lizardfs-chunkserver on lizardfs6 [17:27:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:52:32] PROBLEM - lizardfs4 Current Load on lizardfs4 is CRITICAL: CRITICAL - load average: 5.82, 3.52, 2.13 [17:54:32] RECOVERY - lizardfs4 Current Load on lizardfs4 is OK: OK - load average: 1.44, 2.84, 2.07 [18:03:34] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JegK7 [18:03:36] [02miraheze/puppet] 07paladox 0391c36c1 - gluster: Update to v7 [18:05:05] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JegKA [18:05:06] [02miraheze/puppet] 07paladox 03ab90397 - Update firewall.yaml [18:09:12] !log update GeoLite2-Country.mmdb on ns1 and misc1 (and restart gdnsd) [18:09:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [18:09:42] PROBLEM - lizardfs5 Current Load on lizardfs5 is CRITICAL: CRITICAL - load average: 4.12, 3.15, 2.18 [18:11:41] PROBLEM - lizardfs5 Current Load on lizardfs5 is WARNING: WARNING - load average: 3.97, 3.76, 2.54 [18:13:40] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 0.69, 2.61, 2.27 [19:20:43] PROBLEM - lizardfs5 Current Load on lizardfs5 is CRITICAL: CRITICAL - load average: 5.12, 3.73, 2.29 [19:22:43] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 1.36, 2.70, 2.08 [19:38:46] [02miraheze/mediawiki] 07paladox pushed 0313 commits to 03REL1_34 [+0/-0/±66] 13https://git.io/JegiX [19:38:47] [02miraheze/mediawiki] 07Fomafix 032c565dc - Parser: Hard deprecate getConverterLanguage getConverterLanguage is deprecated since MediaWiki 1.32. getConverterLanguage always return a Language object and never null. Change-Id: Ia0480c76416ef1e925619d9e85c7134c2ecf2296 Depends-On: Iea4771161d129c49f6482e1a6822e1324bf2fb49 (cherry picked from commit 9bae9db4cc296f248ea7c1fba950032e54a5cb03) [19:38:49] [02miraheze/mediawiki] 07cscott 030974c51 - Deprecate Parser::areSubpagesAllowed() / Parser::maybeDoSubpageLink() These are unused outside the Parser and are so short it's not worth renaming them to make them private; just hard-deprecate the methods and inline the implementation in the small # of places they appear. Code search: [19:38:49] https://codesearch.wmflabs.org/deployed/?q=areSubpagesAllowed%7CmaybeDoSubpageLink&i=nope&files=&repos= Bug: T236810 Change-Id: Ia06c65409a3158b083bcc59c9f6e347945b375c0 (cherry picked from commit dcae22c8fa46822148dc5fd87018acb385bf4c91) [19:38:50] [02miraheze/mediawiki] 07cscott 03ce81d00 - Deprecate Parser::splitWhitespace() / Parser::createAssocArgs() These methods are not used anywhere in deployed code. Parser::createAssocArgs() is used in Extension:DataTable2, but it should just copy the implementation from Parser, it's very little code. Code search: https://codesearch.wmflabs.org/deployed/?q=splitWhitespace%7CcreateAssocArgs&i=nope&files=&repos= [19:38:50] Bug: T236810 Change-Id: I4e6a39e2ecf7a3a568e26ad1d8ce1166a44a5ad9 (cherry picked from commit d7cd12e81ff6622bb5e14980297f926a3dec0984) [19:38:51] [ MediaWiki code search ] - codesearch.wmflabs.org [19:38:52] [02miraheze/mediawiki] ... and 10 more commits. [19:38:52] [ MediaWiki code search ] - codesearch.wmflabs.org [19:39:22] paladox: squash? [19:39:30] no? [19:39:36] Why? [19:40:26] Because i git pulled? [19:40:34] and squish makes one commit... [19:40:48] i rather it did git merge for that [19:42:55] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_34 [+0/-0/±1] 13https://git.io/JegiD [19:42:56] [02miraheze/mediawiki] 07paladox 031906729 - Update ModernSkylight [19:42:57] It would be less spammy though next time if possible [20:00:22] !log sudo -u www-data php toggleExtension.php --wiki=metawiki darkmode [20:00:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [20:01:06] paladox: for all wikis [20:01:31] you said meta [20:01:54] you need to be clearer in future please :) [20:03:04] !log /usr/bin/nice -19 /usr/local/bin/foreachwikiindblist /srv/mediawiki/dblist/all.dblist /srv/mediawiki/w/extensions/ManageWiki/maintenance/toggleExtension.php darkmode [20:03:08] paladox: yeah sorry, can you check it works with 1.33 first please as nothing happens from my phone [20:03:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [20:03:32] RhinosF1 i thought you did?!?! [20:03:38] It worked on test and says 1.28+ so it could be my phone [20:03:48] paladox: it worked on test but that’s 1.34 [20:05:23] RhinosF1 sorry i carn't, i have to go now [20:05:24] paladox: it’s working on ATT now [20:05:31] Maybe it was slow [20:05:42] Button wasn’t doing anything [20:06:10] Weird that’s meta [20:06:16] https://www.irccloud.com/pastebin/BrLp1vRk [20:06:16] [ Snippet | IRCCloud ] - www.irccloud.com [20:06:28] Appears on console for meta but ATT is fine [20:07:08] PuppyKun, Reception123, Zppix: ^ can you look into that? [20:32:26] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 5 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb [20:34:20] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [20:50:54] PROBLEM - lizardfs5 Current Load on lizardfs5 is WARNING: WARNING - load average: 2.48, 3.41, 2.41 [20:52:49] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 0.75, 2.50, 2.19 [21:11:51] [02puppet] 07RhinosF1 opened pull request 03#1134: Disable mail notify for rhinosf1 - 13https://git.io/JegX7 [21:12:15] Reception123, PuppyKun, paladox: merge? [21:22:33] [02puppet] 07paladox closed pull request 03#1134: Disable mail notify for rhinosf1 - 13https://git.io/JegX7 [21:22:34] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jeg1l [21:22:35] RhinosF1 merged [21:22:36] [02miraheze/puppet] 07RhinosF1 03c7f8a2a - Disable mail notify for rhinosf1 (#1134) I see them when I’m here on irc and all they do is clog my inbox [21:25:47] paladox: merger [21:25:51] Thanks [21:26:03] My head is not on today [21:46:02] PROBLEM - mw3 Current Load on mw3 is WARNING: WARNING - load average: 7.64, 6.80, 5.96 [21:47:59] RECOVERY - mw3 Current Load on mw3 is OK: OK - load average: 6.12, 6.59, 5.99 [21:55:41] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 107.191.126.23/cpweb [21:55:51] PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 1 backends are down. mw3 [21:56:16] PROBLEM - cp2 Varnish Backends on cp2 is CRITICAL: 1 backends are down. mw3 [21:56:29] PROBLEM - cp4 Varnish Backends on cp4 is CRITICAL: 1 backends are down. mw1 [21:57:48] RECOVERY - cp3 Varnish Backends on cp3 is OK: All 5 backends are healthy [21:58:15] RECOVERY - cp2 Varnish Backends on cp2 is OK: All 5 backends are healthy [21:58:28] RECOVERY - cp4 Varnish Backends on cp4 is OK: All 5 backends are healthy [21:59:35] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [22:14:35] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JegMg [22:14:36] [02miraheze/puppet] 07paladox 03d32c121 - Install gluster on lizardfs6 [22:20:12] PROBLEM - lizardfs6 Puppet on lizardfs6 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 3 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[apt_update_gluster],Exec[apt_update] [22:20:56] PROBLEM - lizardfs4 Current Load on lizardfs4 is CRITICAL: CRITICAL - load average: 6.65, 3.98, 2.44 [22:23:03] RECOVERY - lizardfs4 Current Load on lizardfs4 is OK: OK - load average: 1.93, 3.21, 2.37 [22:23:29] PROBLEM - lizardfs6 GlusterFS port 49152 on lizardfs6 is CRITICAL: connect to address 54.36.165.161 and port 49152: Connection refused [22:23:36] PROBLEM - lizardfs6 GlusterFS port 24007 on lizardfs6 is CRITICAL: connect to address 54.36.165.161 and port 24007: Connection refused [22:25:33] RECOVERY - lizardfs6 GlusterFS port 24007 on lizardfs6 is OK: TCP OK - 0.013 second response time on 54.36.165.161 port 24007 [22:26:02] RECOVERY - lizardfs6 Puppet on lizardfs6 is OK: OK: Puppet is currently enabled, last run 53 seconds ago with 0 failures [22:28:18] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JegMS [22:28:20] [02miraheze/puppet] 07paladox 030b25a73 - Update apt.pp [22:47:39] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JegDL [22:47:40] [02miraheze/puppet] 07paladox 037b91ce6 - Update init.pp [22:47:56] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JegDt [22:47:58] [02miraheze/puppet] 07paladox 03dc89be2 - Update lizardfs6.yaml [22:49:29] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JegDq [22:49:31] [02miraheze/puppet] 07paladox 035067bc3 - Update apt.pp [23:01:08] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://git.io/JegDa [23:01:10] [02miraheze/puppet] 07paladox 03a20c51a - Add gluster mount to test1 [23:01:11] [02puppet] 07paladox created branch 03paladox-patch-7 - 13https://git.io/vbiAS [23:01:13] [02puppet] 07paladox opened pull request 03#1135: Add gluster mount to test1 - 13https://git.io/JegDV [23:01:31] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://git.io/JegDw [23:01:33] [02miraheze/puppet] 07paladox 03706d276 - Update test1.yaml [23:01:34] [02puppet] 07paladox synchronize pull request 03#1135: Add gluster mount to test1 - 13https://git.io/JegDV [23:01:39] [02puppet] 07paladox deleted branch 03paladox-patch-6 - 13https://git.io/vbiAS [23:01:40] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-6 [23:01:42] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://git.io/vbiAS [23:01:43] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [23:01:50] [02puppet] 07paladox closed pull request 03#1135: Add gluster mount to test1 - 13https://git.io/JegDV [23:01:51] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JegDr [23:01:53] [02miraheze/puppet] 07paladox 0320c82a8 - Add gluster mount to test1 (#1135) * Add gluster mount to test1 * Update test1.yaml [23:01:54] [02puppet] 07paladox deleted branch 03paladox-patch-7 - 13https://git.io/vbiAS [23:01:56] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-7 [23:08:48] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JegDX [23:08:49] [02miraheze/puppet] 07paladox 037081a0d - Update mediawiki.pp [23:19:52] PROBLEM - lizardfs5 Current Load on lizardfs5 is CRITICAL: CRITICAL - load average: 6.30, 3.71, 2.19 [23:23:49] RECOVERY - lizardfs5 Current Load on lizardfs5 is OK: OK - load average: 1.30, 3.21, 2.41 [23:24:53] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jegyt [23:24:55] [02miraheze/puppet] 07paladox 0362ef472 - Update init.pp [23:28:55] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jegys [23:28:57] [02miraheze/puppet] 07paladox 03433ebd5 - Update mount.pp [23:38:09] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jegy0 [23:38:10] [02miraheze/puppet] 07paladox 036777c7b - Update wildcard.pp [23:43:14] PROBLEM - cp2 Varnish Backends on cp2 is CRITICAL: 1 backends are down. mw2 [23:43:19] PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 1 backends are down. mw2 [23:44:04] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb [23:44:19] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 5 datacenters are down: 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb [23:45:09] RECOVERY - cp2 Varnish Backends on cp2 is OK: All 5 backends are healthy [23:45:14] RECOVERY - cp3 Varnish Backends on cp3 is OK: All 5 backends are healthy [23:46:04] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [23:46:18] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online