[00:10:14] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjTLf [00:10:16] [02miraheze/services] 07MirahezeSSLBot 034bda34e - BOT: Updating services config for wikis [06:38:31] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 388 [06:46:08] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [06:53:55] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 384 [06:57:54] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [13:00:13] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjT84 [13:00:15] [02miraheze/services] 07MirahezeSSLBot 03fdc1345 - BOT: Updating services config for wikis [18:00:24] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjTg1 [18:00:25] [02miraheze/services] 07MirahezeSSLBot 03f4f2175 - BOT: Updating services config for wikis [18:32:46] [02mw-config] 07The-Voidwalker opened pull request 03#2655: fix pollny install - 13https://git.io/fjT23 [18:39:02] [02mw-config] 07paladox closed pull request 03#2655: fix pollny install - 13https://git.io/fjT23 [18:39:03] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjT2Z [18:39:05] [02miraheze/mw-config] 07The-Voidwalker 033a4601e - fix pollny install (#2655) [18:39:24] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjT2n [18:39:26] [02miraheze/mw-config] 07paladox 033ed8c5f - Upgrade travis to xenial [18:54:23] might want to keep an eye on DB4, space is getting dangerously low [18:57:06] @Void done [18:57:06] PROBLEM - db4 Disk Space on db4 is WARNING: DISK WARNING - free space: / 50103 MB (13% inode=96%); [18:58:52] ty, that looks much better [19:01:44] !log upgrading puppetdb to 6.3, puppet-agent to 6.4 and puppetserver to 6.3 on puppet1 [19:01:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [19:04:08] PROBLEM - misc4 Puppet on misc4 is CRITICAL: CRITICAL: Puppet has 27 failures. Last run 2 minutes ago with 27 failures. Failed resources (up to 3 shown) [19:04:37] PROBLEM - db4 Puppet on db4 is CRITICAL: CRITICAL: Puppet has 16 failures. Last run 2 minutes ago with 16 failures. Failed resources (up to 3 shown) [19:04:48] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Puppet has 206 failures. Last run 2 minutes ago with 206 failures. Failed resources (up to 3 shown) [19:04:51] PROBLEM - bacula1 Puppet on bacula1 is CRITICAL: CRITICAL: Puppet has 14 failures. Last run 2 minutes ago with 14 failures. Failed resources (up to 3 shown) [19:04:57] PROBLEM - cp2 Puppet on cp2 is CRITICAL: CRITICAL: Puppet has 200 failures. Last run 2 minutes ago with 200 failures. Failed resources (up to 3 shown) [19:05:43] PROBLEM - lizardfs2 Puppet on lizardfs2 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 3 minutes ago with 12 failures. Failed resources (up to 3 shown) [19:05:45] PROBLEM - elasticsearch1 Puppet on elasticsearch1 is CRITICAL: CRITICAL: Puppet has 16 failures. Last run 3 minutes ago with 16 failures. Failed resources (up to 3 shown) [19:05:56] PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Puppet has 207 failures. Last run 3 minutes ago with 207 failures. Failed resources (up to 3 shown) [19:06:02] PROBLEM - lizardfs1 Puppet on lizardfs1 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 3 minutes ago with 12 failures. Failed resources (up to 3 shown) [19:06:05] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjT21 [19:06:06] [02miraheze/puppet] 07paladox 0305c557d - Update puppetserver.systemd [19:06:11] PROBLEM - mw3 Puppet on mw3 is CRITICAL: CRITICAL: Puppet has 209 failures. Last run 3 minutes ago with 209 failures. Failed resources (up to 3 shown) [19:06:13] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 60 failures. Last run 4 minutes ago with 60 failures. Failed resources (up to 3 shown) [19:06:14] PROBLEM - mw1 Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 213 failures. Last run 3 minutes ago with 213 failures. Failed resources (up to 3 shown) [19:06:14] PROBLEM - cp5 Puppet on cp5 is CRITICAL: CRITICAL: Puppet has 164 failures. Last run 3 minutes ago with 164 failures. Failed resources (up to 3 shown): File[/etc/rsyslog.d],File[/etc/rsyslog.conf],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades] [19:06:30] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 201 failures. Last run 3 minutes ago with 201 failures. Failed resources (up to 3 shown) [19:06:48] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjT2M [19:06:49] [02miraheze/puppet] 07paladox 03ad66c7e - Update puppetserver.systemd [19:11:45] RECOVERY - elasticsearch1 Puppet on elasticsearch1 is OK: OK: Puppet is currently enabled, last run 9 seconds ago with 0 failures [19:12:53] RECOVERY - bacula1 Puppet on bacula1 is OK: OK: Puppet is currently enabled, last run 11 seconds ago with 0 failures [19:13:43] RECOVERY - lizardfs2 Puppet on lizardfs2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [19:13:54] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 25 seconds ago with 0 failures [19:14:02] RECOVERY - lizardfs1 Puppet on lizardfs1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [19:14:06] RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 20 seconds ago with 0 failures [19:14:08] RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [19:14:11] RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 40 seconds ago with 0 failures [19:14:11] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [19:14:37] RECOVERY - db4 Puppet on db4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [19:14:56] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [19:14:57] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 44 seconds ago with 0 failures [19:16:14] RECOVERY - cp5 Puppet on cp5 is OK: OK: Puppet is currently enabled, last run 57 seconds ago with 0 failures [19:16:29] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:38:44] !log starting import on animatedfeetWIKI (mw1) [22:38:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [22:44:06] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/fjTrr [22:44:07] [02miraheze/dns] 07paladox 031ef8b7c - Add udc-japan.xyz to dns T4183 [22:48:14] !log drop iceriawiki database [22:48:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [22:49:03] !log drop autombilepediawiki and zhautomobilepediawiki database. [22:49:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [22:49:28] !log drop caputiawiki database [22:49:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [22:53:29] !log remove blogpost from hypopediawiki T4205 [22:53:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [22:56:46] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjTrS [22:56:48] [02miraheze/ssl] 07paladox 038c57b27 - Add udc-japan.xyz ssl certificate T4183 [22:56:49] [02ssl] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vxP9L [22:56:51] [02ssl] 07paladox opened pull request 03#155: Add udc-japan.xyz ssl certificate T4183 - 13https://git.io/fjTr9 [22:57:14] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+1/-0/±0] 13https://git.io/fjTrH [22:57:15] [02miraheze/ssl] 07paladox 03f1dab69 - Create udc-japan.xyz.crt [22:57:17] [02ssl] 07paladox synchronize pull request 03#155: Add udc-japan.xyz ssl certificate T4183 - 13https://git.io/fjTr9 [22:57:35] [02ssl] 07paladox closed pull request 03#155: Add udc-japan.xyz ssl certificate T4183 - 13https://git.io/fjTr9 [22:57:36] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/fjTrQ [22:57:38] [02miraheze/ssl] 07paladox 03e608508 - Add udc-japan.xyz ssl certificate T4183 (#155) * Add udc-japan.xyz ssl certificate T4183 * Create udc-japan.xyz.crt [22:57:39] [02ssl] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vxP9L [22:57:41] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-1 [23:00:16] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjTr7 [23:00:18] [02miraheze/services] 07MirahezeSSLBot 03567e158 - BOT: Updating services config for wikis [23:00:28] [02ssl] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vxP9L [23:00:30] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjTr5 [23:00:31] [02miraheze/ssl] 07paladox 03244c15c - Add zw.fontainebleau-avon.fr ssl certificate T4220 [23:00:33] [02ssl] 07paladox opened pull request 03#156: Add zw.fontainebleau-avon.fr ssl certificate T4220 - 13https://git.io/fjTrd [23:01:05] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+1/-0/±0] 13https://git.io/fjTrb [23:01:06] [02miraheze/ssl] 07paladox 03387e293 - Create zw.fontainebleau-avon.fr.crt [23:01:08] [02ssl] 07paladox synchronize pull request 03#156: Add zw.fontainebleau-avon.fr ssl certificate T4220 - 13https://git.io/fjTrd [23:01:27] [02ssl] 07paladox closed pull request 03#156: Add zw.fontainebleau-avon.fr ssl certificate T4220 - 13https://git.io/fjTrd [23:01:28] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/fjTrN [23:01:30] [02miraheze/ssl] 07paladox 032852a00 - Add zw.fontainebleau-avon.fr ssl certificate T4220 (#156) * Add zw.fontainebleau-avon.fr ssl certificate T4220 * Create zw.fontainebleau-avon.fr.crt [23:02:28] [02ssl] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vxP9L [23:02:30] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-1 [23:03:35] [02ssl] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vxP9L [23:03:37] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjTrj [23:03:38] [02miraheze/ssl] 07paladox 032f7bf36 - Remove wiki.taotac.org T4222 [23:03:40] [02ssl] 07paladox opened pull request 03#157: Remove wiki.taotac.org T4222 - 13https://git.io/fjToe [23:03:52] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-1/±0] 13https://git.io/fjTov [23:03:53] [02miraheze/ssl] 07paladox 03aa05414 - Delete wiki.taotac.org.crt [23:03:55] [02ssl] 07paladox synchronize pull request 03#157: Remove wiki.taotac.org T4222 - 13https://git.io/fjToe [23:04:03] [02ssl] 07paladox closed pull request 03#157: Remove wiki.taotac.org T4222 - 13https://git.io/fjToe [23:04:04] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±1] 13https://git.io/fjTof [23:04:06] [02miraheze/ssl] 07paladox 03bf06efb - Remove wiki.taotac.org T4222 (#157) * Remove wiki.taotac.org T4222 * Delete wiki.taotac.org.crt [23:04:21] [02ssl] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vxP9L [23:04:23] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-1 [23:05:11] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjToU [23:05:13] [02miraheze/services] 07MirahezeSSLBot 031ad8535 - BOT: Updating services config for wikis [23:55:48] [02ssl] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vxP9L [23:55:50] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-1/±0] 13https://git.io/fjToo [23:55:51] [02miraheze/ssl] 07paladox 037c1a217 - Delete cornetto.online.crt [23:55:53] [02ssl] 07paladox opened pull request 03#158: Delete cornetto.online.crt - 13https://git.io/fjToK [23:56:13] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjTo6 [23:56:14] [02miraheze/ssl] 07paladox 03f021205 - Update certs.yaml [23:56:16] [02ssl] 07paladox synchronize pull request 03#158: Delete cornetto.online.crt - 13https://git.io/fjToK [23:56:34] [02ssl] 07paladox closed pull request 03#158: Delete cornetto.online.crt - 13https://git.io/fjToK [23:56:36] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±1] 13https://git.io/fjToi [23:56:37] [02miraheze/ssl] 07paladox 0395b6854 - Delete cornetto.online.crt (#158) * Delete cornetto.online.crt * Update certs.yaml [23:56:39] [02ssl] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vxP9L [23:56:40] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-1 [23:59:49] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fjToM [23:59:50] [02miraheze/dns] 07paladox 0311079e7 - Delete cornetto.online