[00:17:28] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvCDG [00:17:29] [02miraheze/dns] 07paladox 0396a92d5 - Add ipv4 address to jobrunner1 [00:40:04] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-5 [+1/-0/±0] 13https://git.io/JvCDo [00:40:06] [02miraheze/puppet] 07paladox 0326f55fb - Install mail1 as a mail server [00:40:07] [02puppet] 07paladox created branch 03paladox-patch-5 - 13https://git.io/vbiAS [00:40:09] [02puppet] 07paladox opened pull request 03#1205: Install mail1 as a mail server - 13https://git.io/JvCDK [00:40:11] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvCD6 [00:40:12] [02miraheze/services] 07MirahezeSSLBot 03ef2e984 - BOT: Updating services config for wikis [00:41:00] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-5 [+0/-0/±1] 13https://git.io/JvCDi [00:41:01] [02miraheze/puppet] 07paladox 033cef100 - Update site.pp [00:41:03] [02puppet] 07paladox synchronize pull request 03#1205: Install mail1 as a mail server - 13https://git.io/JvCDK [00:41:31] [02puppet] 07paladox closed pull request 03#1205: Install mail1 as a mail server - 13https://git.io/JvCDK [00:41:33] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/JvCDX [00:41:34] [02miraheze/puppet] 07paladox 03732361e - Install mail1 as a mail server (#1205) * Install mail1 as a mail server * Update site.pp [00:41:36] [02puppet] 07paladox deleted branch 03paladox-patch-5 - 13https://git.io/vbiAS [00:41:37] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-5 [00:42:51] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 2 datacenters are down: 2400:6180:0:d0::403:f001/cpweb, 51.161.32.127/cpweb [00:44:47] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [01:01:41] paladox|UKInEU: excuse me, there are frequent session loss warnings, it will be from the server? [01:02:19] yes, it restarted 20min ago [01:02:23] (redis) [01:03:41] [02puppet] 07paladox closed pull request 03#1204: mediawiki: Remove internal hostnames - 13https://git.io/JvCi0 [01:03:43] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvCyJ [01:03:44] [02miraheze/puppet] 07paladox 038ebe988 - mediawiki: Remove internal hostnames (#1204) [01:03:46] [02puppet] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbiAS [01:03:47] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-1 [01:03:49] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-5 [+0/-0/±1] 13https://git.io/JvCyU [01:03:50] [02miraheze/puppet] 07paladox 039bb154c - mediawiki: Do not install nginx if host is a jobrunner [01:03:52] [02puppet] 07paladox created branch 03paladox-patch-5 - 13https://git.io/vbiAS [01:03:53] [02puppet] 07paladox opened pull request 03#1206: mediawiki: Do not install nginx if host is a jobrunner - 13https://git.io/JvCyT [01:07:02] [02puppet] 07paladox closed pull request 03#1206: mediawiki: Do not install nginx if host is a jobrunner - 13https://git.io/JvCyT [01:07:03] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvCyL [01:07:05] [02miraheze/puppet] 07paladox 03fc5eebd - mediawiki: Do not install nginx if host is a jobrunner (#1206) [01:07:06] [02puppet] 07paladox deleted branch 03paladox-patch-5 - 13https://git.io/vbiAS [01:07:08] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-5 [01:41:43] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvCyy [01:41:45] [02miraheze/puppet] 07paladox 03d964c91 - redis: Make heap configurable [01:41:46] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [01:41:48] [02puppet] 07paladox opened pull request 03#1207: redis: Make heap configurable - 13https://git.io/JvCyS [01:42:37] [02puppet] 07paladox closed pull request 03#1207: redis: Make heap configurable - 13https://git.io/JvCyS [01:42:38] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvCyH [01:42:40] [02miraheze/puppet] 07paladox 03590f286 - redis: Make heap configurable (#1207) [01:42:41] [02puppet] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbiAS [01:42:43] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-1 [01:49:07] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+1/-0/±0] 13https://git.io/JvCyh [01:49:09] [02miraheze/puppet] 07paladox 03c637f37 - Make jobrunner1 a mediawiki jobrunner Also install redis locally with 1gb of heap [01:49:10] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [01:49:12] [02puppet] 07paladox opened pull request 03#1208: Make jobrunner1 a mediawiki jobrunner - 13https://git.io/JvCyj [01:51:04] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvCSv [01:51:05] [02miraheze/puppet] 07paladox 039dee2fd - Update site.pp [01:51:07] [02puppet] 07paladox synchronize pull request 03#1208: Make jobrunner1 a mediawiki jobrunner - 13https://git.io/JvCyj [01:51:18] [02puppet] 07paladox closed pull request 03#1208: Make jobrunner1 a mediawiki jobrunner - 13https://git.io/JvCyj [01:51:19] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/JvCSf [01:51:21] [02miraheze/puppet] 07paladox 03a5e4e08 - Make jobrunner1 a mediawiki jobrunner (#1208) * Make jobrunner1 a mediawiki jobrunner Also install redis locally with 1gb of heap * Update site.pp [01:51:22] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-1 [01:51:24] [02puppet] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbiAS [02:32:57] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 107.191.126.23/cpweb [02:34:34] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvCSQ [02:34:35] [02miraheze/puppet] 07paladox 0329f95f9 - Add jobrunner1 to firewalls [02:34:37] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [02:34:38] [02puppet] 07paladox opened pull request 03#1209: Add jobrunner1 to firewalls - 13https://git.io/JvCS7 [02:34:55] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [02:35:28] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvCS5 [02:35:29] [02miraheze/puppet] 07paladox 036a18db9 - Update firewall.yaml [02:35:31] [02puppet] 07paladox synchronize pull request 03#1209: Add jobrunner1 to firewalls - 13https://git.io/JvCS7 [02:36:35] [02puppet] 07paladox closed pull request 03#1209: Add jobrunner1 to firewalls - 13https://git.io/JvCS7 [02:36:36] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JvCSF [02:36:38] [02miraheze/puppet] 07paladox 032c8aa33 - Add jobrunner1 to firewalls (#1209) * Add jobrunner1 to firewalls * Update firewall.yaml [02:36:39] [02puppet] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbiAS [02:36:41] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-1 [02:42:42] PROBLEM - lizardfs6 Puppet on lizardfs6 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 59 seconds ago with 0 failures [02:42:44] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvC9e [02:42:45] [02miraheze/puppet] 07paladox 031dc0875 - Update jobrunner1.yaml [02:55:32] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvC9Y [02:55:34] [02miraheze/mw-config] 07paladox 03cdea209 - Update Redis.php [02:56:30] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvC9s [02:56:32] [02miraheze/mw-config] 07paladox 03f6e86ef - Update Redis.php [03:48:11] * hispano76 gives cookies to paladox|UKInEU [03:58:22] * paladox|UKInEU gives some to hispano76 too [05:15:08] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvCQb [05:15:10] [02miraheze/services] 07MirahezeSSLBot 0329f38c9 - BOT: Updating services config for wikis [06:27:30] RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 3053 MB (12% inode=94%); [09:12:47] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 2 datacenters are down: 2604:180:0:33b::2/cpweb, 2607:5300:205:200::17f6/cpweb [09:14:42] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [09:22:50] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 2a00:d880:5:8ea::ebc7/cpweb [09:25:05] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 51.161.32.127/cpweb, 2607:5300:205:200::17f6/cpweb [09:26:51] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [09:29:41] [02puppet] 07sevillafunciona opened pull request 03#1210: Create cp6.yaml - 13https://git.io/JvCNU [09:29:56] [02puppet] 07sevillafunciona opened pull request 03#1211: Update groups.conf - 13https://git.io/JvCNI [09:30:25] [02puppet] 07sevillafunciona opened pull request 03#1212: Update web.pp - 13https://git.io/JvCNm [09:30:32] [02puppet] 07sevillafunciona closed pull request 03#1212: Update web.pp - 13https://git.io/JvCNm [09:30:54] [02puppet] 07sevillafunciona opened pull request 03#1213: Update web.pp - 13https://git.io/JvCNY [09:31:10] [02puppet] 07sevillafunciona opened pull request 03#1214: Update mediawiki.pp - 13https://git.io/JvCNO [09:31:25] [02puppet] 07sevillafunciona opened pull request 03#1215: Update nginx.conf.erb - 13https://git.io/JvCN3 [09:31:36] [02puppet] 07sevillafunciona opened pull request 03#1216: Update vpncloud.pp - 13https://git.io/JvCNs [09:31:47] [02puppet] 07sevillafunciona opened pull request 03#1217: Update miraheze-internal.net.erb - 13https://git.io/JvCNZ [09:33:00] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 81.4.109.133/cpweb [09:34:56] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [09:35:23] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [09:37:05] Hello sevillafunciona! If you have any questions, feel free to ask and someone should answer soon. [09:37:46] Hi!! Please, pull the repo. [09:39:03] paladox|UKInEU Pull repo. [09:39:41] @RhinosF1 Hi??? Pull repo [11:21:11] PUSH REPO. [11:21:21] https://github.com/miraheze/puppet/pulls?q=is%3Aopen+is%3Apr [11:21:22] [ Pull Requests · miraheze/puppet · GitHub ] - github.com [11:47:17] Hi! Here is the list of currently open high priority tasks on Phabricator [11:47:24] No updates for 15 days - https://phabricator.miraheze.org/T5174 - enc.for.uz itermittently using both yandex and miraheze dns - authored by Paladox, assigned to Sf7_uz [11:47:30] PUSH REPO. [11:47:30] No updates for 6 days - https://phabricator.miraheze.org/T5130 - Internal error - authored by Kadmin, assigned to None [13:00:03] PUSH REPO. [13:13:49] PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2650 MB (10% inode=94%); [14:05:07] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWJo [14:05:09] [02miraheze/services] 07MirahezeSSLBot 036c3f624 - BOT: Updating services config for wikis [14:08:06] [02puppet] 07Reception123 commented on pull request 03#1210: Create cp6.yaml - 13https://git.io/JvWJM [14:08:10] [02puppet] 07Reception123 closed pull request 03#1210: Create cp6.yaml - 13https://git.io/JvCNU [14:09:38] [02puppet] 07Reception123 closed pull request 03#1211: Update groups.conf - 13https://git.io/JvCNI [14:09:40] [02puppet] 07Reception123 closed pull request 03#1213: Update web.pp - 13https://git.io/JvCNY [14:09:41] [02puppet] 07Reception123 closed pull request 03#1214: Update mediawiki.pp - 13https://git.io/JvCNO [14:09:43] [02puppet] 07Reception123 closed pull request 03#1215: Update nginx.conf.erb - 13https://git.io/JvCN3 [14:09:44] [02puppet] 07Reception123 closed pull request 03#1216: Update vpncloud.pp - 13https://git.io/JvCNs [14:09:46] [02puppet] 07Reception123 closed pull request 03#1217: Update miraheze-internal.net.erb - 13https://git.io/JvCNZ [14:34:33] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWUy [14:34:34] [02miraheze/dns] 07paladox 032e08a9a - Add mon1 to dns [14:38:04] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWUF [14:38:06] [02miraheze/dns] 07paladox 0380d4b23 - Add phab1 to dns [14:42:36] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWTs [14:42:37] [02miraheze/dns] 07paladox 03a05bdfc - Add rdb2 to dns [14:44:56] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWTl [14:44:57] [02miraheze/dns] 07paladox 031f89d6a - Add services2 to dns [14:47:50] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWTg [14:47:51] [02miraheze/dns] 07paladox 0386a6e48 - Add cp7 to dns [14:49:46] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWT6 [14:49:48] [02miraheze/dns] 07paladox 036d9e466 - Add db7 to dns [15:20:07] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/JvWIe [15:20:08] [02miraheze/puppet] 07paladox 035f7907a - Create phab1.yaml [15:29:57] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/JvWI4 [15:29:58] [02miraheze/puppet] 07paladox 032c06c47 - Create rdb2.yaml [15:31:41] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWIz [15:31:42] [02miraheze/mw-config] 07Reception123 03a5b6e84 - sitenotice and readonly for zhdelwiki database migration to db5 [15:32:30] !log PURGE BINARY LOGS BEFORE '2020-02-12 12:00:00'; on db4 [15:32:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [15:37:15] [02mw-config] 07Reception123 commented on pull request 03#2863: T5014: set navbarIcon to true if Foreground is used - 13https://git.io/JvWID [15:42:16] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/JvWIA [15:42:17] [02miraheze/puppet] 07paladox 03506222a - Create cp7.yaml [15:45:16] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWLm [15:45:18] [02miraheze/mw-config] 07Reception123 03b4513b1 - zhdelwiki to db5 (c2) [15:48:08] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/JvWLl [15:48:10] [02miraheze/puppet] 07paladox 03d9a645e - Create services2.yaml [15:48:31] !log moved zhdelwiki from db4 to db5 [15:48:40] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWL0 [15:48:42] [02miraheze/mw-config] 07Reception123 0317c1644 - rm sitenotice+readonly for zhdelwiki and add sitenotice for toxicfandomsandhatedomswiki moving to db5 [15:48:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [15:49:27] paladox|UKInEU: can I enable Puppet on lizardfs6? [15:49:36] in order to get the sitenotice for this wiki since it is actively used [15:50:21] i guess you can [15:50:34] RECOVERY - lizardfs6 Puppet on lizardfs6 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:52:34] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 2604:180:0:33b::2/cpweb [15:52:37] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 2400:6180:0:d0::403:f001/cpweb [15:54:30] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [15:54:35] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [15:54:48] paladox|UKInEU: why is lizardfs6 puppet disabled anyway? [15:55:11] I am running a matomo script (which is failing on misc2 as it needs more ram to run) [15:55:24] so i hacked the php cli ini file to increase the memory [15:55:27] oh ok [15:59:16] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWLD [15:59:17] [02miraheze/mw-config] 07Reception123 037c316bf - readonly for toxicfandomsandhatedoms [16:07:06] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWtT [16:07:07] [02miraheze/mw-config] 07Reception123 03b79da7f - toxicfandomsandhatedomswiki to db5 (c2) [16:08:26] miraheze/mw-config/master/b79da7f - Reception123 The build was broken. https://travis-ci.org/miraheze/mw-config/builds/649524818 [16:08:47] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWtt [16:08:48] [02miraheze/mw-config] 07Reception123 03bea9c39 - fix [16:09:41] ucroniaswiki no working! http 500 [16:09:56] hispano76: really sorry that was my fault it should be back in a few seconds [16:10:07] miraheze/mw-config/master/bea9c39 - Reception123 The build was fixed. https://travis-ci.org/miraheze/mw-config/builds/649525897 [16:10:13] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/JvWtY [16:10:15] [02miraheze/puppet] 07paladox 037135295 - Create cp6.yaml [16:10:16] PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 3 backends are down. lizardfs6 mw2 mw3 [16:10:23] PROBLEM - cp4 Varnish Backends on cp4 is CRITICAL: 2 backends are down. mw2 mw3 [16:10:36] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWtO [16:10:37] [02miraheze/mw-config] 07Reception123 03c3de905 - rm sitenotice and readonly for toxicfandomsandhatedomswiki [16:10:40] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 4 datacenters are down: 107.191.126.23/cpweb, 128.199.139.216/cpweb, 2a00:d880:5:8ea::ebc7/cpweb, 2607:5300:205:200::17f6/cpweb [16:10:42] Reception123 ^ [16:10:45] paladox|UKInEU: aware, fixed [16:10:51] ok [16:10:54] !log move toxicfandomsandhatedomswiki from db4 to db5 [16:11:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [16:12:11] Zppix: paladox|UKInEU 16G on db4! that should be enough to last until the migration which should be soon [16:12:12] RECOVERY - cp3 Varnish Backends on cp3 is OK: All 6 backends are healthy [16:12:18] ok [16:12:23] RECOVERY - cp4 Varnish Backends on cp4 is OK: All 7 backends are healthy [16:12:38] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [16:14:49] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/JvWtl [16:14:51] [02miraheze/puppet] 07paladox 0371366c9 - Create db7.yaml [16:28:07] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWtN [16:28:08] [02miraheze/puppet] 07paladox 03012a9f8 - salt minion: Allow configuring the master [16:29:00] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWtx [16:29:02] [02miraheze/puppet] 07paladox 03d454fe7 - puppet2: Install salt master && also configure all new servers to use it [16:29:03] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [16:29:05] [02puppet] 07paladox opened pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:29:35] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:29:36] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWqv [16:29:38] [02miraheze/puppet] 07paladox 03d2acf65 - Update puppet2.yaml [16:30:13] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWqJ [16:30:15] [02miraheze/puppet] 07paladox 0330d663e - Update services2.yaml [16:30:16] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:30:25] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWqT [16:30:27] [02miraheze/puppet] 07paladox 03161699f - Update services1.yaml [16:30:28] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:30:37] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWqk [16:30:38] [02miraheze/puppet] 07paladox 0385c56e5 - Update rdb2.yaml [16:30:40] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:30:51] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWqI [16:30:52] [02miraheze/puppet] 07paladox 03b8f0032 - Update rdb1.yaml [16:30:54] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:31:08] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWqL [16:31:10] [02miraheze/puppet] 07paladox 03ceb1ab4 - Update phab1.yaml [16:31:11] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:31:22] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWqt [16:31:24] [02miraheze/puppet] 07paladox 03b6fb8ae - Update mw7.yaml [16:31:25] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:31:34] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWqO [16:31:36] [02miraheze/puppet] 07paladox 03c3116d2 - Update mw6.yaml [16:31:37] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:31:51] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWqs [16:31:53] [02miraheze/puppet] 07paladox 03bdaf656 - Update mw5.yaml [16:31:55] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:32:03] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWqG [16:32:04] [02miraheze/puppet] 07paladox 0359b9b72 - Update mw4.yaml [16:32:06] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:32:15] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWqZ [16:32:16] [02miraheze/puppet] 07paladox 034344905 - Update db7.yaml [16:32:18] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:32:26] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWqn [16:32:28] [02miraheze/puppet] 07paladox 03fc08bf9 - Update db6.yaml [16:32:29] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:32:51] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWqC [16:32:53] [02miraheze/puppet] 07paladox 03311ccf7 - Update cp7.yaml [16:32:54] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:33:02] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWq8 [16:33:03] [02miraheze/puppet] 07paladox 03e76e01e - Update cp6.yaml [16:33:05] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:33:20] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWq4 [16:33:22] [02miraheze/puppet] 07paladox 0326ec6d1 - Update gluster1.yaml [16:33:23] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:33:35] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWqB [16:33:37] [02miraheze/puppet] 07paladox 03bb0a166 - Update jobrunner1.yaml [16:33:38] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:33:47] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWq0 [16:33:49] [02miraheze/puppet] 07paladox 039dcc68a - Update mail1.yaml [16:33:50] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:34:11] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWqE [16:34:12] [02miraheze/puppet] 07paladox 03ef6c4da - Update test2.yaml [16:34:14] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:35:21] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWq2 [16:35:23] [02miraheze/puppet] 07paladox 035dab1da - Update site.pp [16:35:24] [02puppet] 07paladox synchronize pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:35:51] [02puppet] 07paladox closed pull request 03#1218: puppet2: Install salt master && also configure all new servers to use it - 13https://git.io/JvWtp [16:35:52] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±19] 13https://git.io/JvWqw [16:35:54] [02miraheze/puppet] 07paladox 03ef5d351 - puppet2: Install salt master && also configure all new servers to use it (#1218) [16:35:55] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-1 [16:35:57] [02puppet] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbiAS [16:37:21] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWqr [16:37:23] [02miraheze/puppet] 07paladox 032c253d7 - Update master.pp [16:38:29] [02mw-config] 07RhinosF1 commented on pull request 03#2863: T5014: set navbarIcon to true if Foreground is used - 13https://git.io/JvWqo [16:39:23] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWq6 [16:39:24] [02miraheze/puppet] 07paladox 0351b2150 - Update minion.pp [16:44:07] Reception123: around? [16:44:33] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWqb [16:44:34] [02miraheze/puppet] 07paladox 035de8e6e - salt: Fix installing [16:44:36] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [16:44:37] [02puppet] 07paladox opened pull request 03#1219: salt: Fix installing - 13https://git.io/JvWqN [16:45:28] RhinosF1: kind of [16:45:46] Reception123: did you github ban https://github.com/sevillafunciona [16:45:47] [ sevillafunciona · GitHub ] - github.com [16:46:10] RhinosF1: I tried but there was an error twice [16:46:14] so I'm waiting before trying again [16:46:47] Reception123: kk, try now before i update the github abuse ticket [16:46:53] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWme [16:46:54] [02miraheze/puppet] 07paladox 034fed796 - Update key.pp [16:46:56] [02puppet] 07paladox synchronize pull request 03#1219: salt: Fix installing - 13https://git.io/JvWqN [16:46:59] .mhca sevillafunciona [16:46:59] https://meta.miraheze.org/wiki/Special:CentralAuth/sevillafunciona [16:47:04] .wmca sevillafunciona [16:47:04] https://meta.wikimedia.org/wiki/Special:CentralAuth/sevillafunciona [16:47:07] [02puppet] 07paladox closed pull request 03#1219: salt: Fix installing - 13https://git.io/JvWqN [16:47:09] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JvWmv [16:47:10] [02miraheze/puppet] 07paladox 0318872ff - salt: Fix installing (#1219) * salt: Fix installing * Update key.pp [16:48:19] Reception123: irc ban widens now to 144.178.13* to cover [16:48:38] I’m going to do a whois and range check [16:51:01] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-1 [16:51:02] [02puppet] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbiAS [16:54:53] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWml [16:54:54] [02miraheze/puppet] 07paladox 032479960 - salt: Update salt master finger for new servers [17:09:33] Reception, thanks for helping with wikEd. FYI... Download as PDF still has cookie banner. [17:10:01] hi [17:10:06] I'm looking into why it doesn't work [17:10:08] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWYT [17:10:10] [02miraheze/puppet] 07paladox 038cbac1e - Install phabricator on phab1 [17:10:10] as for PDF paladox|UKInEU ^^ [17:10:11] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [17:10:13] [02puppet] 07paladox opened pull request 03#1220: Install phabricator on phab1 - 13https://git.io/JvWYk [17:10:21] Weee GH soam [17:10:23] Spam [17:11:06] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWYq [17:11:08] [02miraheze/puppet] 07paladox 03b73e9b3 - Update site.pp [17:11:09] [02puppet] 07paladox synchronize pull request 03#1220: Install phabricator on phab1 - 13https://git.io/JvWYk [17:14:53] [02puppet] 07paladox closed pull request 03#1220: Install phabricator on phab1 - 13https://git.io/JvWYk [17:14:55] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JvWYG [17:14:56] [02miraheze/puppet] 07paladox 0301cc1ac - Install phabricator on phab1 (#1220) * Install phabricator on phab1 * Update site.pp [17:14:58] [02puppet] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbiAS [17:14:59] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-1 [17:15:30] :) [17:18:18] RhinosF1: any ideas why wikiEd wouldn't work? do you have time to give it a test on test1/testwiki? [17:18:33] I can't figure out why it's not working on https://nenawiki.org/ [17:18:36] [ NENA Knowledge Base ] - nenawiki.org [17:18:38] Reception123: not really [17:18:50] Zppix: ^ [17:20:00] I could crack at it maybe if i have time but no promises Reception123 (im a bit busy with the thing in -staff [17:20:17] ok [17:21:08] Then again wikied i dont think is even supported by/on wmf projects Reception123 [17:21:35] So it maybe as simple as its no longer supported and/or compatible [17:22:46] really? [17:23:26] wikied were removed and is now unsupported. [17:23:34] i think i removed it [17:23:34] Thats what i thought [17:23:59] I can still look at it if time permits [17:24:15] well I tried the Gadget way [17:24:18] Reception & Zppix, if you're searching for it, it is actually spelled wikEd. [17:24:56] Oh [17:25:25] https://www.mediawiki.org/wiki/Extension:Education_Program [17:25:26] [ Extension:Education Program - MediaWiki ] - www.mediawiki.org [17:25:36] paladox|UKInEU: thats not what hes talking about [17:25:42] oh [17:25:49] What hes talking about is completely different then what we thought [17:25:59] So i dont know how much help ill be [17:26:59] yes, it's an editor which I enabled as a gadget on the wiki [17:27:24] Zppix: MediaWiki:Gadget-wikEd.js [17:27:27] * https://nenawiki.org/wiki/MediaWiki:Gadget-wikied.js [17:27:28] [ MediaWiki:Gadget-wikied.js - NENA Knowledge Base ] - nenawiki.org [17:28:42] Idk [17:30:42] Page doesnt exist Reception123 ? [17:32:01] ** MediaWiki:Gadget-wikEd.js [17:32:10] it's case sensitive [17:32:39] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWYN [17:32:40] [02miraheze/puppet] 07paladox 032a1504d - Update phab1.yaml [17:32:52] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWYx [17:32:54] [02miraheze/puppet] 07paladox 037911980 - Update services2.yaml [17:33:03] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWYh [17:33:04] [02miraheze/puppet] 07paladox 033be6569 - Update services1.yaml [17:33:15] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWOe [17:33:17] [02miraheze/puppet] 07paladox 0376c5a0d - Update db6.yaml [17:33:27] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWOv [17:33:29] [02miraheze/puppet] 07paladox 03e7b68a5 - Update db7.yaml [17:33:38] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWOf [17:33:39] [02miraheze/puppet] 07paladox 03d61c172 - Update gluster1.yaml [17:33:58] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWOU [17:34:00] [02miraheze/puppet] 07paladox 039a4d6c8 - Update cp6.yaml [17:34:09] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWOT [17:34:11] [02miraheze/puppet] 07paladox 037b23afb - Update cp7.yaml [17:34:25] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWOt [17:34:27] [02miraheze/puppet] 07paladox 0350e7e92 - Update rdb1.yaml [17:34:39] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWOq [17:34:40] [02miraheze/puppet] 07paladox 038c13adf - Update rdb2.yaml [17:35:13] Reception123: have you tried using https:// instead of // [17:37:04] I used https:// [17:38:14] Reception123: you try just using the code directly instead of calling it from enwiki [17:40:19] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/JvWOC [17:40:20] [02miraheze/puppet] 07paladox 03e3978e8 - Create ns2.yaml [17:40:22] ok [17:47:31] Reception123: any luck? [17:49:14] Zppix: just did now so looking [17:53:55] Zppix: nope :( [17:57:42] any other ideas? [17:58:01] It looks I might be able to install it myself. I did it on ndg.nenawiki.org by following the instructions here: https://en.wikipedia.org/wiki/User:Cacycle/wikEd_installation [17:58:02] [WIKIPEDIA] User:Cacycle/wikEd installation | "..." [17:59:15] MikeV: I also tried those but it didn't seem to work [18:00:14] Reception, All I did was paste the code on that page into Mediawiki:common.js [18:00:31] MikeV: Oh, I didn't know that was an option I thought it had to be a Gadget [18:01:07] MikeV: Oh yes, now I see that's a way easier way [18:01:13] does CodeEditor work? [18:01:14] Reception, they have quite a few installation options. [18:03:08] Reception, I didn't see it. wikEd can be activated by clicking on a new icon that appears next to "Log out" on top menu. I don't know if there would be a conflict because I seleted "CodeEditor" extension. [18:04:01] Reception, maybe you should remove the gadget. [18:04:45] MikeV: yes, since the easier option works [18:09:30] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvW3Y [18:09:32] [02miraheze/puppet] 07paladox 030dce610 - Install dns on ns2 [18:11:40] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvW3G [18:11:42] [02miraheze/puppet] 07paladox 030ca5aaf - dns: Fix installing on buster [18:16:04] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvW34 [18:16:06] [02miraheze/puppet] 07paladox 0346d1e60 - Install redis on rdb2 [18:17:39] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvW30 [18:17:41] [02miraheze/puppet] 07paladox 03b446af5 - Install services on services2 [18:17:42] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [18:17:44] [02puppet] 07paladox opened pull request 03#1221: Install services on services2 - 13https://git.io/JvW3E [18:18:22] [02puppet] 07paladox closed pull request 03#1221: Install services on services2 - 13https://git.io/JvW3E [18:18:24] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvW3u [18:18:25] [02miraheze/puppet] 07paladox 039db90c1 - Install services on services2 (#1221) [18:18:27] [02puppet] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbiAS [18:18:28] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-1 [18:19:14] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvW3g [18:19:16] [02miraheze/puppet] 07paladox 03234c742 - fix [18:28:17] paladox|UKInEU: is it still just cp8 pooled? [18:28:50] cp2/cp3/cp4/cp8 are pooled [18:28:58] paladox|UKInEU: from ovh [18:29:20] oh, yeh cp8 is the only thing pooled [18:29:34] Kk [18:39:58] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvW3j [18:40:00] [02miraheze/puppet] 07paladox 035656cf6 - Install varnish on cp[67] [18:40:01] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [18:40:03] [02puppet] 07paladox opened pull request 03#1222: Install varnish on cp[67] - 13https://git.io/JvWse [18:40:40] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWsv [18:40:41] [02miraheze/puppet] 07paladox 031bcdb34 - Update cp6.yaml [18:40:43] [02puppet] 07paladox synchronize pull request 03#1222: Install varnish on cp[67] - 13https://git.io/JvWse [18:40:55] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWsf [18:40:56] [02miraheze/puppet] 07paladox 03f5f4557 - Update cp7.yaml [18:40:58] [02puppet] 07paladox synchronize pull request 03#1222: Install varnish on cp[67] - 13https://git.io/JvWse [18:41:01] Reception123: yeah idk sorry [18:42:47] [02puppet] 07paladox closed pull request 03#1222: Install varnish on cp[67] - 13https://git.io/JvWse [18:42:49] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/JvWsT [18:42:50] [02miraheze/puppet] 07paladox 0374969eb - Install varnish on cp[67] (#1222) * Install varnish on cp[67] * Update cp6.yaml * Update cp7.yaml [18:42:51] [02puppet] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbiAS [18:42:53] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-1 [18:43:09] paladox|UKInEU: im shocked you've not managed to get Not-5a0a klined :P [18:43:16] heh [18:43:51] Zppix: it should be exempt from most things [18:44:04] RhinosF1: ik it was a joke [18:44:24] * RhinosF1 thinks he set it up for here when sigyn was added [18:45:01] I did icinga-miraheze for sure [18:45:07] I can’t remember now [18:53:11] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 107.191.126.23/cpweb, 128.199.139.216/cpweb [18:55:14] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 2 datacenters are down: 81.4.109.133/cpweb, 51.161.32.127/cpweb [18:57:13] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [18:57:18] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [19:08:34] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 2607:5300:205:200::17f6/cpweb [19:11:41] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 81.4.109.133/cpweb [19:12:38] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [19:13:39] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [19:16:41] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWsp [19:16:43] [02miraheze/puppet] 07paladox 0330d7b53 - Install mariadb on db7 [19:16:44] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [19:16:46] [02puppet] 07paladox opened pull request 03#1223: Install mariadb on db7 - 13https://git.io/JvWsh [19:18:27] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWGv [19:18:28] [02miraheze/puppet] 07paladox 034ed3857 - Update site.pp [19:18:30] [02puppet] 07paladox synchronize pull request 03#1223: Install mariadb on db7 - 13https://git.io/JvWsh [19:21:05] [02puppet] 07paladox closed pull request 03#1223: Install mariadb on db7 - 13https://git.io/JvWsh [19:21:07] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JvWGk [19:21:08] [02miraheze/puppet] 07paladox 039baeb7c - Install mariadb on db7 (#1223) [19:22:24] [02puppet] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbiAS [19:22:25] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-1 [19:22:58] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWGt [19:22:59] [02miraheze/puppet] 07paladox 03f9242f3 - Update config.pp [19:25:46] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 4 datacenters are down: 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb, 51.161.32.127/cpweb, 2607:5300:205:200::17f6/cpweb [19:27:42] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [19:42:01] PROBLEM - cp8 Current Load on cp8 is WARNING: WARNING - load average: 1.67, 1.81, 1.29 [19:43:59] RECOVERY - cp8 Current Load on cp8 is OK: OK - load average: 0.73, 1.41, 1.21 [19:45:08] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWGH [19:45:09] [02miraheze/services] 07MirahezeSSLBot 030a943d1 - BOT: Updating services config for wikis [19:47:55] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/JvWGb [19:47:56] [02miraheze/puppet] 07paladox 03f5b157c - Create mon1.yaml [19:48:36] PROBLEM - cp8 Current Load on cp8 is CRITICAL: CRITICAL - load average: 15.70, 6.59, 3.20 [19:50:30] PROBLEM - cp8 Varnish Backends on cp8 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [19:54:43] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 2604:180:0:33b::2/cpweb [19:57:09] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWZL [19:57:10] [02miraheze/dns] 07paladox 03afeb5f9 - Depool cp2 [19:57:33] RECOVERY - cp8 Varnish Backends on cp8 is OK: All 7 backends are healthy [19:58:38] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [20:00:45] paladox|UKInEU: is cp2 gone forever now or? [20:03:57] PROBLEM - cp8 Puppet on cp8 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [20:05:56] RECOVERY - cp8 Puppet on cp8 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [20:16:32] RhinosF1: i believe so [20:17:18] RhinosF1 it will soon, all i've done is depooled it. [20:26:49] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWZA [20:26:50] [02miraheze/puppet] 07paladox 033bb9fab - mon1: Install Grafana, Icinga2 and Prometheus Also setup irc bots [20:26:52] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [20:26:53] [02puppet] 07paladox opened pull request 03#1224: mon1: Install Grafana, Icinga2 and Prometheus - 13https://git.io/JvWZx [20:27:23] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWZp [20:27:25] [02miraheze/puppet] 07paladox 03649e895 - Update mon1.yaml [20:27:26] [02puppet] 07paladox synchronize pull request 03#1224: mon1: Install Grafana, Icinga2 and Prometheus - 13https://git.io/JvWZx [20:28:00] [02puppet] 07paladox closed pull request 03#1224: mon1: Install Grafana, Icinga2 and Prometheus - 13https://git.io/JvWZx [20:28:01] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JvWZj [20:28:03] [02miraheze/puppet] 07paladox 03ae175a3 - mon1: Install Grafana, Icinga2 and Prometheus (#1224) [20:28:04] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-1 [20:28:06] [02puppet] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbiAS [20:29:35] chanserv maint starting soon so imma op myself in case [20:30:10] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWnv [20:30:12] [02miraheze/puppet] 07paladox 034d981ae - Update init.pp [20:31:04] paladox|UKInEU: kk, will update wiki later [20:33:39] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWnI [20:33:41] [02miraheze/puppet] 07paladox 03cd7c4ba - ircecho: Add support for python3.7 [20:35:48] PROBLEM - cp8 Current Load on cp8 is WARNING: WARNING - load average: 0.55, 0.76, 1.81 [20:36:43] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWnO [20:36:44] [02miraheze/puppet] 07paladox 039630ece - fix [20:37:23] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWns [20:37:24] [02miraheze/puppet] 07paladox 030d5c5bf - fix [20:37:48] RECOVERY - cp8 Current Load on cp8 is OK: OK - load average: 0.56, 0.70, 1.66 [20:38:42] well heh [20:42:26] !log create more various dummy dbs to prevent issues with wiki creations [20:42:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [20:43:35] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [20:47:23] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWnz [20:47:25] [02miraheze/puppet] 07paladox 03c80cdc1 - Update init.pp [20:49:40] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [20:49:55] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWnP [20:49:56] [02miraheze/puppet] 07paladox 034aa0ced - monitoring: Use ::mariadb::packages [20:51:35] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWnX [20:51:37] [02miraheze/puppet] 07paladox 031156fe8 - monitoring: Fix installing mariadb-client [20:52:02] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWnM [20:52:03] [02miraheze/puppet] 07paladox 0342ec49a - mon1: Install mariadb 10.4 (client) [21:12:00] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWc3 [21:12:02] [02miraheze/puppet] 07paladox 036be7b8a - mon1: Install php7.3 [21:12:07] damn, typo, ment to be 7.4 [21:12:19] [02puppet] 07paladox commented on commit 036be7b8a2a9975b7b4af9e3ef82a87b8c7d8e2732 - 13https://git.io/JvWcs [21:13:17] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWcc [21:13:17] [02miraheze/puppet] 07paladox 03262e4b7 - php: Add support for php7.4 [21:13:17] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [21:13:17] [02puppet] 07paladox opened pull request 03#1225: php: Add support for php7.4 - 13https://git.io/JvWcC [21:13:34] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvWcl [21:13:35] [02miraheze/puppet] 07paladox 035b3379e - Update init.pp [21:13:37] [02puppet] 07paladox synchronize pull request 03#1225: php: Add support for php7.4 - 13https://git.io/JvWcC [21:15:29] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+2/-0/±1] 13https://git.io/JvWcu [21:15:31] [02miraheze/puppet] 07paladox 03a9354b5 - Update init.pp [21:15:32] [02puppet] 07paladox synchronize pull request 03#1225: php: Add support for php7.4 - 13https://git.io/JvWcC [21:15:50] [02puppet] 07paladox closed pull request 03#1225: php: Add support for php7.4 - 13https://git.io/JvWcC [21:15:51] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+2/-0/±2] 13https://git.io/JvWcg [21:15:53] [02miraheze/puppet] 07paladox 03455b164 - php: Add support for php7.4 (#1225) * php: Add support for php7.4 * Update init.pp [21:15:54] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-1 [21:15:56] [02puppet] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbiAS [21:17:34] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWcw [21:17:36] [02miraheze/puppet] 07paladox 037349915 - Update mon1.yaml [21:24:05] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvWcH [21:24:07] [02miraheze/puppet] 07paladox 03028fa2b - mont1: Downgrade to php 7.3 [21:32:43] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 2 datacenters are down: 2a00:d880:5:8ea::ebc7/cpweb, 51.161.32.127/cpweb [21:34:39] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [21:34:52] Hey apap [21:35:23] hi [21:35:44] FYI: The internal wiki for zppixbot will be updated soon. This should not affect ZppixBot but apologies if anything breaks. [22:27:44] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+1/-0/±0] 13https://git.io/JvWWA [22:27:46] [02miraheze/puppet] 07paladox 03d4eb989 - Introduce cloud role [22:27:48] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [22:27:49] [02puppet] 07paladox opened pull request 03#1226: Introduce cloud role - 13https://git.io/JvWWx [22:29:59] paladox|UKInEU: upgrading mediawiki is boring I feel sorry for you [22:30:06] heh [22:33:29] paladox|UKInEU: I am doing zppixbot wiki 1.31.0 - 1.34.0 going 1.31.0 - 1.31.6 and then up from each 1.3x branch to avoid breaking too much [22:33:36] heh [22:34:08] !log begging migration from lizardfs6 to gluster1 [22:34:12] err [22:34:17] typo [22:34:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [22:35:38] paladox|UKInEU: you probably will be begging for it to do something by the end so it can't be far off but good luck [22:51:57] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2607:5300:205:200::17f6/cpweb [22:53:57] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [22:57:54] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 2 datacenters are down: 2a00:d880:5:8ea::ebc7/cpweb, 2607:5300:205:200::17f6/cpweb [22:58:06] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 2400:6180:0:d0::403:f001/cpweb, 51.161.32.127/cpweb [22:59:51] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [23:00:04] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [23:04:06] RECOVERY - lizardfs6 GlusterFS port 49152 on lizardfs6 is OK: TCP OK - 0.016 second response time on 54.36.165.161 port 49152 [23:04:39] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2a00:d880:5:8ea::ebc7/cpweb, 51.161.32.127/cpweb, 2607:5300:205:200::17f6/cpweb [23:04:53] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 5 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 81.4.109.133/cpweb, 2607:5300:205:200::17f6/cpweb [23:06:41] PROBLEM - cp8 Stunnel Http for mw2 on cp8 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:06:55] PROBLEM - cp3 Stunnel Http for mw2 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:07:25] PROBLEM - cp2 Stunnel Http for mw2 on cp2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:07:27] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [23:08:44] RECOVERY - cp8 Stunnel Http for mw2 on cp8 is OK: HTTP OK: HTTP/1.1 200 OK - 15302 bytes in 6.068 second response time [23:08:59] RECOVERY - cp3 Stunnel Http for mw2 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15288 bytes in 5.340 second response time [23:09:26] RECOVERY - cp2 Stunnel Http for mw2 on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 15302 bytes in 2.568 second response time [23:11:27] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb [23:15:27] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [23:16:59] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [23:20:25] PROBLEM - lizardfs6 GlusterFS port 49152 on lizardfs6 is CRITICAL: connect to address 54.36.165.161 and port 49152: Connection refused [23:20:26] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvW8J [23:20:27] [02miraheze/puppet] 07paladox 03e0904b2 - Update gluster1.yaml [23:21:38] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvW8U [23:21:40] [02miraheze/puppet] 07paladox 03a40a924 - Update gluster1.yaml [23:24:19] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvW8k [23:24:20] [02miraheze/puppet] 07paladox 032280fdc - Update init.pp [23:25:34] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvW8L [23:25:35] [02miraheze/puppet] 07paladox 03390a26f - Update gluster1.yaml [23:38:26] !log reboot lizardfs6 [23:41:00] RECOVERY - lizardfs6 GlusterFS port 49152 on lizardfs6 is OK: TCP OK - 0.013 second response time on 54.36.165.161 port 49152 [23:42:49] !log apt-get upgrade - lizardfs67 [23:42:59] !log lizardfs6 not 67 [23:43:48] PROBLEM - lizardfs6 JobRunner Service on lizardfs6 is CRITICAL: PROCS CRITICAL: 0 processes with args 'redisJobRunnerService' [23:43:54] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 7 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 51.161.32.127/cpweb, 2607:5300:205:200::17f6/cpweb [23:43:54] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 5 datacenters are down: 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 51.161.32.127/cpweb [23:43:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [23:44:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [23:44:36] PROBLEM - lizardfs6 JobChron Service on lizardfs6 is CRITICAL: PROCS CRITICAL: 0 processes with args 'redisJobChronService' [23:50:03] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [23:50:08] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvW8M [23:50:10] [02miraheze/services] 07MirahezeSSLBot 03b33e801 - BOT: Updating services config for wikis [23:50:15] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [23:51:50] RECOVERY - lizardfs6 JobRunner Service on lizardfs6 is OK: PROCS OK: 1 process with args 'redisJobRunnerService' [23:53:02] RECOVERY - lizardfs6 JobChron Service on lizardfs6 is OK: PROCS OK: 1 process with args 'redisJobChronService' [23:55:41] PROBLEM - lizardfs6 GlusterFS port 49152 on lizardfs6 is CRITICAL: connect to address 54.36.165.161 and port 49152: Connection refused