[00:03:27] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 5 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb [00:03:46] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 3 datacenters are down: 107.191.126.23/cpweb, 128.199.139.216/cpweb, 81.4.109.133/cpweb [00:05:25] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [00:05:46] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [01:32:35] [02miraheze/puppet] 07paladox pushed 032 commits to 03master [+3/-0/±6] 13https://git.io/JemGX [01:32:37] [02miraheze/puppet] 07paladox 03fd42698 - Initial creation of gluster module [01:32:38] [02miraheze/puppet] 07paladox 03943540f - Merge branch 'master' of github.com:miraheze/puppet [01:36:26] Hello Username02! If you have any questions feel free to ask and someone should answer soon. [01:43:18] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±6] 13https://git.io/JemGd [01:43:20] [02miraheze/puppet] 07paladox 0330a1bce - Merge branch 'master' of github.com:miraheze/puppet [02:03:55] PROBLEM - lizardfs5 Puppet on lizardfs5 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [02:43:55] RECOVERY - lizardfs5 Puppet on lizardfs5 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:54:40] !log renamed sudo -u www-data php /srv/mediawiki/w/extensions/CreateWiki/maintenance/renameWiki.php --wiki=loginwiki --rename ogftreveldwiki evsmapswiki Reception123 [05:54:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [05:58:20] !log sudo -u www-data php /srv/mediawiki/w/extensions/CreateWiki/maintenance/renameWiki.php --wiki=loginwiki --rename uncyclopedia2wiki uncyc2wiki Reception123 [05:58:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [06:00:11] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jemch [06:00:12] [02miraheze/services] 07MirahezeSSLBot 03aa3b96d - BOT: Updating services config for wikis [06:11:55] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JemCk [06:11:56] [02miraheze/mw-config] 07Reception123 03d0fe0a9 - openhatchwiki readonly for db migration [06:14:09] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JemCI [06:14:10] [02miraheze/mw-config] 07Reception123 036398355 - correction [06:20:03] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JemCO [06:20:05] [02miraheze/mw-config] 07Reception123 035550059 - openhatchwiki readonly for db migration [06:26:56] RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 3126 MB (12% inode=94%); [06:29:10] PROBLEM - cp4 Stunnel Http for mw3 on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [06:29:17] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb [06:31:07] RECOVERY - cp4 Stunnel Http for mw3 on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 24500 bytes in 0.006 second response time [06:31:17] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [06:31:48] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JemCn [06:31:49] [02miraheze/mw-config] 07Reception123 0339b6ebe - rm readonly+sitenotice for openhatchwiki [06:36:41] RECOVERY - mw6 Puppet on mw6 is OK: OK: Puppet is currently enabled, last run 37 seconds ago with 0 failures [06:54:41] PROBLEM - mw6 Puppet on mw6 is CRITICAL: CRITICAL: Puppet has 4 failures. Last run 2 minutes ago with 4 failures. Failed resources (up to 3 shown): Exec[mathoid_npm],Exec[3d2png_npm],Exec[git_pull_MediaWiki core],Exec[/mnt/mediawiki-static] [07:01:36] !log sudo -u www-data php /srv/mediawiki/w/extensions/CreateWiki/maintenance/deleteWikis.php --wiki loginwiki --delete Reception123 [07:01:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [07:05:52] !log deleted and dropped agilesoftwaredevelopmentwiki; alphaanarchywiki; animefurryfeetwiki aqferwiki; article13wiki; asfrwiki; ayurbookswiki; bchitguideswiki besttvchannelswiki; biblioenricowiki birdhousewiki;blooperwikiwiki; bubbawiki;bus42ruswiki; capgeminiplaybookwiki;casual4casualswiki; ccwpwiki; cgsimwiki;dellplanetwikiwiki;dreddbookwiki;mirahezewiki; [07:05:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [07:06:36] !log purge binary logs before '2019-09-11 02:00:00'; on db4 [07:06:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [08:02:14] PROBLEM - glusterfs1 Puppet on glusterfs1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [08:23:10] PROBLEM - glusterfs2 Puppet on glusterfs2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [08:52:15] RECOVERY - glusterfs1 Puppet on glusterfs1 is OK: OK: Puppet is currently enabled, last run 14 seconds ago with 0 failures [09:23:10] RECOVERY - glusterfs2 Puppet on glusterfs2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [12:10:09] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JemEH [12:10:11] [02miraheze/services] 07MirahezeSSLBot 03875edb1 - BOT: Updating services config for wikis [14:18:39] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jema8 [14:18:40] [02miraheze/puppet] 07paladox 036093f07 - Include gluster role on glusterfs[12] [14:20:17] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JemaB [14:20:19] [02miraheze/puppet] 07paladox 03b525360 - Update init.pp [14:22:14] PROBLEM - glusterfs1 Puppet on glusterfs1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:23:10] PROBLEM - glusterfs2 Puppet on glusterfs2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:24:14] RECOVERY - glusterfs1 Puppet on glusterfs1 is OK: OK: Puppet is currently enabled, last run 30 seconds ago with 0 failures [14:25:10] RECOVERY - glusterfs2 Puppet on glusterfs2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:26:36] PROBLEM - glusterfs1 GlusterFS ip 185.52.2.113 on port 49153 on glusterfs1 is CRITICAL: connect to address 81.4.100.90 and port 49153: Connection refused [14:26:42] PROBLEM - glusterfs1 GlusterFS ip 185.52.1.75 on port 49153 on glusterfs1 is CRITICAL: connect to address 81.4.100.90 and port 49153: Connection refused [14:26:48] PROBLEM - glusterfs1 GlusterFS ip 185.52.1.76 on port 49153 on glusterfs1 is CRITICAL: connect to address 81.4.100.90 and port 49153: Connection refused [14:26:50] PROBLEM - glusterfs2 GlusterFS ip 185.52.1.76 on port 49153 on glusterfs2 is CRITICAL: connect to address 81.4.100.77 and port 49153: Connection refused [14:26:51] PROBLEM - glusterfs1 GlusterFS ip 185.52.2.243 on port 49153 on glusterfs1 is CRITICAL: connect to address 81.4.100.90 and port 49153: Connection refused [14:26:52] PROBLEM - glusterfs1 GlusterFS ip 81.4.100.77 on port 49153 on glusterfs1 is CRITICAL: connect to address 81.4.100.90 and port 49153: Connection refused [14:26:53] PROBLEM - glusterfs2 GlusterFS ip 81.4.100.90 on port 49153 on glusterfs2 is CRITICAL: connect to address 81.4.100.77 and port 49153: Connection refused [14:26:56] PROBLEM - glusterfs1 GlusterFS ip 81.4.100.90 on port 49153 on glusterfs1 is CRITICAL: connect to address 81.4.100.90 and port 49153: Connection refused [14:26:57] PROBLEM - glusterfs2 GlusterFS ip 81.4.100.77 on port 49153 on glusterfs2 is CRITICAL: connect to address 81.4.100.77 and port 49153: Connection refused [14:26:57] PROBLEM - glusterfs1 GlusterFS ip 81.4.121.113 on port 49153 on glusterfs1 is CRITICAL: connect to address 81.4.100.90 and port 49153: Connection refused [14:27:09] PROBLEM - glusterfs2 GlusterFS ip 185.52.1.75 on port 49153 on glusterfs2 is CRITICAL: connect to address 81.4.100.77 and port 49153: Connection refused PROBLEM - glusterfs2 GlusterFS ip 81.4.121.113 on port 49153 on glusterfs2 is CRITICAL: connect to address 81.4.100.77 and port 49153: Connection refused [14:27:21] PROBLEM - glusterfs2 GlusterFS ip 185.52.2.113 on port 49153 on glusterfs2 is CRITICAL: connect to address 81.4.100.77 and port 49153: Connection refused [14:27:26] PROBLEM - glusterfs2 GlusterFS ip 185.52.2.243 on port 49153 on glusterfs2 is CRITICAL: connect to address 81.4.100.77 and port 49153: Connection refused [14:28:55] oh [14:28:56] bugger [14:30:37] [02MirahezeMagic] 07RhinosF1 created branch 03RhinosF1-patch-1 - 13https://git.io/fQRGX [14:30:38] [02miraheze/MirahezeMagic] 07RhinosF1 pushed 031 commit to 03RhinosF1-patch-1 [+0/-0/±1] 13https://git.io/Jemah [14:30:40] [02miraheze/MirahezeMagic] 07RhinosF1 033df692a - update i18n [14:30:41] [02MirahezeMagic] 07RhinosF1 opened pull request 03#91: update i18n - 13https://git.io/Jemaj [14:31:49] ^ Don't merge [14:32:04] miraheze/MirahezeMagic/RhinosF1-patch-1/3df692a - RhinosF1 The build passed. https://travis-ci.org/miraheze/MirahezeMagic/builds/583692479 [14:34:55] [02miraheze/MirahezeMagic] 07RhinosF1 pushed 031 commit to 03RhinosF1-patch-1 [+0/-0/±1] 13https://git.io/JemVk [14:34:56] [02miraheze/MirahezeMagic] 07RhinosF1 031c3abab - Update en.json [14:34:58] [02MirahezeMagic] 07RhinosF1 synchronize pull request 03#91: update i18n - 13https://git.io/Jemaj [14:36:11] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JemVt [14:36:12] [02miraheze/puppet] 07paladox 0363ce021 - Update init.pp [14:37:52] [02MirahezeMagic] 07RhinosF1 closed pull request 03#91: update i18n - 13https://git.io/Jemaj [14:37:53] [02miraheze/MirahezeMagic] 07RhinosF1 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JemVs [14:37:55] [02miraheze/MirahezeMagic] 07RhinosF1 03f8b72fc - update i18n (#91) [14:39:32] * RhinosF1 will deploy soon [14:44:50] [02miraheze/mediawiki] 07RhinosF1 pushed 031 commit to 03REL1_33 [+0/-0/±1] 13https://git.io/JemV2 [14:44:52] [02miraheze/mediawiki] 07RhinosF1 037902d19 - Updating i18n [14:46:15] [02MirahezeMagic] 07paladox deleted branch 03RhinosF1-patch-1 - 13https://git.io/fQRGX [14:46:17] [02miraheze/MirahezeMagic] 07paladox deleted branch 03RhinosF1-patch-1 [14:46:43] thx paladox [14:52:13] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:52:16] PROBLEM - db4 Puppet on db4 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:52:20] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:52:20] PROBLEM - glusterfs2 Puppet on glusterfs2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:52:25] paladox: ^ per -staff [14:52:33] gc [14:52:43] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:52:49] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:52:52] PROBLEM - lizardfs5 Puppet on lizardfs5 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:52:53] PROBLEM - misc4 Puppet on misc4 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:52:55] PROBLEM - glusterfs1 Puppet on glusterfs1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:53:04] PROBLEM - misc3 Puppet on misc3 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:53:05] PROBLEM - mw4 Puppet on mw4 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:53:18] PROBLEM - misc2 Puppet on misc2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:53:31] PROBLEM - cp2 Puppet on cp2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:53:38] PROBLEM - bacula1 Puppet on bacula1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:53:45] PROBLEM - db5 Puppet on db5 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:53:47] PROBLEM - lizardfs4 Puppet on lizardfs4 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:54:05] PROBLEM - cp4 Puppet on cp4 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:54:05] PROBLEM - mw3 Puppet on mw3 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 3 minutes ago with 8 failures. Failed resources (up to 3 shown) [14:58:58] !log sudo -u www-data php srv/Mediawiki/w/maintenance/mergeMessageFileList.php --output /srv/mediawiki/config/ExtensionMessageFiles.php --wiki loginwiki (on test1 and mw2) [14:59:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [15:00:32] !log sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildLocalisationCache.php --wiki loginwiki [15:00:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [15:03:05] RECOVERY - misc2 Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 10 seconds ago with 0 failures [15:03:30] RECOVERY - lizardfs4 Puppet on lizardfs4 is OK: OK: Puppet is currently enabled, last run 45 seconds ago with 0 failures [15:03:33] RECOVERY - db5 Puppet on db5 is OK: OK: Puppet is currently enabled, last run 48 seconds ago with 0 failures RECOVERY - bacula1 Puppet on bacula1 is OK: OK: Puppet is currently enabled, last run 48 seconds ago with 0 failures [15:03:46] RECOVERY - db4 Puppet on db4 is OK: OK: Puppet is currently enabled, last run 54 seconds ago with 0 failures [15:03:56] RECOVERY - cp4 Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 29 seconds ago with 0 failures [15:03:57] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:04:01] RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 24 seconds ago with 0 failures [15:04:02] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:04:10] RECOVERY - glusterfs2 Puppet on glusterfs2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:04:11] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:04:21] RECOVERY - lizardfs5 Puppet on lizardfs5 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:04:39] RECOVERY - glusterfs1 Puppet on glusterfs1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:04:45] RECOVERY - misc3 Puppet on misc3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:04:46] RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:05:09] !log +on mw3 as well (last 2) [15:05:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [15:05:20] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 58 seconds ago with 0 failures [15:05:31] PROBLEM - mw2 Current Load on mw2 is CRITICAL: CRITICAL - load average: 8.07, 7.06, 4.82 [15:06:49] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:07:29] PROBLEM - mw2 Current Load on mw2 is WARNING: WARNING - load average: 6.65, 6.99, 5.06 [15:07:32] ^ load is me [15:08:37] RECOVERY - mw4 Puppet on mw4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:08:55] !log and on mw1 finally [15:09:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [15:09:27] RECOVERY - mw2 Current Load on mw2 is OK: OK - load average: 4.81, 6.21, 5.01 [15:16:22] PROBLEM - mw2 Current Load on mw2 is WARNING: WARNING - load average: 7.39, 7.80, 6.16 [15:18:22] RECOVERY - mw2 Current Load on mw2 is OK: OK - load average: 4.50, 6.75, 5.99 [17:02:47] PROBLEM - glusterfs2 Puppet on glusterfs2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:03:45] PROBLEM - glusterfs1 Puppet on glusterfs1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [17:23:46] RECOVERY - glusterfs1 Puppet on glusterfs1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [17:42:47] RECOVERY - glusterfs2 Puppet on glusterfs2 is OK: OK: Puppet is currently enabled, last run 54 seconds ago with 0 failures [17:43:46] PROBLEM - glusterfs1 Puppet on glusterfs1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [19:00:09] !log importing images for oceumenewiki T4958 [19:00:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [19:03:46] RECOVERY - glusterfs1 Puppet on glusterfs1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [19:33:22] with regards to the new RFC, why does it skip Proposal 2.7? [19:38:09] Voidwalker: not sure [19:38:18] Think it might have been taken out [19:39:18] Voidwalker: yes, 2.6 and 2.7 were merged [19:39:49] ah, makes sense then [19:44:32] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/JemXn [19:44:36] [02miraheze/puppet] 07paladox 0390c3f93 - Add gluster::mount to mediawiki role [19:45:11] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/JemXc [19:45:13] [02miraheze/puppet] 07paladox 031a03b10 - Add gluster::mount to mediawiki role [19:48:54] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JemXl [19:48:56] [02miraheze/puppet] 07paladox 03d6e42f3 - Ensure the mount directory is created [19:53:55] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JemXB [19:53:56] [02miraheze/puppet] 07paladox 03cda7ea6 - Ensure /var/lib/glusterd/secure-access is created [19:56:49] !log reinstalling glusterfs[12] [19:56:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [20:03:50] PROBLEM - glusterfs1 Puppet on glusterfs1 is CRITICAL: connect to address 81.4.100.90 port 5666: Connection refusedconnect to host 81.4.100.90 port 5666: Connection refused [20:03:54] PROBLEM - glusterfs1 Disk Space on glusterfs1 is CRITICAL: connect to address 81.4.100.90 port 5666: Connection refusedconnect to host 81.4.100.90 port 5666: Connection refused [20:04:04] PROBLEM - glusterfs1 Current Load on glusterfs1 is CRITICAL: connect to address 81.4.100.90 port 5666: Connection refusedconnect to host 81.4.100.90 port 5666: Connection refused [20:04:50] PROBLEM - glusterfs2 Puppet on glusterfs2 is CRITICAL: connect to address 81.4.100.77 port 5666: Connection refusedconnect to host 81.4.100.77 port 5666: Connection refused [20:05:17] PROBLEM - glusterfs2 Disk Space on glusterfs2 is CRITICAL: connect to address 81.4.100.77 port 5666: Connection refusedconnect to host 81.4.100.77 port 5666: Connection refused [20:06:08] PROBLEM - glusterfs2 Current Load on glusterfs2 is CRITICAL: connect to address 81.4.100.77 port 5666: Connection refusedconnect to host 81.4.100.77 port 5666: Connection refused [20:13:46] PROBLEM - glusterfs1 Puppet on glusterfs1 is UNKNOWN: NRPE: Unable to read output [20:13:49] RECOVERY - glusterfs1 Disk Space on glusterfs1 is OK: DISK OK - free space: / 312226 MB (99% inode=99%); [20:14:00] RECOVERY - glusterfs1 Current Load on glusterfs1 is OK: OK - load average: 1.68, 1.04, 0.47 [20:15:17] RECOVERY - glusterfs2 Disk Space on glusterfs2 is OK: DISK OK - free space: / 312383 MB (99% inode=99%); [20:16:08] RECOVERY - glusterfs2 Current Load on glusterfs2 is OK: OK - load average: 1.85, 1.05, 0.52 [20:16:47] PROBLEM - glusterfs2 Puppet on glusterfs2 is UNKNOWN: UNKNOWN: Failed to check. Reason is: no_summary_file [20:23:46] RECOVERY - glusterfs1 Puppet on glusterfs1 is OK: OK: Puppet is currently enabled, last run 47 seconds ago with 0 failures [20:24:47] RECOVERY - glusterfs2 Puppet on glusterfs2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [20:49:18] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/Jem1s [20:49:20] [02miraheze/puppet] 07paladox 037626316 - gluster: Fix firewall rules [20:49:21] [02puppet] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vbiAS [20:49:23] [02puppet] 07paladox opened pull request 03#1083: gluster: Fix firewall rules - 13https://git.io/Jem1G [20:51:12] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/Jem1c [20:51:14] [02miraheze/puppet] 07paladox 03b9d7877 - Update init.pp [20:51:15] [02puppet] 07paladox synchronize pull request 03#1083: gluster: Fix firewall rules - 13https://git.io/Jem1G [20:54:53] [02puppet] 07paladox synchronize pull request 03#1083: gluster: Fix firewall rules - 13https://git.io/Jem1G [20:54:55] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/Jem1C [20:54:56] [02miraheze/puppet] 07paladox 03cf7763d - Update firewall.yaml [20:55:02] [02puppet] 07paladox closed pull request 03#1083: gluster: Fix firewall rules - 13https://git.io/Jem1G [20:55:03] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/Jem1W [20:55:05] [02miraheze/puppet] 07paladox 03b058662 - gluster: Fix firewall rules (#1083) * gluster: Fix firewall rules * Update init.pp * Update firewall.yaml [20:55:06] [02puppet] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vbiAS [20:55:08] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-2 [20:57:20] Hello [20:57:28] Pongles: hi [20:57:37] How's it going Rhinos? [20:57:45] Pongles: good, u? [20:57:52] I'm doing okay. [20:57:59] nice to hear [20:58:56] how can we help today? [20:58:56] I was wondering if there was a simple way to export every page from a miraheze mediawiki with full edit history and templates. [20:59:42] pongles: Special:DataDump [21:00:43] Ah, I have to be a wiki administrator for that. Unfortunately the wiki I need to pull from is own by someone who left the group the wiki is for due to political reasons. [21:00:52] Pongles: link? [21:00:59] https://elgoonishshive.miraheze.org/wiki/Special:DataDump [21:01:00] [ Permission error - Explain El Goonish Shive ] - elgoonishshive.miraheze.org [21:01:37] I just want to make sure the work is saved, as I know they didn't really want to be the admin for long on it [21:01:49] paladox: I'm looking into it now [21:01:52] Pongles: ^ [21:02:01] Thank you [21:03:34] PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2650 MB (10% inode=94%); [21:10:46] ^moved to discord PM (doing) [21:26:27] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jem1j [21:26:29] [02miraheze/puppet] 07paladox 038819bee - Update init.pp [21:46:49] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JemM2 [21:46:50] [02miraheze/puppet] 07paladox 038a6d15f - Ensure ssl files created on the client [21:50:03] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JemMa [21:50:04] [02miraheze/puppet] 07paladox 032b9b66d - Fix syntax [21:52:13] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [21:53:04] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JemMo [21:53:05] [02miraheze/puppet] 07paladox 032cc165d - Fix syntax part 2 [21:55:09] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JemM6 [21:55:11] [02miraheze/services] 07MirahezeSSLBot 03bb3ceac - BOT: Updating services config for wikis [21:56:01] paladox: is that you with test1? [21:56:06] yup [22:00:24] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 40 seconds ago with 0 failures [22:11:45] PROBLEM - cp3 Stunnel Http for test1 on cp3 is CRITICAL: HTTP CRITICAL - No data received from host [22:11:47] PROBLEM - test1 HTTPS on test1 is CRITICAL: connect to address 185.52.2.243 and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [22:12:01] PROBLEM - cp2 Stunnel Http for test1 on cp2 is CRITICAL: HTTP CRITICAL - No data received from host [22:12:25] PROBLEM - test1 php-fpm on test1 is CRITICAL: PROCS CRITICAL: 0 processes with command name 'php-fpm7.3' [22:13:00] !log reboot test1 [22:13:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [22:13:17] PROBLEM - cp4 Stunnel Http for test1 on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:14:25] RECOVERY - test1 php-fpm on test1 is OK: PROCS OK: 3 processes with command name 'php-fpm7.3' [22:25:32] PROBLEM - test1 Disk Space on test1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:25:49] PROBLEM - test1 Current Load on test1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:26:44] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Puppet has 11 failures. Last run 3 minutes ago with 11 failures. Failed resources (up to 3 shown) [22:27:30] RECOVERY - test1 Disk Space on test1 is OK: DISK OK - free space: / 9018 MB (22% inode=98%); [22:27:43] RECOVERY - test1 Current Load on test1 is OK: OK - load average: 0.55, 0.31, 0.12 [22:29:48] RECOVERY - cp4 Stunnel Http for test1 on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 24499 bytes in 0.009 second response time [22:30:40] RECOVERY - cp2 Stunnel Http for test1 on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 24499 bytes in 0.490 second response time [22:30:50] RECOVERY - test1 HTTPS on test1 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 444 bytes in 0.010 second response time [22:30:51] RECOVERY - cp3 Stunnel Http for test1 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 24499 bytes in 0.862 second response time [22:32:55] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:40:23] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JemDU [22:40:25] [02miraheze/puppet] 07paladox 038688dfc - Make ssl files be owned by root [22:41:04] !log reset git dir on puppet1 back to what's in the git repo [22:41:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [22:42:53] PROBLEM - test1 Puppet on test1 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 11 minutes ago with 0 failures [23:06:45] PROBLEM - mw2 Current Load on mw2 is CRITICAL: CRITICAL - load average: 8.86, 7.11, 5.41 [23:08:43] RECOVERY - mw2 Current Load on mw2 is OK: OK - load average: 4.88, 6.33, 5.34 [23:12:39] PROBLEM - mw2 Current Load on mw2 is WARNING: WARNING - load average: 7.53, 6.93, 5.80 [23:14:37] PROBLEM - mw2 Current Load on mw2 is CRITICAL: CRITICAL - load average: 8.28, 7.26, 6.04 [23:16:35] PROBLEM - mw2 Current Load on mw2 is WARNING: WARNING - load average: 6.28, 6.99, 6.09 [23:18:33] RECOVERY - mw2 Current Load on mw2 is OK: OK - load average: 5.93, 6.56, 6.03 [23:30:24] PROBLEM - mw2 Current Load on mw2 is CRITICAL: CRITICAL - load average: 9.26, 7.24, 6.40 [23:32:23] PROBLEM - mw2 Current Load on mw2 is WARNING: WARNING - load average: 6.98, 6.98, 6.41 [23:34:25] PROBLEM - mw2 Current Load on mw2 is CRITICAL: CRITICAL - load average: 10.26, 8.14, 6.89 [23:36:23] PROBLEM - mw2 Current Load on mw2 is WARNING: WARNING - load average: 7.12, 7.57, 6.83 [23:38:22] RECOVERY - mw2 Current Load on mw2 is OK: OK - load average: 4.65, 6.47, 6.51 [23:53:31] PROBLEM - lizardfs5 Puppet on lizardfs5 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [23:59:31] !log enable quota and set limit for / to 584GB on gluster [23:59:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master