[00:00:01] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/fbpgR [00:00:03] [02miraheze/puppet] 07paladox 03d66be32 - swift: increase workers [00:01:56] RECOVERY - Puppet on swift1 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:03:26] PROBLEM - Puppet on misc2 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:04:06] PROBLEM - Puppet on ns1 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:11:26] RECOVERY - Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 41 seconds ago with 0 failures [00:12:06] RECOVERY - Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:18:47] PROBLEM - Varnish Backends on cp5 is CRITICAL: 3 backends are down. mw1 mw2 mw3 [00:21:45] PROBLEM - GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 4 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 172.104.111.8/cpweb, 2400:8902::f03c:91ff:fe07:444e/cpweb [00:21:47] PROBLEM - GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 4 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 172.104.111.8/cpweb, 2400:8902::f03c:91ff:fe07:444e/cpweb [00:22:45] RECOVERY - Varnish Backends on cp4 is OK: All 3 backends are healthy [00:22:47] RECOVERY - Varnish Backends on cp5 is OK: All 3 backends are healthy [00:23:47] RECOVERY - GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [01:49:26] PROBLEM - Varnish Backends on cp4 is CRITICAL: 1 backends are down. mw1 [01:51:26] RECOVERY - Varnish Backends on cp4 is OK: All 3 backends are healthy [02:04:45] PROBLEM - Varnish Backends on cp2 is CRITICAL: 1 backends are down. mw2 [02:04:47] PROBLEM - GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 81.4.109.133/cpweb [02:05:14] thats because i am currently uploading in swift [02:05:19] otherwise it seems stable [02:05:25] using swift upload [02:08:26] PROBLEM - Varnish Backends on cp4 is CRITICAL: 1 backends are down. mw3 [02:08:46] RECOVERY - Varnish Backends on cp2 is OK: All 3 backends are healthy [02:08:48] RECOVERY - GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [02:12:46] PROBLEM - GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 3 datacenters are down: 107.191.126.23/cpweb, 172.104.111.8/cpweb, 2400:8902::f03c:91ff:fe07:444e/cpweb [02:44:00] !log upgrade phabricator on misc1 [02:44:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [06:32:37] PROBLEM - Current Load on misc1 is WARNING: WARNING - load average: 1.83, 1.38, 0.85 [06:34:37] RECOVERY - Current Load on misc1 is OK: OK - load average: 1.54, 1.40, 0.92 [06:56:26] PROBLEM - HTTP 4xx/5xx ERROR Rate on cp4 is WARNING: WARNING - NGINX Error Rate is 47% [06:58:27] PROBLEM - HTTP 4xx/5xx ERROR Rate on cp4 is CRITICAL: CRITICAL - NGINX Error Rate is 61% [07:02:25] RECOVERY - HTTP 4xx/5xx ERROR Rate on cp4 is OK: OK - NGINX Error Rate is 5% [11:13:27] PROBLEM - Puppet on misc2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Apt_key[Add key: F86AA916A2195E121AEDB11437BBEE3F7AD95B3F from Apt::Source grafana_apt] [11:17:07] PROBLEM - Disk Space on db4 is WARNING: DISK WARNING - free space: / 76919 MB (20% inode=95%): [11:21:27] RECOVERY - Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 43 seconds ago with 0 failures [11:53:27] PROBLEM - Puppet on misc2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Apt_key[Add key: F86AA916A2195E121AEDB11437BBEE3F7AD95B3F from Apt::Source grafana_apt] [12:01:25] RECOVERY - Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 24 seconds ago with 0 failures [13:21:17] PROBLEM - Puppet on test1 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 7 minutes ago with 0 failures [13:25:17] PROBLEM - Puppet on nfs1 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 4 minutes ago with 0 failures [13:37:14] Hello Combusto82! If you have any questions feel free to ask and someone should answer soon. [13:38:45] i just request my wiki when it will up? [14:40:52] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbhBJ [14:40:54] [02miraheze/puppet] 07paladox 03ee0e14c - Add back support for dumps in static.miraheze.org fixes T3311 [14:43:10] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbhBl [14:43:11] [02miraheze/puppet] 07paladox 03041b7bb - Update proxy-server.conf.erb [14:45:16] RECOVERY - Puppet on nfs1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:46:33] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbhBQ [14:46:34] [02miraheze/puppet] 07paladox 037434082 - Update mediawiki.conf [14:49:16] RECOVERY - Puppet on test1 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:04:13] can anyone see the logo/favicon on my wiki? union.miraheze.org [15:07:09] Hmm, [15:07:12] I can’t [15:07:17] paladox: ^^ [15:08:32] ...and wiki1776.miraheze.org [15:08:35] PROBLEM - Bacula - Phabricator Static on bacula1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [15:08:58] hmm nope [15:09:41] paladox: any ideas how to fix it [15:10:20] nope, not currently, having a look [15:10:50] Uh [15:11:43] I think misc1 is having network issues again as phab isn’t loading either [15:12:12] paladox: ^^ [15:12:17] ok [15:13:52] And now all of our bots have quit [15:14:13] Though RC already recovered [15:14:26] yep [15:15:11] phab works now [15:15:31] Other bots just need a restart now [15:15:40] (icingabot anf [15:16:02] and adminbot)* [15:16:16] PROBLEM - Bacula - Private Git on bacula1 is WARNING: WARNING: Full, 1350 files, 1.432MB, 2018-06-24 00:05:00 (1.7 weeks ago) [15:18:34] Wiki-1776 i think i know why some of your images are not working. [15:18:44] Just looking into a fix which may take a while [15:18:47] sorry about that Wiki-1776 [15:23:03] OK [15:23:08] ok * [15:43:12] PROBLEM - Puppet on nfs1 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 2 minutes ago with 0 failures [16:09:11] [02miraheze/mw-config] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbhKm [16:09:13] [02miraheze/mw-config] 07MacFan4000 03d371492 - Rm notice [16:10:53] FYI db4 has disk space warning again [16:11:26] Though swift1 is very low with ~4GB remaining [16:13:34] That’s obviously known [16:13:56] the point of swift is to increase and distribute usage for static [16:24:28] hmm, it looks like CollapsibleVector may not be compatible with the current version of mw [17:03:57] Hi! Just contacting you to tell that here are no high priority open tasks. Nice job! [17:04:36] heh, nope, it'd be broken no matter which version of MW you're using [17:09:56] Voidwalker oh? [17:10:03] Voidwalker any idea how to fix it? [17:10:12] * paladox is the maintainer of that ext. [17:10:33] yeh i want to reimage nfs1 as soon as i get things working (today) [17:10:35] https://github.com/wikimedia/mediawiki-extensions-CollapsibleVector/blob/master/modules/ext.collapsiblevector.collapsibleNav.js#L135 change .delegate to .on [17:10:35] Title: [ mediawiki-extensions-CollapsibleVector/ext.collapsiblevector.collapsibleNav.js at master · wikimedia/mediawiki-extensions-CollapsibleVector · GitHub ] - github.com [17:11:05] Voidwalker ok thanks! [17:11:06] doing [17:11:31] and it'd likely resolve https://phabricator.wikimedia.org/T193779 [17:11:31] Title: [ ⚓ T193779 Replaced jquery delegate breaks compatibility with Vector Skin of REL_1_31 (RC0) ] - phabricator.wikimedia.org [17:12:23] Voidwalker like https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/CollapsibleVector/+/444029/ ? [17:12:23] Title: [ Gerrit Code Review ] - gerrit.wikimedia.org [17:12:42] yup [17:12:47] ok great merging [17:15:17] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_31 [+0/-0/±1] 13https://git.io/fbhMp [17:15:19] [02miraheze/mediawiki] 07paladox 033fac245 - Update CollapsibleVector to upstream [17:17:36] Voidwalker i've deployed it now ^^ [17:17:40] Could you test please? :) [17:17:48] sure [17:18:21] thanks! [17:19:37] .weather [17:19:39] Yahoo! Weather - Portland, ME, US: Partly Cloudy, 27°C (80°F), Humidity: 71%, Fresh breeze 9.8m/s (↑) [17:19:45] .weather [17:19:46] Yahoo! Weather - London, England, GB: Mostly Cloudy, 29°C (84°F), Humidity: 26%, Gentle breeze 4.9m/s (↘) [17:19:51] .weather northampton [17:19:52] Yahoo! Weather - Northampton, England, GB: Partly Cloudy, 28°C (82°F), Humidity: 29%, Gentle breeze 4.9m/s (↘) [17:20:01] More like 90°F :P [17:20:07] Very hot here [17:20:14] even hotter here :) [17:21:29] And it’s way more humid here [17:21:57] hmm, something's still misbehaving [17:22:20] paladox: ^^ [17:22:26] Voidwalker oh? [17:26:00] yeah, I'm doing some quick testing to try and figure out what's wrong [17:27:21] nah, looks like a caching issue [17:27:30] ok [17:27:34] Voidwalker ah so it works? [17:27:38] Hello :) [17:27:54] yup [17:27:58] hi [17:28:06] Hi [17:28:10] Voidwalker great! [18:30:35] Hi, on https://meta.miraheze.org/wiki/Special:ListFiles is some files with the error "Error creating thumbnail: File missing", is normal ? [18:30:36] Title: [ File list - Miraheze Meta ] - meta.miraheze.org [18:32:56] Videojeux4 hi, nope, we are currently fixing some issues from our rollout of swift! [18:35:56] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbhFB [18:35:58] [02miraheze/puppet] 07paladox 03b8a82b8 - Update mediawiki.conf [19:39:53] hmm https://union.miraheze.org/wiki/Especial:ListaIm%C3%A1genes [19:39:53] Title: [ Error de permisos - Unión ] - union.miraheze.org [19:43:23] paladox: ^ [20:05:06] Wiki-1776 hi, known, been looking into a fix as its not decoding url porperly [20:05:11] JohnLewis ^^ [20:05:52] but its not working and theres zero notice to people [20:07:10] JohnLewis some images work and some doin't but yeh, my main priority is to fix this [20:07:52] but a notice takes 30 seconds and greatly improves communication and public confidence [20:09:47] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbjvw [20:09:49] [02miraheze/mw-config] 07paladox 03cd60d1f - Update Sitenotice.php [20:09:53] JohnLewis done ^^ [20:11:30] I apologize if I misunderstood the ad as a "claim." I'm just warning about any bugs that show up. Thank you all for keeping an eye on us. :) [20:17:25] I’ll work on a tweet [20:18:19] https://mobile.twitter.com/miraheze/status/1014966645356457984 [20:18:20] Title: [ Twitter ] - mobile.twitter.com [20:19:58] thanks [20:32:34] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fbjJM [20:32:36] [02miraheze/puppet] 07paladox 033a99dc2 - swift: Many bug fixes [20:32:37] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [20:32:39] [02puppet] 07paladox opened pull request 03#764: swift: Many bug fixes - 13https://git.io/fbjJD [20:34:53] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fbjJp [20:34:54] [02miraheze/puppet] 07paladox 032277c49 - Update rewrite.py [20:34:56] [02puppet] 07paladox synchronize pull request 03#764: swift: Many bug fixes - 13https://git.io/fbjJD [20:54:09] [02miraheze/mediawiki] 07JohnFLewis pushed 031 commit to 03REL1_31 [+4/-0/±1] 13https://git.io/fbjTd [20:54:11] [02miraheze/mediawiki] 07JohnFLewis 0303ea6a4 - add 4 new extensions [21:12:03] [02miraheze/mw-config] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/fbjIG [21:12:04] [02miraheze/mw-config] 07JohnFLewis 035855635 - add new extensions [21:13:12] miraheze/mw-config/master/5855635 - John Lewis The build was broken. https://travis-ci.org/miraheze/mw-config/builds/400620872 [21:13:23] PROBLEM - Puppet on misc2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Apt_key[Add key: F86AA916A2195E121AEDB11437BBEE3F7AD95B3F from Apt::Source grafana_apt] [21:14:21] oh [21:14:23] JohnLewis ^^ [21:14:53] [02miraheze/mw-config] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbjIa [21:14:54] [02miraheze/mw-config] 07JohnFLewis 03e0e747f - fix syntax [21:15:06] ik ik [21:15:36] ok [21:15:50] miraheze/mw-config/master/e0e747f - John Lewis The build was fixed. https://travis-ci.org/miraheze/mw-config/builds/400621834 [21:16:11] PROBLEM - HTTP 4xx/5xx ERROR Rate on cp4 is WARNING: WARNING - NGINX Error Rate is 54% [21:17:23] PROBLEM - JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - check plugin (check_jobqueue) or PHP errors [21:18:13] RECOVERY - HTTP 4xx/5xx ERROR Rate on cp4 is OK: OK - NGINX Error Rate is 31% [21:19:35] [02miraheze/mw-config] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbjIA [21:19:37] [02miraheze/mw-config] 07JohnFLewis 031929a2c - fix Use in var name [21:21:23] RECOVERY - JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [21:33:32] PROBLEM - Current Load on test1 is CRITICAL: CRITICAL - load average: 2.25, 1.71, 0.99 [21:37:32] PROBLEM - Current Load on test1 is WARNING: WARNING - load average: 1.69, 1.81, 1.20 [21:37:59] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fbjt0 [21:38:00] [02miraheze/puppet] 07paladox 03240a1d7 - Update rewrite.py [21:38:02] [02puppet] 07paladox synchronize pull request 03#764: swift: Many bug fixes - 13https://git.io/fbjJD [21:39:26] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fbjto [21:39:28] [02miraheze/puppet] 07paladox 03961466c - Update mediawiki.conf [21:39:29] [02puppet] 07paladox synchronize pull request 03#764: swift: Many bug fixes - 13https://git.io/fbjJD [21:39:32] RECOVERY - Current Load on test1 is OK: OK - load average: 1.37, 1.64, 1.20 [21:39:53] JohnLewis im going to merge ^^ now :) [21:40:01] [02puppet] 07paladox closed pull request 03#764: swift: Many bug fixes - 13https://git.io/fbjJD [21:40:02] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/fbjtX [21:40:03] [02miraheze/puppet] 07paladox 03c81ede6 - swift: Many bug fixes (#764) * swift: Many bug fixes * Update rewrite.py * Update rewrite.py * Update mediawiki.conf [21:41:23] RECOVERY - Puppet on nfs1 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [21:41:31] mkay [21:41:41] [02puppet] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbiAS [21:41:43] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-1 [21:42:39] [02miraheze/mw-config] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/fbjtj [21:42:41] [02miraheze/mw-config] 07JohnFLewis 0300b252a - remove MyVariables [21:43:23] PROBLEM - Puppet on misc2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Apt_key[Add key: F86AA916A2195E121AEDB11437BBEE3F7AD95B3F from Apt::Source grafana_apt] [21:45:23] RECOVERY - Puppet on test1 is OK: OK: Puppet is currently enabled, last run 40 seconds ago with 0 failures [21:55:21] PROBLEM - JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 2614 [21:56:23] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbjmc [21:56:24] [02miraheze/puppet] 07paladox 03640a21f - redirect /dumps to /dumps/ [21:58:55] heh that jobqueue warning is super interesting [21:59:13] JohnLewis why? :) [21:59:59] the page it's updating hasn't been touched since 2016 [22:00:19] oh [22:00:38] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbjmF [22:00:39] [02miraheze/mw-config] 07paladox 03342926f - Update Sitenotice.php [22:00:52] JohnLewis things seem to be working now so i removed ^^ [22:00:53] :) [22:01:05] Wiki-1776 hi, we have resolved the issue now! [22:02:07] kk [22:08:06] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbjYO [22:08:08] [02miraheze/puppet] 07paladox 0344475ca - Create /srv/files to replace /mnt/mediawiki-static [22:14:32] Well, although I don't know why I don't see the favicon but maybe it's the cache, otherwise" I see it well and thank you. [22:32:16] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbj3Z [22:32:17] [02miraheze/puppet] 07paladox 03101ea59 - Update init.pp [22:34:56] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbj3g [22:34:58] [02miraheze/puppet] 07paladox 03e6961b9 - Switch Export aesbasewiki xml dump montly cron to swift uploads [22:35:10] JohnLewis ^^ [22:35:16] please review! [22:35:47] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbj3V [22:35:49] [02miraheze/puppet] 07paladox 03b959f68 - Install python-swiftclient on all mw* [22:36:14] looks okay [22:45:33] PROBLEM - Puppet on mw2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[python-swiftlient] [22:45:35] PROBLEM - Puppet on mw3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[python-swiftlient] [22:45:45] hmm [22:45:47] oh [22:45:59] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbjsW [22:46:00] [02miraheze/puppet] 07paladox 037455e85 - Update packages.pp [22:47:23] PROBLEM - Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[python-swiftlient] [22:47:33] PROBLEM - Puppet on test1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[python-swiftlient] [22:48:14] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbjsg [22:48:16] [02miraheze/puppet] 07paladox 038010502 - Update dumps.pp [22:49:23] RECOVERY - JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [22:53:33] RECOVERY - Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 41 seconds ago with 0 failures [22:55:21] RECOVERY - Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:55:23] RECOVERY - Puppet on test1 is OK: OK: Puppet is currently enabled, last run 39 seconds ago with 0 failures [23:03:22] PROBLEM - Puppet on misc2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Apt_key[Add key: F86AA916A2195E121AEDB11437BBEE3F7AD95B3F from Apt::Source grafana_apt] [23:03:32] PROBLEM - Puppet on bacula1 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [23:11:22] RECOVERY - Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 41 seconds ago with 0 failures [23:11:32] RECOVERY - Puppet on bacula1 is OK: OK: Puppet is currently enabled, last run 58 seconds ago with 0 failures [23:37:32] PROBLEM - Puppet on swift1 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 6 minutes ago with 0 failures [23:40:14] [02miraheze/rings] 07paladox pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/fbjno [23:40:15] [02miraheze/rings] 07paladox 03242ff08 - Set swift1 weight to 0 [23:45:36] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbjnH [23:45:37] [02miraheze/dns] 07paladox 03e05d360 - Add misc4 to dns [23:46:53] are you tabbing? [23:47:11] uh, github is [23:47:14] JohnLewis ^^ [23:48:08] [02miraheze/dns] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbjnj [23:48:10] [02miraheze/dns] 07JohnFLewis 03dd9bf42 - use tabs not spaces [23:50:18] [02miraheze/dns] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbjcT [23:50:19] [02miraheze/dns] 07JohnFLewis 03f01f072 - retab miraheze.org [23:50:29] everything you've changed has spaces not tabs [23:50:33] ok [23:52:59] PROBLEM - Current Load on misc4 is CRITICAL: connect to address 185.52.3.121 port 5666: Connection refused [23:53:11] PROBLEM - Disk Space on misc4 is CRITICAL: connect to address 185.52.3.121 port 5666: Connection refused [23:53:19] PROBLEM - Puppet on misc4 is UNKNOWN: NRPE: Unable to read output [23:53:32] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbjcn [23:53:34] [02miraheze/puppet] 07paladox 03ba0a0d7 - Update prometheus.yml [23:54:59] RECOVERY - Current Load on misc4 is OK: OK - load average: 0.47, 0.38, 0.16 [23:55:09] RECOVERY - Disk Space on misc4 is OK: DISK OK - free space: / 60482 MB (98% inode=99%): [23:55:40] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/fbjc8 [23:55:41] [02miraheze/puppet] 07paladox 030c0d030 - Setup swift proxy on misc4 [23:56:12] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbjcB [23:56:14] [02miraheze/puppet] 07paladox 03d8b67ff - Update site.pp [23:57:25] [02miraheze/puppet] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fbjcR [23:57:27] [02miraheze/puppet] 07JohnFLewis 033f8ab81 - add phab and bacula client to misc4' [23:59:19] PROBLEM - Puppet on misc4 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 seconds ago with 1 failures. Failed resources (up to 3 shown): Service[nginx]