[00:04:30] Hello mihipte! If you have any questions feel free to ask and someone should answer soon. [01:08:44] PROBLEM - bacula1 Bacula Static on bacula1 is WARNING: WARNING: Full, 4442156 files, 398.9GB, 2019-06-16 01:05:00 (2.1 weeks ago) [01:35:29] PROBLEM - wiki.kybermc.tk - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:20:43] PROBLEM - bacula1 Bacula Phabricator Static on bacula1 is WARNING: WARNING: Full, 79592 files, 1.998GB, 2019-06-16 02:18:00 (2.1 weeks ago) [04:33:32] PROBLEM - misc2 Current Load on misc2 is CRITICAL: CRITICAL - load average: 4.10, 3.41, 2.94 [04:35:28] PROBLEM - misc2 Current Load on misc2 is WARNING: WARNING - load average: 3.86, 3.47, 3.00 [04:37:27] RECOVERY - misc2 Current Load on misc2 is OK: OK - load average: 2.68, 3.19, 2.96 [04:41:27] PROBLEM - misc2 Current Load on misc2 is CRITICAL: CRITICAL - load average: 4.06, 3.63, 3.19 [04:59:27] PROBLEM - misc2 Current Load on misc2 is WARNING: WARNING - load average: 1.89, 3.45, 3.70 [05:03:27] RECOVERY - misc2 Current Load on misc2 is OK: OK - load average: 1.55, 2.51, 3.26 [07:05:49] :q [07:06:02] ahhh, sorry, ignore me [07:40:15] [02miraheze/MirahezeMagic] 07translatewiki pushed 031 commit to 03master [+0/-0/±5] 13https://git.io/fjKmC [07:40:16] [02miraheze/MirahezeMagic] 07translatewiki 0370fa6d7 - Localisation updates from https://translatewiki.net. [07:40:18] [02miraheze/ManageWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjKmW [07:40:19] [02miraheze/ManageWiki] 07translatewiki 03995df29 - Localisation updates from https://translatewiki.net. [07:40:21] [02miraheze/WikiDiscover] 07translatewiki pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/fjKml [07:40:22] [02miraheze/WikiDiscover] 07translatewiki 03c5a90ff - Localisation updates from https://translatewiki.net. [07:43:37] miraheze/WikiDiscover/master/c5a90ff - translatewiki.net The build is still failing. https://travis-ci.org/miraheze/WikiDiscover/builds/552639536 [11:28:28] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [11:30:56] PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [11:31:14] PROBLEM - cp3 Disk Space on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [11:31:34] PROBLEM - cp3 SSH on cp3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:31:48] PROBLEM - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [11:31:57] PROBLEM - cp3 HTTPS on cp3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:32:03] PROBLEM - cp3 Current Load on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [11:32:17] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 128.199.139.216/cpweb [11:32:51] PROBLEM - Host cp3 is DOWN: PING CRITICAL - Packet loss = 100% [11:34:17] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [11:34:54] RECOVERY - Host cp3 is UP: PING OK - Packet loss = 0%, RTA = 180.49 ms [11:34:57] RECOVERY - cp3 Varnish Backends on cp3 is OK: All 5 backends are healthy [11:35:19] RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 4890 MB (20% inode=94%); [13:06:28] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [13:08:56] PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [13:09:14] PROBLEM - cp3 Disk Space on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [13:09:34] PROBLEM - cp3 SSH on cp3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:09:47] cp3 is up [13:09:48] PROBLEM - cp3 HTTP 4xx/5xx ERROR Rate on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [13:09:57] PROBLEM - cp3 HTTPS on cp3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:10:03] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb [13:10:17] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb [13:10:30] PROBLEM - cp3 Current Load on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [13:10:43] PROBLEM - Host cp3 is DOWN: PING CRITICAL - Packet loss = 100% [13:12:03] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [13:16:17] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [13:17:44] RECOVERY - Host cp3 is UP: PING OK - Packet loss = 0%, RTA = 186.32 ms [14:28:13] @RhinosF1 I dont maintain the log bot [14:28:54] @Zppix: do you not? anyway it works now - must be that monthly usage cap thing [14:29:14] @RhinosF1 i maintain ZppixBot and @Miraheze-IRC / MH-Discord [14:29:38] @Zppix yeah, i meant @Miraheze-IRC [14:29:44] Oh [14:30:46] @RhinosF1 more than likely just the quota since it works now and its july 1st [14:31:04] @Zppix probably, how r u? [14:31:16] Im breathing so fine ig [14:31:36] @RhinosF1 [15:45:12] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjKlc [15:45:14] [02miraheze/services] 07MirahezeSSLBot 036a3e418 - BOT: Updating services config for wikis [17:07:13] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 409 [17:11:09] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [18:43:57] [02ManageWiki] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vpSns [18:43:59] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03paladox-patch-1 [+1/-0/±0] 13https://git.io/fjKBD [18:44:00] [02miraheze/ManageWiki] 07paladox 033d6f066 - Change ns_core to default to 0 [18:44:02] [02ManageWiki] 07paladox opened pull request 03#110: Change ns_core to default to 0 - 13https://git.io/fjKBy [18:44:39] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjKBS [18:44:41] [02miraheze/ManageWiki] 07paladox 03d6f6d4b - Update mw_namespaces.sql [18:44:42] [02ManageWiki] 07paladox synchronize pull request 03#110: Change ns_core to default to 0 - 13https://git.io/fjKBy [18:45:24] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjKB9 [18:45:26] [02miraheze/ManageWiki] 07paladox 03b460068 - Update ManageWikiHooks.php [18:45:27] [02ManageWiki] 07paladox synchronize pull request 03#110: Change ns_core to default to 0 - 13https://git.io/fjKBy [18:46:57] [02ManageWiki] 07paladox synchronize pull request 03#110: Change ns_core to default to 0 - 13https://git.io/fjKBy [18:46:59] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjKBQ [18:47:00] [02miraheze/ManageWiki] 07paladox 036fa5c36 - Update patch-namespace-core-alter.sql [18:50:07] miraheze/ManageWiki/paladox-patch-1/b460068 - paladox The build passed. https://travis-ci.org/miraheze/ManageWiki/builds/552919035 [18:51:07] miraheze/ManageWiki/paladox-patch-1/6fa5c36 - paladox The build passed. https://travis-ci.org/miraheze/ManageWiki/builds/552919594 [18:51:33] [02ManageWiki] 07paladox synchronize pull request 03#110: Change ns_core to default to 0 - 13https://git.io/fjKBy [18:51:34] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjKBF [18:51:36] [02miraheze/ManageWiki] 07paladox 0394eba13 - Update patch-namespace-core-alter.sql [18:54:07] [02ManageWiki] 07paladox synchronize pull request 03#110: Change ns_core to default to 0 - 13https://git.io/fjKBy [18:54:08] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fjKBp [18:54:10] [02miraheze/ManageWiki] 07paladox 032016bad - Update patch-namespace-core-alter.sql [18:55:29] [02ManageWiki] 07paladox closed pull request 03#110: Change ns_core to default to 0 - 13https://git.io/fjKBy [18:55:31] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03master [+1/-0/±2] 13https://git.io/fjKRv [18:55:32] [02miraheze/ManageWiki] 07paladox 033d23fd6 - Change ns_core to default to 0 (#110) * Change ns_core to default to 0 * Update mw_namespaces.sql * Update ManageWikiHooks.php * Update patch-namespace-core-alter.sql * Update patch-namespace-core-alter.sql * Update patch-namespace-core-alter.sql [18:56:54] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+0/-0/±1] 13https://git.io/fjKRT [18:56:56] [02miraheze/mediawiki] 07paladox 034b101f4 - Update MW [18:57:21] !log running ALTER TABLE mw_namespaces MODIFY COLUMN ns_core INT(1) NOT NULL DEFAULT 0; [18:57:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [18:58:27] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_33 [+0/-0/±1] 13https://git.io/fjKRq [18:58:28] [02miraheze/mediawiki] 07paladox 03ce346df - Update MW [21:00:51] [02mw-config] 07Hispano76 opened pull request 03#2691: Update projects, retire old projects - 13https://git.io/fjK0S [21:45:12] ping paladox [21:45:17] hi [21:45:43] [02mw-config] 07paladox closed pull request 03#2691: Update projects, retire old projects - 13https://git.io/fjK0S [21:45:45] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjKEa [21:45:46] [02miraheze/mw-config] 07Hispano76 035b21363 - Update projects, retire old projects (#2691) [21:46:27] you can erase these wikis: Apunteswiki, Centralwiki, Destinoswiki, Infowiki, Tallerwiki. This is part of a restructuring of my projects. [21:46:32] paladox, [21:47:18] Hispano76 you need to request that on i think CN now. [21:48:28] ooh [21:48:31] ok [21:49:33] paladox, in Community noticeboard? [21:49:41] Hispano76 yup [21:50:03] hum, no in SN? [21:50:17] oh [21:50:19] yes SN [21:52:43] done paladox [21:53:04] thanks [21:54:30] to you [21:58:05] https://meta.miraheze.org/wiki/Special:Contributions/Fungster - what is with this guys 'proposals'?? [21:58:07] [ User contributions for Fungster - Miraheze Meta ] - meta.miraheze.org [22:17:12] Oh paladox: https://meta.miraheze.org/wiki/Special:RequestWikiQueue [22:17:13] [ Wiki requests queue - Miraheze Meta ] - meta.miraheze.org [22:17:14] pls [22:18:02] done [22:46:49] PROBLEM - cp2 Puppet on cp2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [22:48:46] PROBLEM - cp2 Puppet on cp2 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 3 days ago with 0 failures [22:49:11] what u done? [22:49:25] PROBLEM - cp2 Varnish Backends on cp2 is WARNING: No backends detected. If this is an error, see readme.txt [22:49:32] I've done nothing. [22:49:42] but forgot to re enable puppet, which i've done now [22:50:30] paladox: ah, so the error was you just not turning it on. Should have just @Zppix wanted to charge a phone so unplugged it [22:50:43] ok [22:50:43] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:51:24] PROBLEM - cp2 Varnish Backends on cp2 is CRITICAL: 1 backends are down. mw1 [22:52:24] paladox: did u turn it all back on? mw1 backend still down [22:52:39] Yup, i did. [22:52:58] what's causing this issue then? [22:53:24] RECOVERY - cp2 Varnish Backends on cp2 is OK: All 5 backends are healthy [22:53:24] im not sure. [22:53:31] though seems recovered now [22:53:49] fixed anyway. @zppix did you unplug your phone? [22:54:02] go stick it in the WMF servers next time [23:29:26] RhinosF1: I don't like the WMF sites going down, though!