[00:02:44] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:03:06] PROBLEM - cp2 Puppet on cp2 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:03:36] PROBLEM - lizardfs2 Puppet on lizardfs2 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:03:46] PROBLEM - cp4 Puppet on cp4 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:03:56] PROBLEM - misc2 Puppet on misc2 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:04:00] PROBLEM - lizardfs1 Puppet on lizardfs1 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:04:12] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:04:24] PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:04:26] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:06:28] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 57 seconds ago with 0 failures [00:09:40] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp9KY [00:09:41] [02miraheze/puppet] 07paladox 035a03159 - Update icinga2.pp [00:10:07] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp9KO [00:10:09] [02miraheze/puppet] 07paladox 03c78faeb - Update grafana.pp [00:10:44] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 2 seconds ago with 0 failures [00:11:36] RECOVERY - lizardfs2 Puppet on lizardfs2 is OK: OK: Puppet is currently enabled, last run 45 seconds ago with 0 failures [00:11:46] RECOVERY - cp4 Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 44 seconds ago with 0 failures [00:11:56] RECOVERY - misc2 Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 59 seconds ago with 0 failures [00:11:58] RECOVERY - lizardfs1 Puppet on lizardfs1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:12:14] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:12:19] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp9Kc [00:12:20] [02miraheze/puppet] 07paladox 03357bf9d - Update grafana.pp [00:12:24] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:12:32] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp9KC [00:12:33] [02miraheze/puppet] 07paladox 039661d61 - Update icinga2.pp [00:12:45] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-1/±0] 13https://git.io/fp9K8 [00:12:47] [02miraheze/puppet] 07paladox 0363e1f13 - Drop the old icinga role for icinga 1.x [00:13:04] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:16:29] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:17:21] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp9Ku [00:17:22] [02miraheze/puppet] 07paladox 035058597 - Update roundcubemail.pp [00:17:33] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp9Kz [00:17:34] [02miraheze/puppet] 07paladox 03ecc2d6f - Update icinga2.pp [00:17:45] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp9Ka [00:17:47] [02miraheze/puppet] 07paladox 03e0f61c0 - Update grafana.pp [00:20:29] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:21:18] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp9Ki [00:21:19] [02miraheze/puppet] 07paladox 035c80243 - Update cert.pp [00:21:35] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp9KP [00:21:37] [02miraheze/puppet] 07paladox 037845cae - Update postgis.pp [00:21:51] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp9KX [00:21:52] [02miraheze/puppet] 07paladox 038ce8327 - Update dirs.pp [00:22:09] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp9K1 [00:22:10] [02miraheze/puppet] 07paladox 030a2a6e3 - Update quickdatacopy.pp [00:22:25] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:22:27] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp9KM [00:22:29] [02miraheze/puppet] 07paladox 038274721 - Update extension.pp [00:22:44] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp9KD [00:22:45] [02miraheze/puppet] 07paladox 03465130f - Update parameters.pp [00:23:02] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp9Ky [00:23:04] [02miraheze/puppet] 07paladox 031c6485a - Update db.pp [00:23:28] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp9K9 [00:23:28] [02miraheze/puppet] 07paladox 03a3891f3 - Update master.pp [00:24:52] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp9K7 [00:24:53] [02miraheze/puppet] 07paladox 0374a6d28 - Update user.pp [00:26:31] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp9KN [00:26:32] [02miraheze/puppet] 07paladox 031c0ef8b - Update server.pp [00:26:55] RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 34 seconds ago with 0 failures [00:27:17] RECOVERY - bacula1 Puppet on bacula1 is OK: OK: Puppet is currently enabled, last run 55 seconds ago with 0 failures [00:27:19] RECOVERY - misc3 Puppet on misc3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:28:42] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp96k [00:28:43] [02miraheze/puppet] 07paladox 03b0b2863 - Expect "webmail.m.org" to return 401 [00:29:21] RECOVERY - cp5 Puppet on cp5 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:29:35] RECOVERY - db4 Puppet on db4 is OK: OK: Puppet is currently enabled, last run 14 seconds ago with 0 failures [00:31:02] RECOVERY - misc1 webmail.miraheze.org HTTPS on misc1 is OK: HTTP OK: Status line output matched "HTTP/1.1 401 Unauthorized" - 5607 bytes in 0.027 second response time [01:49:00] PROBLEM - db4 Current Load on db4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [01:51:06] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 107.191.126.23/cpweb [01:53:05] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [01:56:45] PROBLEM - db4 Current Load on db4 is WARNING: WARNING - load average: 6.31, 7.84, 5.94 [02:00:43] RECOVERY - db4 Current Load on db4 is OK: OK - load average: 2.22, 5.42, 5.44 [05:52:05] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 172.104.111.8/cpweb [05:52:35] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 172.104.111.8/cpweb [05:56:35] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [05:58:03] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [06:26:05] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 172.104.111.8/cpweb [06:28:03] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [07:17:15] [02miraheze/WikiDiscover] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fp97V [07:17:17] [02miraheze/WikiDiscover] 07translatewiki 03ce2b23b - Localisation updates from https://translatewiki.net. [07:17:18] [02miraheze/MirahezeMagic] 07translatewiki pushed 031 commit to 03master [+1/-0/±3] 13https://git.io/fp97w [07:17:20] [02miraheze/MirahezeMagic] 07translatewiki 0326d9038 - Localisation updates from https://translatewiki.net. [07:17:21] [02miraheze/ManageWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±14] 13https://git.io/fp97r [07:17:23] [02miraheze/ManageWiki] 07translatewiki 03cf7f771 - Localisation updates from https://translatewiki.net. [07:17:24] [02miraheze/CreateWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/fp97o [07:17:26] [02miraheze/CreateWiki] 07translatewiki 03f05b2a9 - Localisation updates from https://translatewiki.net. [07:47:54] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 3202 [08:57:52] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [10:17:55] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 6524 [11:15:15] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fpHkd [11:15:17] [02miraheze/services] 07MirahezeSSLBot 03817b65f - BOT: Updating services config for wikis [12:15:12] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fpHYy [12:15:14] [02miraheze/services] 07MirahezeSSLBot 0322e88dd - BOT: Updating services config for wikis [13:49:53] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [15:57:12] Hi! Just contacting you to tell that here are no high priority open tasks. Nice job! [17:03:30] paladox: I'm having issues with the IABot Management Interface. When my dev environment is tunneling into WMF Labs to establish a connection to your DB, it works as expected. [17:03:40] yup [17:03:54] But when the Web Service is trying to access the DB, it times out and falls back to the API [17:04:16] ah, what ip is the web service using? [17:04:32] No idea. Wouldn't you know being a toolforge admin? [17:05:07] I doin't use toolforge so not sure [17:05:19] what's the web address? [17:07:17] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fpH1I [17:07:19] [02miraheze/puppet] 07paladox 03adfe3ae - Whitelist another of cyber power ip in db4 [17:30:21] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [17:30:23] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fpHys [17:30:24] [02miraheze/puppet] 07paladox 03822df56 - Update ufw Updates to https://github.com/attachmentgenie/attachmentgenie-ufw/ [17:30:25] Title: [ GitHub - attachmentgenie/attachmentgenie-ufw ] - github.com [17:30:26] [02puppet] 07paladox opened pull request 03#929: Update ufw - 13https://git.io/fpHyZ [17:31:44] [02puppet] 07paladox synchronize pull request 03#929: Update ufw - 13https://git.io/fpHyZ [17:31:46] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fpHyl [17:31:47] [02miraheze/puppet] 07paladox 03a6fae21 - Update allow.pp [18:04:39] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fpHHE [18:04:40] [02miraheze/puppet] 07paladox 03a5a65ed - Use labster miraheze email for some miraheze emails [18:09:08] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/fpHHb [18:09:09] [02miraheze/puppet] 07paladox 03ef85de5 - Remove internal email lists (bacula-mail, apt-list) [18:09:12] [02puppet] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vbiAS [18:09:12] [02puppet] 07paladox opened pull request 03#930: Remove internal email lists (bacula-mail, apt-list) - 13https://git.io/fpHHA [18:09:49] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/fpHQe [18:09:51] [02miraheze/puppet] 07paladox 036582da9 - Update 50unattended-upgrades [18:09:53] [02puppet] 07paladox synchronize pull request 03#930: Remove internal email lists (bacula-mail, apt-list) - 13https://git.io/fpHHA [18:10:06] [02puppet] 07paladox closed pull request 03#930: Remove internal email lists (bacula-mail, apt-list) - 13https://git.io/fpHHA [18:10:08] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/fpHQJ [18:10:09] [02miraheze/puppet] 07paladox 032053e14 - Remove internal email lists (bacula-mail, apt-list) (#930) * Remove internal email lists (bacula-mail, apt-list) * Update 50unattended-upgrades [18:10:11] [02puppet] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vbiAS [18:10:12] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-2 [18:11:49] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fpHQZ [18:11:50] [02miraheze/puppet] 07paladox 033a26e77 - Remove internal email aliases [20:36:19] getting http 503 error [20:36:22] :( [20:36:40] nevermind [21:10:14] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fpQUz [21:10:16] [02miraheze/services] 07MirahezeSSLBot 039d1f40f - BOT: Updating services config for wikis [21:10:39] PROBLEM - cp5 Varnish Backends on cp5 is CRITICAL: 1 backends are down. mw3 [21:12:39] RECOVERY - cp5 Varnish Backends on cp5 is OK: All 5 backends are healthy [21:28:22] [02puppet] 07The-Voidwalker opened pull request 03#931: spelling - 13https://git.io/fpQkG [21:50:05] [02puppet] 07paladox closed pull request 03#931: spelling - 13https://git.io/fpQkG [21:50:06] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fpQLT [21:50:08] [02miraheze/puppet] 07The-Voidwalker 03a9e35d6 - spelling (#931) [21:50:10] [02puppet] 07paladox commented on pull request 03#931: spelling - 13https://git.io/fpQLL [23:51:00] Hello Posaf! If you have any questions feel free to ask and someone should answer soon. [23:51:58] How do you change the wiki's logo, i cant seem to find out how to [23:52:47] Posaf hi, through Special:ManageWikiSettings [23:53:24] Yes, how do I make the logo so it uploads [23:54:51] have you uploaded the logo to the wiki as a file? [23:55:17] Yes [23:56:33] if you right click on the image, and open it in a new tab, you should have a url beginning with https://static.miraheze.org [23:56:34] Title: [ 403 Forbidden ] - static.miraheze.org [23:57:17] this is what you need to enter into the logo field in Special:ManageWikiSettings in order to display the logo [23:58:09] Ahh, I see thanks for the help! [23:58:35] no problem, let me know if you have any more questions [23:59:13] Okay, thanks