[00:00:09] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvcRQ [00:00:10] [02miraheze/services] 07MirahezeSSLBot 03890cab1 - BOT: Updating services config for wikis [00:12:45] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 128.199.139.216/cpweb, 2a00:d880:5:8ea::ebc7/cpweb [00:14:40] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [00:15:46] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-8 [+0/-0/±1] 13https://git.io/Jvc0L [00:15:47] [02miraheze/puppet] 07paladox 03999ffa5 - cloud[12]: Enable root ssh [00:15:49] [02puppet] 07paladox created branch 03paladox-patch-8 - 13https://git.io/vbiAS [00:15:50] [02puppet] 07paladox opened pull request 03#1194: cloud[12]: Enable root ssh - 13https://git.io/Jvc0t [00:16:08] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-8 [+0/-0/±1] 13https://git.io/Jvc0q [00:16:09] [02miraheze/puppet] 07paladox 0313534f2 - Update cloud2.yaml [00:16:11] [02puppet] 07paladox synchronize pull request 03#1194: cloud[12]: Enable root ssh - 13https://git.io/Jvc0t [00:16:44] [02puppet] 07paladox closed pull request 03#1194: cloud[12]: Enable root ssh - 13https://git.io/Jvc0t [00:16:45] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/Jvc0m [00:16:46] [02miraheze/puppet] 07paladox 03340cfb9 - cloud[12]: Enable root ssh (#1194) * cloud[12]: Enable root ssh * Update cloud2.yaml [00:16:48] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-8 [00:16:49] [02puppet] 07paladox deleted branch 03paladox-patch-8 - 13https://git.io/vbiAS [01:17:12] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 2a00:d880:5:8ea::ebc7/cpweb [01:19:10] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [01:22:03] PROBLEM - cp3 Stunnel Http for mw2 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [01:24:04] RECOVERY - cp3 Stunnel Http for mw2 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15302 bytes in 4.360 second response time [02:01:01] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/Jvcul [02:01:02] [02miraheze/puppet] 07paladox 03c5b8fb7 - gluster1: Set puppetmaster to puppet2 [02:02:30] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 128.199.139.216/cpweb [02:04:25] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [02:13:41] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvcuz [02:13:42] [02miraheze/puppet] 07paladox 034bba6ab - Setup gluster on gluster1 [02:18:14] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 2400:6180:0:d0::403:f001/cpweb [02:19:16] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 107.191.126.23/cpweb [02:19:24] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvcuV [02:19:26] [02miraheze/puppet] 07paladox 0369f8325 - Update firewall.yaml [02:21:45] !log probe gluster1 from lizardfs(gluster) but do not add it as a brick [02:22:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [02:23:18] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [02:24:21] PROBLEM - cp2 Stunnel Http for mw2 on cp2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [02:26:22] RECOVERY - cp2 Stunnel Http for mw2 on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 15302 bytes in 4.251 second response time [02:28:06] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [02:31:27] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvcuD [02:31:28] [02miraheze/dns] 07paladox 033f3b6b0 - Add test2 to dns [02:53:33] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 2400:6180:0:d0::403:f001/cpweb [02:54:43] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/Jvcup [02:54:45] [02miraheze/puppet] 07paladox 0361a90d3 - test2: Set puppetmaster to puppet2 [02:55:00] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/Jvcze [02:55:02] [02miraheze/puppet] 07paladox 034db01f7 - mw4: Set puppetmaster to puppet2 [03:01:36] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 3 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb [03:03:36] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [03:05:43] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [05:24:47] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 3 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb [05:28:00] PROBLEM - cp2 Stunnel Http for mw3 on cp2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [05:28:41] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [05:31:49] PROBLEM - cp3 Stunnel Http for mw3 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [05:32:12] PROBLEM - cp4 Stunnel Http for mw3 on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [05:32:59] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: connect to address 192.184.82.120 port 5666: Connection refusedconnect to host 192.184.82.120 port 5666: Connection refused [05:33:55] RECOVERY - cp3 Stunnel Http for mw3 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15302 bytes in 4.910 second response time [05:34:15] RECOVERY - cp4 Stunnel Http for mw3 on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 15302 bytes in 3.856 second response time [05:34:21] RECOVERY - cp2 Stunnel Http for mw3 on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 15302 bytes in 4.412 second response time [05:34:54] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [06:26:06] RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 2976 MB (12% inode=94%); [10:59:54] PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2646 MB (10% inode=94%); [13:35:27] PROBLEM - db5 Current Load on db5 is CRITICAL: CRITICAL - load average: 7.67, 4.57, 2.56 [13:37:25] PROBLEM - db5 Current Load on db5 is WARNING: WARNING - load average: 3.35, 3.97, 2.58 [13:41:24] RECOVERY - db5 Current Load on db5 is OK: OK - load average: 1.19, 2.68, 2.38 [13:56:27] [02miraheze/ManageWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvcyO [13:56:28] [02miraheze/ManageWiki] 07translatewiki 0396cb7c4 - Localisation updates from https://translatewiki.net. [13:56:29] [ Main page - translatewiki.net ] - translatewiki.net. [14:16:06] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvcyH [14:16:08] [02miraheze/dns] 07paladox 035c29193 - Add rdb1 to dns [14:48:52] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/JvcSN [14:48:53] [02miraheze/puppet] 07paladox 0300f6b1b - rdb1: Set puppetmaster to puppet2 [14:50:09] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvcSh [14:50:10] [02miraheze/services] 07MirahezeSSLBot 037728817 - BOT: Updating services config for wikis [14:57:17] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvc9W [14:57:18] [02miraheze/puppet] 07paladox 030d5f107 - Setup redis on rdb1 [15:00:22] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvc9u [15:00:23] [02miraheze/puppet] 07paladox 03974b94f - redis: Fix support for buster [15:06:28] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvc9D [15:06:30] [02miraheze/puppet] 07paladox 03d143060 - Add MediaWiki to test2 [15:07:45] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvc9H [15:07:47] [02miraheze/puppet] 07paladox 035a8f026 - Update test2.yaml [15:10:52] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [15:12:54] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [16:33:32] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvc53 [16:33:34] [02miraheze/puppet] 07paladox 039ffda38 - db: Add test2 to firewall [17:28:57] RhinosF1: hi. what was I pinged for? [17:29:28] Reception123: ensuring social profile and wikibase we’re fully updated [17:29:48] + regetting wikibase tested per email to tech@ from wmde [17:30:59] ok [17:32:39] RhinosF1: https://github.com/miraheze/mediawiki/commit/fc9ef4c6d9fa82896a9906955f4e2af00561f744 paladox did Wikibase already [17:32:40] [ Update Wikibase · miraheze/mediawiki@fc9ef4c · GitHub ] - github.com [17:35:22] Reception123: I’m half asleep, just don’t break social profile, fix it. Watch out for sql stuff [17:35:31] See my channel yesterday [17:35:49] https://phabricator.wikimedia.org/T244656 [17:35:49] [ ⚓ T244656 Trying to play QuizGame on a Miraheze Wiki, you can't get past the "Loading" display ] - phabricator.wikimedia.org [17:36:04] RhinosF1: yeah, it seems there are some major changes... [17:36:16] so I'm not sure I want to push the update, since it seems to require some SQL changes [17:36:20] Reception123: it’s a tad outdated [17:36:34] Let’s get it ready to push and then we can review [17:36:49] paladox|UKInEU and John are stuck installing new servers though [17:37:45] yeah, so we should probably wait for that [17:37:51] unless Zppix wants to help me break stuff :P [17:38:13] Reception123: get it ready to show progress [17:38:23] And then we can check wiki base [17:38:30] Although that might be messy [17:38:40] it definitly will. [17:38:45] it will require core changes [17:38:46] At least Lydia might be able to help with that see your email [17:39:34] They have telegram I think for support [17:53:00] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvcbT [17:53:02] [02miraheze/puppet] 07paladox 030b8eb99 - Add test2 to gluster firewall [17:57:30] RECOVERY - lizardfs6 Puppet on lizardfs6 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:03:33] paladox|UKInEU, this seems to be incorrect: https://github.com/miraheze/mw-config/blob/e59f7fed7e130309d4476793483e9293b54902bb/LocalSettings.php#L2056 [18:03:34] [ mw-config/LocalSettings.php at e59f7fed7e130309d4476793483e9293b54902bb · miraheze/mw-config · GitHub ] - github.com [18:03:49] Voidwalker oh? [18:04:29] Voidwalker what should it be? :) [18:04:58] if I'm understanding https://phabricator.miraheze.org/T5206#99192 correctly, it's saving the setting as ['nsid'=>'nstalkid'], when it should be ['nsid','nstalkid'] [18:04:59] [ ⚓ T5206 Visual Editor fails to load ] - phabricator.miraheze.org [18:05:28] basically, the values are being saved as a keyed array when they should be a regular array [18:07:06] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/Jvcbi [18:07:07] [02miraheze/mw-config] 07paladox 038c7f723 - wgManageWikiNamespacesAdditional: Fix override value [18:07:09] [02mw-config] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbvb3 [18:07:10] ohhV^ [18:07:10] [02mw-config] 07paladox opened pull request 03#2874: wgManageWikiNamespacesAdditional: Fix override value - 13https://git.io/JvcbP [18:07:17] *Void ^ [18:07:25] * Voidwalker ^ [18:08:42] no, that should be fine, the issue is that any custom values (anything that's not the empty default) are not stored properly [18:08:59] see the task I linked for what I'm talking about [18:09:17] oh [18:09:23] Voidwalker so this is a bug in MW? [18:09:32] I keep getting ssl protocol errors on git.io [18:09:33] most likely [18:09:41] [02puppet] 07IsabellaFan2-stack opened pull request 03#1195: Please add cp5.miraheze.org - 13https://git.io/JvcbQ [18:09:59] Whose that [18:10:45] host 116.93.119.26 [18:10:45] 26.119.93.116.in-addr.arpa domain name pointer s01.godhand8.com. [18:11:11] [02puppet] 07paladox closed pull request 03#1195: Please add cp5.miraheze.org - 13https://git.io/JvcbQ [18:11:35] paladox|UKInEU: can you block them? It’s ATS evading a github ban [18:11:40] Reception123: ^ [18:11:58] And lock the Pr [18:12:04] [02puppet] 07IsabellaFan2-stack commented on pull request 03#1195: Please add cp5.miraheze.org - 13https://git.io/Jvcbh [18:13:01] done [18:13:11] Voidwalker: can you cu https://meta.miraheze.org/wiki/Special:Contributions/IsabellaFan2 and delete thier wiki [18:13:14] [ User contributions for IsabellaFan2 - Miraheze Meta ] - meta.miraheze.org [18:13:33] paladox|UKInEU: have you banned then in github? [18:13:40] yes [18:14:38] will investigate shortly [18:16:58] Good [18:17:09] Voidwalker: i’ve blocked meta + commons [18:17:15] And deleted lots of spam [18:17:59] The pioneer accepted their wiki [18:18:04] Voidwalker has to be https://github.com/miraheze/ManageWiki/blob/532701a6eac70b41dc7bc30bf8d667e5c049ef0f/includes/formFactory/ManageWikiFormFactoryBuilder.php#L420 ? [18:18:04] [ ManageWiki/ManageWikiFormFactoryBuilder.php at 532701a6eac70b41dc7bc30bf8d667e5c049ef0f · miraheze/ManageWiki · GitHub ] - github.com [18:18:46] no I think it's the submit function [18:19:39] oh [18:20:05] Voidwalker which line? [18:22:58] I'm not sure [18:25:02] PROBLEM - portalsofphereon.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.portalsofphereon.com' expires in 15 day(s) (Wed 26 Feb 2020 06:22:14 PM GMT +0000). [18:25:02] PROBLEM - www.portalsofphereon.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.portalsofphereon.com' expires in 15 day(s) (Wed 26 Feb 2020 06:22:14 PM GMT +0000). [18:25:21] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvcNK [18:25:23] [02miraheze/ssl] 07MirahezeSSLBot 03ca3c8b8 - Bot: Update SSL cert for www.portalsofphereon.com [18:26:21] paladox|UKInEU: i’ve reported them for abuse [18:27:33] Ticket ID: 556655 [18:29:40] ok, didn't really need to know :) [18:30:10] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvcNd [18:30:11] [02miraheze/services] 07MirahezeSSLBot 03ef0ce8b - BOT: Updating services config for wikis [18:31:25] It’s an FYI since my main issue is ban evading our repos [18:32:59] RECOVERY - www.portalsofphereon.com - LetsEncrypt on sslhost is OK: OK - Certificate 'www.portalsofphereon.com' will expire on Sun 10 May 2020 05:25:15 PM GMT +0000. [18:33:02] RECOVERY - portalsofphereon.com - LetsEncrypt on sslhost is OK: OK - Certificate 'www.portalsofphereon.com' will expire on Sun 10 May 2020 05:25:15 PM GMT +0000. [18:35:07] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvcAq [18:35:09] [02miraheze/services] 07MirahezeSSLBot 03fa34fd9 - BOT: Updating services config for wikis [18:36:01] Voidwalker yup, i see in the db "{"wgExtraSignatureNamespaces":true,"wgNamespacesToPostIn":true,"wgVisualEditorAvailableNamespaces":true}" [18:37:26] Voidwalker: check pm pls [18:40:09] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvcAl [18:40:11] [02miraheze/services] 07MirahezeSSLBot 03ed68a78 - BOT: Updating services config for wikis [18:45:08] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvcAV [18:45:09] [02miraheze/services] 07MirahezeSSLBot 03a21803b - BOT: Updating services config for wikis [18:50:09] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvcAD [18:50:11] [02miraheze/services] 07MirahezeSSLBot 03763715e - BOT: Updating services config for wikis [18:55:07] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvcAp [18:55:09] [02miraheze/services] 07MirahezeSSLBot 0329aa04c - BOT: Updating services config for wikis [19:00:10] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvcxt [19:00:11] [02miraheze/services] 07MirahezeSSLBot 03bdefc2b - BOT: Updating services config for wikis [19:05:08] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvcxz [19:05:10] [02miraheze/services] 07MirahezeSSLBot 039eec503 - BOT: Updating services config for wikis [19:10:08] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvcxx [19:10:10] [02miraheze/services] 07MirahezeSSLBot 03e00e0fb - BOT: Updating services config for wikis [19:11:58] because the Dark Mode is global? curiosity :) [19:12:03] * hispano76 greetings [19:12:11] hispano76: ? [19:12:45] paladox|UKInEU: can you calm services down? It’s a wiki just deletes [19:12:56] s/tes/ted [19:12:57] RhinosF1 meant to say: paladox|UKInEU: can you calm services down? It’s a wiki just deleted [19:13:44] By default, the "button" appears on all wikis or at least on most of them (I don't remember enabling it on my wikis) [19:13:53] * [19:13:57] hispano76: yeah its global [19:14:11] paladox|UKInEU: whose deleted it? No logs on meta??? [19:14:48] Wasen't me. [19:15:08] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvcpm [19:15:09] paladox|UKInEU: it shows as 404 [19:15:09] [02miraheze/services] 07MirahezeSSLBot 03f4d227c - BOT: Updating services config for wikis [19:15:20] It should be deleted fwiw [19:15:29] But services is in a mood now [19:15:34] It would be if someone pressed delete in the ui... [19:16:06] paladox|UKInEU: ok, I mean can we calm services down or drop it [19:16:18] No, we carn't. [19:17:02] paladox|UKInEU: do either? So services will spam until.... [19:17:56] I have other things on my plate, unless it's an emergency then i will leave as is. [19:19:08] Reception123: still around? [19:20:09] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvcp0 [19:20:11] [02miraheze/services] 07MirahezeSSLBot 03c34399e - BOT: Updating services config for wikis [19:23:20] RhinosF1 i think he's off for the night. (though could be wrong) [19:25:07] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvcpD [19:25:09] [02miraheze/services] 07MirahezeSSLBot 03d186628 - BOT: Updating services config for wikis [19:30:09] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvchk [19:30:10] [02miraheze/services] 07MirahezeSSLBot 0336c1642 - BOT: Updating services config for wikis [19:31:32] RhinosF1: yes [19:31:39] paladox|UKInEU: nope :) [19:31:53] i'm wrong :P [19:35:08] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvchG [19:35:09] [02miraheze/services] 07MirahezeSSLBot 0300719bd - BOT: Updating services config for wikis [19:35:31] Reception123: can you drop issabeleefan2wiki or whatever it is [19:35:40] The one setting off them alerts [19:35:57] It has no right to be on Miraheze [19:37:18] RhinosF1: hm? what's its status? has it been deleted? [19:37:45] Reception123: it looks like ot [19:37:46] since if you are asking me to 'drop' you are implying that someone deleted it on mw1 [19:37:50] paladox|UKInEU: ^ can you confirm? [19:38:10] Reception123 i didn't delete it. [19:38:12] Reception123: I can’t tell who deleted it how [19:38:18] But it’s showing 404 [19:38:27] It’s a wiki of an ATS sock [19:38:27] RhinosF1: well that could just be on-wiki deletion [19:38:36] Reception123: no log to tell [19:38:41] RhinosF1: and if it's deleted on-wiki it's fine tbh it'll be dropped when the 14 days pass [19:39:00] it can't be seen or undeleted by anyone else but staff so I'll just delete it when I delete the others for inactivity or whatever [19:39:07] Reception123: I just want services to be shut up tbh from adding and deleting it [19:39:57] RhinosF1: oh you mean the spam from IRC [19:40:02] if I see one more I'll take care of it [19:40:08] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvch8 [19:40:10] [02miraheze/services] 07MirahezeSSLBot 033225999 - BOT: Updating services config for wikis [19:40:30] RhinosF1: wow... it's like the bot is human [19:41:10] Reception123: yeah... [19:41:54] !log deleted isabellafan2wiki [19:42:17] !log MariaDB [(none)]> DROP DATABASE isabellafan2wiki; [19:42:20] now be quiet Not-5a0a [19:42:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [19:42:34] Thanks [19:42:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [19:45:09] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvchw [19:45:10] [02miraheze/services] 07MirahezeSSLBot 03bd1f466 - BOT: Updating services config for wikis [19:46:06] Not-5a0a: why the hell do you keep adding it? [19:46:20] Reception123: whats up with services? [19:47:58] Reception123 did you remove it from mhglobal to? [19:48:47] paladox|UKInEU: if you mean use the WikiManager method then yes [19:49:03] yup [19:49:09] just did what the docs said, don't think anything changed [19:50:09] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvch1 [19:50:11] [02miraheze/services] 07MirahezeSSLBot 03cef5a94 - BOT: Updating services config for wikis [19:55:08] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jvchb [19:55:09] [02miraheze/services] 07MirahezeSSLBot 030879f81 - BOT: Updating services config for wikis [19:56:08] what is going on... [19:56:32] heh [19:56:55] paladox|UKInEU: where could the services bot still be getting data for the wiki? [19:56:55] Reception123: i dont understand. This has never happened before [19:57:11] https://github.com/miraheze/MirahezeMagic/blob/master/maintenance/addWikiToServices.php [19:57:12] [ MirahezeMagic/addWikiToServices.php at master · miraheze/MirahezeMagic · GitHub ] - github.com [19:57:33] oh [19:57:35] i found it [19:57:41] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvchA [19:57:43] [02miraheze/puppet] 07paladox 037558667 - Update test2.yaml [19:57:43] can confirm it's not in cw_wikis [19:57:51] paladox|UKInEU: what is it? [19:59:03] Reception123: paladox messed up puppet and added services to anything on test2 [19:59:14] ohh [19:59:18] paladox|UKInEU: is that even production yey? [19:59:22] Yet? [19:59:25] RhinosF1: and uh since when do we have a test2? [19:59:48] Reception123: its one of the new ovh cloud servers [20:00:13] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvcjJ [20:00:15] [02miraheze/services] 07MirahezeSSLBot 03e40248f - BOT: Updating services config for wikis [20:00:43] RhinosF1 no, and test hosts are not production [20:00:48] Reception123: https://phabricator.miraheze.org/P259 [20:00:48] test1 is not production either [20:00:49] [ ✎ P259 New Server Setup ] - phabricator.miraheze.org [20:01:01] paladox|UKInEU: well an active usable testwiki [20:01:06] RhinosF1: yes, technically no test servers are 'production' :) [20:01:44] I know [20:02:22] paladox|UKInEU: going to have to get used to the new server names :) [20:02:29] heh [20:03:21] paladox|UKInEU: what's rdb and mon? [20:03:23] Reception123: i was going to do a comparison table for my own sake but I’m lazy [20:03:28] rdb is redis [20:03:31] Reception123: read the comments [20:03:31] mon is monitoring [20:03:35] RhinosF1: that's a good idea [20:03:37] paladox|UKInEU: ah :) [20:03:44] RhinosF1: I can help maybe after they become prod [20:04:07] Reception123: I was going to do a table on wiki with old server and what server it is [20:07:41] RhinosF1: oh that would be very useful, even just for me to be able to check :P [20:07:50] we could try a draft one of these days [20:08:14] Reception123: i could [20:13:09] Reception123: https://meta.miraheze.org/wiki/Tech:Cloud_Server_Comparison started [20:13:10] [ Tech:Cloud Server Comparison - Miraheze Meta ] - meta.miraheze.org [20:16:48] RhinosF1: ok [20:17:01] I'll try to edit but hope I don't make any mistakes [20:18:03] oh I guess you're doing it :) don't want an edit conflict [20:18:16] Reception123: I’ve populated a lot but I’m done for now [20:19:07] There is still cp6,db6,gluster1,jobrunner1,mail1 on cloud1 to add [20:19:12] Last 3 replace misc1 [20:19:50] And test2,cp7,db7,mon1,phab1,rdb2,services2 [20:19:58] ok [20:20:00] All but first 3 replace misc1 [20:20:20] Reception123: we also need to create individual tech: pages for the servers [20:20:32] RhinosF1: well not jobrunner [20:21:20] Reception123: jobrunner is on misc currently [20:21:31] RhinosF1: oh... [20:21:32] Is it not? [20:22:05] I have no idea I just didn't think it was [20:22:14] Oh ok [20:23:12] Reception123: I can probably have a bot iterate to create most of the Tech: pages for indvidual servers [20:23:39] RhinosF1: good, though we should wait till prod [20:23:45] That’ll have to wait until next week though at least [20:24:02] I’ll publish code when i have it finalised [20:24:13] Reception123: yeah the table will be enough for now [20:24:24] RhinosF1: do you know which cp is which? [20:24:44] Reception123: no [20:24:48] paladox|UKInEU: ? [20:24:53] ? [20:25:06] paladox|UKInEU: which cp is replacing which [20:25:39] paladox|UKInEU: cp2=cp6 and cp4=cp7? [20:25:52] cp4 will be replaced with cp6, cp2 will be replaced with cp2 [20:26:03] you mean cp2 with cp7? [20:26:07] no [20:26:15] paladox|UKInEU: " cp2 will be replaced with cp2" [20:26:17] cp2 is being moved to a vps in ovh [20:26:21] Okay [20:26:21] yup [20:26:28] paladox|UKInEU: oh, then why is there cp7? [20:26:39] for server 2 [20:26:43] that's in europe [20:26:51] specifically uk [20:27:00] paladox|UKInEU: so we're going to have two Europe cps? [20:27:04] yup [20:27:15] paladox|UKInEU: ok... why two in Europe but only one US? [20:27:17] Cool [20:27:29] Because it's costing us none to have 2 in europe :P [20:27:40] Ah fair enough [20:30:42] Reception123: https://phabricator.miraheze.org/T5221 so I don’t go insane and forget + discussiom [20:30:43] [ ⚓ T5221 Document RN to OVH Migration ] - phabricator.miraheze.org [20:31:36] RhinosF1: am I missing anything? [20:31:40] also guess test1 is going to need a rename soon [20:32:23] Reception123: test1 -> test2 I think [20:33:06] yup [20:33:13] Reception123: want to stick it as a subtask of the above [20:33:15] oh whoops I wrote test instead of test1 [20:33:37] RhinosF1: and ok [20:34:03] Reception123: you’re mssing lizardfs6 to glusterfs1 [20:34:14] Gluster1 even [20:34:53] ok adding [20:35:08] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvCe8 [20:35:09] [02miraheze/services] 07MirahezeSSLBot 03f79b99b - BOT: Updating services config for wikis [20:35:28] Why do we need that? [20:35:36] Reception123: you’ve also got puppet2 on twice [20:35:42] :( [20:35:51] paladox|UKInEU: so we're not confused about which server is which now :) [20:35:53] paladox|UKInEU: so me and reception remeber whats what [20:39:04] MikeV: hi! [20:40:29] RhinosF1: do you remember how we fixed the cookie issue? (see https://phabricator.miraheze.org/T4445) [20:40:30] [ ⚓ T4445 Update PDF Rendering for nenawiki.org and ndg.nenawiki.org ] - phabricator.miraheze.org [20:40:33] I don't see how it mysteriously came back [20:41:07] Hi, Reception [20:41:43] MikeV: I assume you're here because of the "cookie monster" issue [20:41:45] Yes, we blacklisted the ip [20:42:04] paladox|UKInEU: oh yes, so then what explains that it's back? [20:42:18] paladox|UKInEU: could it be related to the cloud transition in some way? [20:42:21] Had some time, interested in miraheze in general. [20:42:35] MikeV: Oh :) Please feel free to ask us anything here [20:42:55] Though at the moment we are currently working on transferring our servers to a new provider [20:46:29] Reception123 https://github.com/miraheze/mw-config/blob/3d9bc58cebcad7b698e505273f475651d144c84d/LocalWiki.php#L75 [20:46:30] [ mw-config/LocalWiki.php at 3d9bc58cebcad7b698e505273f475651d144c84d · miraheze/mw-config · GitHub ] - github.com [20:46:45] wgCookieWarningEnabled isn't even used heh [20:47:08] paladox|UKInEU: hm? what do you mean by not even used? [20:47:13] it worked until now... [20:47:16] OK, it look like Gadgets in installed, but it does not show up on the ManagerWiki/settings menu. I was looking for wikEd which can be installed as a gadget. [20:47:30] as you can see, it's not controling whether to enable CookieWarning [20:48:22] oh [20:48:30] actually, wgCookieWarningEnabled is an config within the extension [20:49:29] paladox|UKInEU: yeah [20:50:20] MikeV: Oh, Gadgets doesn't work with ManageWiki. You can read more about that here: https://www.mediawiki.org/wiki/Extension:Gadgets [20:50:21] [ Extension:Gadgets - MediaWiki ] - www.mediawiki.org [20:50:39] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JvCeM [20:50:41] [02miraheze/mw-config] 07paladox 03a92d157 - Put CookieWarning behind wmgCookieWarningEnabled This is a workaround to disable the notice for misc3. [20:50:42] [02mw-config] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vbvb3 [20:50:44] [02mw-config] 07paladox opened pull request 03#2875: Put CookieWarning behind wmgCookieWarningEnabled - 13https://git.io/JvCeD [20:51:36] MikeV: and also https://en.wikipedia.org/wiki/User:Cacycle/wikEd_installation#Creating_a_gadget [20:51:36] [WIKIPEDIA] User:Cacycle/wikEd installation#Creating a gadget | "..." [20:51:54] if you need any help with installing wikied I can help [20:52:01] miraheze/mw-config/paladox-patch-2/a92d157 - paladox The build passed. https://travis-ci.org/miraheze/mw-config/builds/648601510 [20:52:04] I see that AllTheTropes has an editor. Is it wikEd? [20:52:25] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JvCeQ [20:52:26] [02miraheze/mw-config] 07paladox 0345512e1 - Update LocalSettings.php [20:52:27] [02mw-config] 07paladox synchronize pull request 03#2875: Put CookieWarning behind wmgCookieWarningEnabled - 13https://git.io/JvCeD [20:53:24] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JvCe7 [20:53:26] [02miraheze/mw-config] 07paladox 03d38c9af - Update LocalWiki.php [20:53:27] [02mw-config] 07paladox synchronize pull request 03#2875: Put CookieWarning behind wmgCookieWarningEnabled - 13https://git.io/JvCeD [20:53:47] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JvCe5 [20:53:48] [02miraheze/mw-config] 07paladox 03d7e544b - Update LocalSettings.php [20:53:50] [02mw-config] 07paladox synchronize pull request 03#2875: Put CookieWarning behind wmgCookieWarningEnabled - 13https://git.io/JvCeD [20:53:59] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JvCeF [20:54:00] [02miraheze/mw-config] 07paladox 03e5360f8 - Update GlobalExtensions.php [20:54:02] [02mw-config] 07paladox synchronize pull request 03#2875: Put CookieWarning behind wmgCookieWarningEnabled - 13https://git.io/JvCeD [20:54:09] What would be the best way to find writers for my wiki on miraheze? [20:54:18] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JvCeN [20:54:19] [02miraheze/mw-config] 07paladox 034af7030 - Update LocalExtensions.php [20:54:21] [02mw-config] 07paladox synchronize pull request 03#2875: Put CookieWarning behind wmgCookieWarningEnabled - 13https://git.io/JvCeD [20:54:27] MikeV: I think they use https://www.mediawiki.org/wiki/Extension:CodeEditor [20:54:29] [ Extension:CodeEditor - MediaWiki ] - www.mediawiki.org [20:55:06] BurningPrincess: that's a tough one. have you added it here? https://meta.miraheze.org/wiki/Gazetteer_of_wikis [20:55:12] [02mw-config] 07paladox synchronize pull request 03#2875: Put CookieWarning behind wmgCookieWarningEnabled - 13https://git.io/JvCeD [20:55:13] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JvCex [20:55:15] [02miraheze/mw-config] 07paladox 037ee0eff - Update LocalSettings.php [20:55:19] [ Gazetteer of wikis - Miraheze Meta ] - meta.miraheze.org [20:55:37] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JvCep [20:55:38] [02miraheze/mw-config] 07paladox 0314bd0fe - Update LocalExtensions.php [20:55:40] [02mw-config] 07paladox synchronize pull request 03#2875: Put CookieWarning behind wmgCookieWarningEnabled - 13https://git.io/JvCeD [20:55:43] miraheze/mw-config/paladox-patch-2/4af7030 - paladox The build passed. https://travis-ci.org/miraheze/mw-config/builds/648602939 [20:55:44] Reception123, I would think going right to their user pages is spammy [20:56:22] BurningPrincess: yes, other methods would be preferred [20:56:55] Well, DevintArt rcuiting is running out of hits [20:58:22] Reception, can these two editors (CodeEditor and wikEd) be enabled so we can try one-at-a-time? Or do you have to do it? [20:58:51] I have no idea where to recuit [20:58:58] MikeV: You can enable CodeEditor from ManageWiki/extensions, while wikiEd would be a gadget. Would you like me to help you with setting up the gadget? [20:59:08] BurningPrincess: Unfortunately, I'm not sure where either... [20:59:10] CodeEditor is used for the MediaWiki: Namespace, and WikiEditor (everything else and can also be used for the MediaWiki: namespace) [20:59:30] [02mw-config] 07paladox closed pull request 03#2875: Put CookieWarning behind wmgCookieWarningEnabled - 13https://git.io/JvCeD [20:59:31] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/JvCvU [20:59:33] [02miraheze/mw-config] 07paladox 0377d5750 - Put CookieWarning behind wmgCookieWarningEnabled (#2875) * Put CookieWarning behind wmgCookieWarningEnabled This is a workaround to disable the notice for misc3. * Update LocalSettings.php * Update LocalWiki.php * Update LocalSettings.php * Update GlobalExtensions.php * Update LocalExtensions.php * Update LocalSettings.php * Update LocalExtensions.php [20:59:35] Reception123 ^ try that [20:59:41] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-2 [20:59:43] [02mw-config] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vbvb3 [20:59:45] Reception123, Well, I could try WorldForge or start recuiting via dA notes , but that is all I could think of [21:01:15] BurningPrincess: yeah, that coudl work [21:01:17] *could [21:01:54] Reception123, I just consider notes a bit pushy as well, its the same thing as going right up to one's talk page [21:02:10] paladox|UKInEU: cookie notice still there :( [21:02:19] did you run puppet? [21:02:30] Reception, if you set up wikEd as a gadget, can I enable/disable it? [21:02:39] MikeV: yes, in your preferences [21:02:41] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 3 datacenters are down: 107.191.126.23/cpweb, 2400:6180:0:d0::403:f001/cpweb, 2a00:d880:5:8ea::ebc7/cpweb [21:02:50] paladox|UKInEU: well it's 2 shouldn't it have run by itself? [21:02:56] A qestiomn about the back end, how did you set up many wikis on one sevrer? [21:03:09] puppet takes more then 2 mins to run [21:03:14] due to the load [21:03:15] paladox|UKInEU: ok then I'll wait [21:03:20] Reception123 anyways try now? [21:04:13] BurningPrincess: a wiki can be served from any mw* server in the production cluster [21:04:27] paladox|UKInEU: still there :( [21:04:38] ok :( [21:04:43] RhinosF1, I saw about how to set up many wikis on one sevrer, but it was kinda too much for me [21:05:05] Reception, OK. Please set up widEd. Do I need to do a Maniphest request? [21:05:21] [02miraheze/mw-config] 07paladox pushed 031 commit to 03revert-2875-paladox-patch-2 [+0/-0/±4] 13https://git.io/JvCvO [21:05:23] [02miraheze/mw-config] 07paladox 038d15a77 - Revert "Put CookieWarning behind wmgCookieWarningEnabled (#2875)" This reverts commit 77d5750e7e636c48be0de6ec85798b81ce6a76ad. [21:05:24] [02mw-config] 07paladox created branch 03revert-2875-paladox-patch-2 - 13https://git.io/vbvb3 [21:05:27] [02mw-config] 07paladox opened pull request 03#2876: Revert "Put CookieWarning behind wmgCookieWarningEnabled" - 13https://git.io/JvCv3 [21:05:33] [02mw-config] 07paladox closed pull request 03#2876: Revert "Put CookieWarning behind wmgCookieWarningEnabled" - 13https://git.io/JvCv3 [21:05:35] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/JvCvs [21:05:36] [02miraheze/mw-config] 07paladox 0389c9429 - Revert "Put CookieWarning behind wmgCookieWarningEnabled (#2875)" (#2876) This reverts commit 77d5750e7e636c48be0de6ec85798b81ce6a76ad. [21:05:38] [02mw-config] 07paladox deleted branch 03revert-2875-paladox-patch-2 - 13https://git.io/vbvb3 [21:05:39] [02miraheze/mw-config] 07paladox deleted branch 03revert-2875-paladox-patch-2 [21:05:43] BurningPrincess: I mean we have over 4000 running on one installation [21:05:44] MikeV: It won't be necessary, I can do it now [21:06:41] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [21:06:50] miraheze/mw-config/revert-2875-paladox-patch-2/8d15a77 - paladox The build has errored. https://travis-ci.org/miraheze/mw-config/builds/648608199 [21:06:51] RhinosF1, damm, that is scary [21:07:06] MikeV: You can now try to enable it in https://nenawiki.org/wiki/Special:Preferences#mw-prefsection-gadgets and see if it works [21:07:11] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 2400:6180:0:d0::403:f001/cpweb [21:07:11] [ Log in - NENA Knowledge Base ] - nenawiki.org [21:08:49] BurningPrincess: the only limit is your server/database capacity [21:09:10] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [21:09:30] RhinosF1, I just foubnd the stuff about setting up many wikis on a sevrer kinda too much [21:10:22] BurningPrincess: okay, we’re happy to answer any questions [21:10:38] What were you reading that confused you? [21:12:22] RhinosF1, A mediawiki guide on setting up many wikis on one sevrer [21:12:34] BurningPrincess: link? [21:12:36] * BurningPrincess hugs RhinosF1 [21:13:26] https://www.mediawiki.org/wiki/Manual:Wiki_family [21:13:26] [ Manual:Wiki family - MediaWiki ] - www.mediawiki.org [21:15:34] I’ll make a note and see if we can simplify it [21:16:46] RhinosF1, how? [21:19:58] RhinosF1, the 2ed way seems like the best for me , but it seems hard [21:20:55] Unfortunately I will have to leave now, but you can either ask the others here (RhinosF1 and/or paladox|UKInEU) or email me if you need anything else and I'll take a look later [21:21:32] Bye [21:21:33] BurningPrincess: I’ll make the instructions simpler [21:21:38] Not the process [21:21:39] RhinosF1, thanks [21:21:50] It might take a while though [21:21:58] And i have a useless memory [21:22:07] Probly will help a lot of others a swell [21:22:41] Yeah [21:24:01] Reception, Thanks! I'll try it. [21:34:23] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 128.199.139.216/cpweb [21:36:22] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [21:37:21] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvCfT [21:37:22] [02miraheze/puppet] 07paladox 035367cd1 - Update test2.yaml [22:01:16] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvCfi [22:01:18] [02miraheze/dns] 07paladox 03adb3b70 - Add services1 to dns [22:04:23] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvCfy [22:04:25] [02miraheze/dns] 07paladox 03e5d9b8c - fix [22:11:01] * BurningPrincess feed Voidwalker a coookie [22:12:55] thanks, network has been rather annoying recently :( [22:14:27] How urgently does Miraheze need the donations? [22:22:26] are there bots here that could create default pages with information extracted from Wikidata for example? curiosity example: https://hispano.miraheze.org/wiki/Info:Sara_Ram%C3%ADrez [22:22:27] [ Info:Sara Ramírez - Hispano ] - hispano.miraheze.org [22:22:27] Miraheze can always use your donations [22:30:48] ^ [22:51:01] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/JvCJF [22:51:02] [02miraheze/puppet] 07paladox 032032fcc - services1: Set puppetmaster to puppet2 [23:00:01] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-8 [+0/-0/±1] 13https://git.io/JvCUG [23:00:02] [02miraheze/puppet] 07paladox 0376c5c30 - Enable services on services1.m.org [23:00:04] [02puppet] 07paladox created branch 03paladox-patch-8 - 13https://git.io/vbiAS [23:00:05] [02puppet] 07paladox opened pull request 03#1196: Enable services on services1.m.org - 13https://git.io/JvCUZ [23:01:12] :) [23:01:59] [02puppet] 07paladox closed pull request 03#1196: Enable services on services1.m.org - 13https://git.io/JvCUZ [23:02:00] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvCUC [23:02:02] [02miraheze/puppet] 07paladox 03a2ff785 - Enable services on services1.m.org (#1196) [23:02:09] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-8 [23:02:10] [02puppet] 07paladox deleted branch 03paladox-patch-8 - 13https://git.io/vbiAS [23:03:26] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 107.191.126.23/cpweb [23:05:21] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [23:10:42] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvCU6 [23:10:44] [02miraheze/puppet] 07paladox 035c6a4ec - services: Install make [23:11:54] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvCUi [23:11:56] [02miraheze/puppet] 07paladox 036a10086 - services: Install g++ [23:13:04] PROBLEM - cp2 Stunnel Http for mw3 on cp2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:13:31] PROBLEM - cp3 Stunnel Http for mw3 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:14:03] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 2604:180:0:33b::2/cpweb [23:15:01] RECOVERY - cp2 Stunnel Http for mw3 on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 15288 bytes in 0.609 second response time [23:15:33] RECOVERY - cp3 Stunnel Http for mw3 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15302 bytes in 2.727 second response time [23:16:03] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [23:22:00] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 2 datacenters are down: 2604:180:0:33b::2/cpweb, 2a00:d880:5:8ea::ebc7/cpweb [23:23:58] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [23:50:57] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvCTX [23:50:59] [02miraheze/puppet] 07paladox 03b85e156 - sysctl: Tune some configs [23:53:55] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+4/-0/±0] 13https://git.io/JvCTS [23:53:57] [02miraheze/puppet] 07paladox 038a4db59 - Add sysfs module This is copied from https://github.com/wikimedia/puppet/tree/e0b15db0f5b5205456a26551babcd99bbed09439/modules/sysfs [23:53:57] [ puppet/modules/sysfs at e0b15db0f5b5205456a26551babcd99bbed09439 · wikimedia/puppet · GitHub ] - github.com