[00:15:09] <Not-5a0a>	 [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvZO0
[00:15:10] <Not-5a0a>	 [02miraheze/services] 07MirahezeSSLBot 032a73c79 - BOT: Updating services config for wikis
[00:20:09] <Not-5a0a>	 [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvZOg
[00:20:10] <Not-5a0a>	 [02miraheze/services] 07MirahezeSSLBot 03120a7c4 - BOT: Updating services config for wikis
[01:16:17] <icinga-miraheze>	 PROBLEM - mw3 Puppet on mw3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 4 minutes ago with 1 failures. Failed resources (up to 3 shown)
[01:22:20] <icinga-miraheze>	 RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 33 seconds ago with 0 failures
[01:34:19] <icinga-miraheze>	 PROBLEM - mw3 Puppet on mw3 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown)
[01:41:25] <paladox|UKInEU>	 !log reboot mw3
[01:41:42] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master
[01:42:21] <icinga-miraheze>	 RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 55 seconds ago with 0 failures
[06:26:46] <icinga-miraheze>	 RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 3021 MB (12% inode=94%);
[07:05:28] <RhinosF1>	 Reception123: can you check https://meta.miraheze.org/w/index.php?title=Community_noticeboard/Archive_8&diff=next&oldid=96218
[07:05:30] <ZppixBot>	 [ Difference between revisions of "Community noticeboard/Archive 8" - Miraheze Meta ] - meta.miraheze.org
[07:29:01] <Reception123>	 RhinosF1: well to me it's okay because there was no response after all it shouldn't be archived with no resp
[11:09:34] <icinga-miraheze>	 PROBLEM - cp4 Current Load on cp4 is WARNING: WARNING - load average: 0.38, 1.83, 1.57
[11:11:33] <icinga-miraheze>	 RECOVERY - cp4 Current Load on cp4 is OK: OK - load average: 0.18, 1.01, 1.30
[11:36:46] <icinga-miraheze>	 PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2643 MB (10% inode=94%);
[15:26:09] <RhinosF1>	 Reception123: i meant the changing signatures
[15:27:18] <Reception123>	 Oh
[15:46:45] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[15:47:07] <icinga-miraheze>	 PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 1 backends are down. mw3
[15:47:31] <icinga-miraheze>	 PROBLEM - cp4 Stunnel Http for mw3 on cp4 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 328 bytes in 0.128 second response time
[15:48:05] <icinga-miraheze>	 PROBLEM - cp4 Varnish Backends on cp4 is CRITICAL: 1 backends are down. mw3
[15:48:21] <icinga-miraheze>	 PROBLEM - cp2 Varnish Backends on cp2 is CRITICAL: 2 backends are down. mw1 mw3
[15:48:46] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[15:50:22] <icinga-miraheze>	 PROBLEM - cp3 Stunnel Http for mw1 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[15:50:55] <icinga-miraheze>	 PROBLEM - cp4 Stunnel Http for mw1 on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[15:51:36] <icinga-miraheze>	 PROBLEM - cp2 Stunnel Http for mw1 on cp2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[15:51:45] <icinga-miraheze>	 PROBLEM - cp2 Stunnel Http for mw2 on cp2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[15:52:13] <icinga-miraheze>	 PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 4 datacenters are down: 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[15:52:23] <icinga-miraheze>	 RECOVERY - cp3 Stunnel Http for mw1 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15301 bytes in 0.500 second response time
[15:52:45] <RhinosF1>	 paladox|UKInEU: ^
[15:52:54] <icinga-miraheze>	 RECOVERY - cp4 Stunnel Http for mw1 on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 15301 bytes in 0.004 second response time
[15:53:18] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 2604:180:0:33b::2/cpweb
[15:53:33] <icinga-miraheze>	 RECOVERY - cp2 Stunnel Http for mw1 on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 15301 bytes in 0.501 second response time
[15:53:50] <icinga-miraheze>	 RECOVERY - cp2 Stunnel Http for mw2 on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 15288 bytes in 6.716 second response time
[15:54:09] <icinga-miraheze>	 RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online
[15:54:12] <icinga-miraheze>	 RECOVERY - cp4 Stunnel Http for mw3 on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 15288 bytes in 0.227 second response time
[15:54:17] <icinga-miraheze>	 RECOVERY - cp4 Varnish Backends on cp4 is OK: All 7 backends are healthy
[15:54:28] <icinga-miraheze>	 RECOVERY - cp2 Varnish Backends on cp2 is OK: All 7 backends are healthy
[15:55:13] <icinga-miraheze>	 RECOVERY - cp3 Varnish Backends on cp3 is OK: All 6 backends are healthy
[15:55:15] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[18:35:22] <ZppixBot>	 Hello MikeV! If you have any questions, feel free to ask and someone should answer soon.
[20:18:04] <MikeV>	 Is anyone available to help with Maniphest T5197?
[20:21:43] <paladox|UKInEU>	 MikeV hi
[20:22:04] <MikeV>	 Hello paladox
[20:22:18] <paladox|UKInEU>	 MikeV would you be able to explain https://phabricator.miraheze.org/T5197 to me, what carn't the user edit?
[20:22:19] <ZppixBot>	 [ ⚓ T5197 Help! Users lost edit rights on nenawiki.org. ] - phabricator.miraheze.org
[20:22:27] <paladox|UKInEU>	 the group has the edit rights
[20:23:15] <MikeV>	 RhinosF1 said that something has to be changed in LS.php
[20:24:45] <MikeV>	 Apparently, there is a namespace issue.
[20:25:29] <paladox|UKInEU>	 MikeV Special:ManageWiki/namespaces
[20:26:48] <icinga-miraheze>	 PROBLEM - mw1 MediaWiki Rendering on mw1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[20:26:51] <icinga-miraheze>	 PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 128.199.139.216/cpweb, 2400:6180:0:d0::403:f001/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[20:27:06] <paladox|UKInEU>	 ^ caused by a wp outage
[20:27:18] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 3 datacenters are down: 2604:180:0:33b::2/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb
[20:27:25] <icinga-miraheze>	 PROBLEM - cp3 Varnish Backends on cp3 is CRITICAL: 2 backends are down. mw1 mw3
[20:27:34] <icinga-miraheze>	 PROBLEM - cp2 Stunnel Http for mw2 on cp2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[20:28:21] <icinga-miraheze>	 PROBLEM - cp3 Stunnel Http for mw3 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[20:28:32] <icinga-miraheze>	 PROBLEM - cp4 Stunnel Http for mw3 on cp4 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[20:28:51] <icinga-miraheze>	 PROBLEM - cp4 Varnish Backends on cp4 is CRITICAL: 2 backends are down. mw2 mw3
[20:28:58] <icinga-miraheze>	 PROBLEM - cp2 Varnish Backends on cp2 is CRITICAL: 3 backends are down. lizardfs6 mw2 mw3
[20:28:58] <icinga-miraheze>	 PROBLEM - mw3 MediaWiki Rendering on mw3 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 1351 bytes in 0.061 second response time
[20:31:19] <icinga-miraheze>	 PROBLEM - cp2 Stunnel Http for mw3 on cp2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[20:31:47] <icinga-miraheze>	 PROBLEM - cp3 Stunnel Http for mw2 on cp3 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds.
[20:32:53] <icinga-miraheze>	 RECOVERY - cp3 Stunnel Http for mw3 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15288 bytes in 0.506 second response time
[20:33:00] <icinga-miraheze>	 RECOVERY - cp4 Stunnel Http for mw3 on cp4 is OK: HTTP OK: HTTP/1.1 200 OK - 15296 bytes in 0.384 second response time
[20:33:18] <icinga-miraheze>	 RECOVERY - cp2 Stunnel Http for mw3 on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 15296 bytes in 0.558 second response time
[20:33:28] <icinga-miraheze>	 RECOVERY - mw3 MediaWiki Rendering on mw3 is OK: HTTP OK: HTTP/1.1 200 OK - 18698 bytes in 1.097 second response time
[20:33:43] <icinga-miraheze>	 RECOVERY - mw1 MediaWiki Rendering on mw1 is OK: HTTP OK: HTTP/1.1 200 OK - 18699 bytes in 0.358 second response time
[20:33:45] <icinga-miraheze>	 RECOVERY - cp3 Stunnel Http for mw2 on cp3 is OK: HTTP OK: HTTP/1.1 200 OK - 15296 bytes in 0.572 second response time
[20:33:50] <icinga-miraheze>	 RECOVERY - cp3 Varnish Backends on cp3 is OK: All 6 backends are healthy
[20:33:51] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[20:34:19] <icinga-miraheze>	 RECOVERY - cp2 Stunnel Http for mw2 on cp2 is OK: HTTP OK: HTTP/1.1 200 OK - 15302 bytes in 0.392 second response time
[20:35:00] <icinga-miraheze>	 RECOVERY - cp4 Varnish Backends on cp4 is OK: All 7 backends are healthy
[20:35:05] <icinga-miraheze>	 RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online
[20:35:17] <icinga-miraheze>	 RECOVERY - cp2 Varnish Backends on cp2 is OK: All 7 backends are healthy
[20:40:40] <paladox|UKInEU>	 resolved
[20:41:04] <MikeV>	 paladox, I don't see anything on Special:ManageWiki/namespaces that look different than another site ndg.nenawiki.org that does not have this problem.
[20:41:41] <MikeV>	 Our site is set up so that only members of editors and administrators can edit content.  Now, editors cannot edit pages.
[20:41:52] <paladox|UKInEU>	 MikeV i'm not exactly sure what the issue you have. I inititally thought you ment users loosing the ability to edit perms, but i guess you mean users have lost that group from them?
[20:42:58] <RhinosF1>	 paladox|UKInEU: see my phab comment
[20:43:05] <paladox|UKInEU>	 i did
[20:43:15] <paladox|UKInEU>	 which is what is confusing me
[20:54:25] <RhinosF1>	 paladox|UKInEU: is the right listed in LS.php
[20:54:35] <paladox|UKInEU>	 nope
[20:54:46] <RhinosF1>	 paladox|UKInEU: it needs to be
[20:55:05] <paladox|UKInEU>	 where though?
[20:55:38] <RhinosF1>	 paladox|UKInEU: where permissions are added to groups
[20:55:46] <RhinosF1>	 .github miraheze/config
[20:55:46] <ZppixBot>	 https://github.com/miraheze/config
[20:55:53] <paladox|UKInEU>	 wgManageWikiPermissionsAdditionalRights
[20:57:35] <Not-5a0a>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvZ9h
[20:57:37] <Not-5a0a>	 [02miraheze/mw-config] 07paladox 03eee39f6 - nenawikiwiki: Add edit-content-pages to editor
[20:57:38] <Not-5a0a>	 [02mw-config] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbvb3
[20:57:40] <Not-5a0a>	 [02mw-config] 07paladox opened pull request 03#2873: nenawikiwiki: Add edit-content-pages to editor - 13https://git.io/JvZ9j
[20:57:45] <paladox|UKInEU>	 RhinosF1 ^
[20:58:53] <RhinosF1>	 MikeV: Do you need editor to have edit-talkpages as well
[20:58:56] <travis-ci>	 miraheze/mw-config/paladox-patch-1/eee39f6 - paladox The build passed. https://travis-ci.org/miraheze/mw-config/builds/647052489
[20:59:11] <RhinosF1>	 Can you confirm any right we need to add to grouos
[20:59:35] <RhinosF1>	 paladox|UKInEU: LGTM
[20:59:46] <paladox|UKInEU>	 Do we know why it's not showing?
[21:00:15] <RhinosF1>	 paladox|UKInEU: it wont show in ManageWiki
[21:00:19] <paladox|UKInEU>	 why?
[21:00:35] <RhinosF1>	 paladox|UKInEU: custom groups dont
[21:00:55] <paladox|UKInEU>	 oh, but that's a perm, not a group
[21:01:09] <RhinosF1>	 paladox|UKInEU: rights even
[21:01:13] * RhinosF1 is sleepy
[21:01:20] <paladox|UKInEU>	 ok
[21:02:06] <paladox|UKInEU>	 RhinosF1 so nothing uses it Like an extension? (only asking to make sure i doin't cause a security issue by merging it)
[21:05:53] <RhinosF1>	 paladox|UKInEU: no
[21:06:39] <RhinosF1>	 paladox|UKInEU: wait for MikeV to confirm exactly which group should have what because his edit-talkpage and edit-whatever else need adding
[21:08:42] <MikeV>	 editors need to be able to edit pages, both content and talk.  nenamembers may only edit talk pages.  users can't edit anything.
[21:10:01] <Not-5a0a>	 [02mw-config] 07paladox synchronize pull request 03#2873: nenawikiwiki: Add edit-content-pages to editor - 13https://git.io/JvZ9j
[21:10:02] <Not-5a0a>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvZHw
[21:10:03] <paladox|UKInEU>	 MikeV ^
[21:10:03] <Not-5a0a>	 [02miraheze/mw-config] 07paladox 03f6ac24c - Update LocalSettings.php
[21:11:52] <RhinosF1>	 paladox|UKInEU: nenamembers need edit-talkpages ?
[21:12:08] <RhinosF1>	 Check the spelling in Special:ListGroupRights at the bottom
[21:12:14] <RhinosF1>	 And admin the adminpages one
[21:13:34] <paladox|UKInEU>	 no such think as edit-talkpages?
[21:14:32] <MikeV>	 edit-talkpages is (and has been) in nenamembers
[21:14:34] <RhinosF1>	 There is
[21:14:45] <RhinosF1>	 paladox|UKInEU: https://nenawiki.org/wiki/Special:ListGroupRights
[21:14:47] <ZppixBot>	 [ User group rights - NENA Knowledge Base ] - nenawiki.org
[21:14:58] <paladox|UKInEU>	 oh
[21:14:58] <RhinosF1>	 Look under namespace restricitons
[21:15:13] <RhinosF1>	 Make sure edit-admin-pages
[21:15:18] <RhinosF1>	 Is done as well
[21:15:28] <Not-5a0a>	 [02mw-config] 07paladox synchronize pull request 03#2873: nenawikiwiki: Add edit-content-pages to editor - 13https://git.io/JvZ9j
[21:15:29] <paladox|UKInEU>	 RhinosF1 MikeV &
[21:15:30] <Not-5a0a>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvZHX
[21:15:31] <Not-5a0a>	 [02miraheze/mw-config] 07paladox 0324a54aa - Update LocalSettings.php
[21:15:31] <paladox|UKInEU>	 *^
[21:15:51] <paladox|UKInEU>	 RhinosF1 for which group?
[21:15:57] <RhinosF1>	 paladox|UKInEU: sysop
[21:16:01] <paladox|UKInEU>	 ok
[21:16:36] <paladox|UKInEU>	 RhinosF1 done
[21:16:37] <Not-5a0a>	 [02mw-config] 07paladox synchronize pull request 03#2873: nenawikiwiki: Add edit-content-pages to editor - 13https://git.io/JvZ9j
[21:16:38] <Not-5a0a>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JvZHS
[21:16:40] <Not-5a0a>	 [02miraheze/mw-config] 07paladox 035d5e416 - Update LocalSettings.php
[21:17:30] <RhinosF1>	 paladox|UKInEU: LGTM but fix the commit title
[21:18:02] <Not-5a0a>	 [02mw-config] 07paladox edited pull request 03#2873: nenawikiwiki: Adjust wgManageWikiPermissionsAdditionalRights - 13https://git.io/JvZ9j
[21:21:39] <Not-5a0a>	 [02mw-config] 07paladox closed pull request 03#2873: nenawikiwiki: Adjust wgManageWikiPermissionsAdditionalRights - 13https://git.io/JvZ9j
[21:21:40] <Not-5a0a>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JvZHb
[21:21:42] <Not-5a0a>	 [02miraheze/mw-config] 07paladox 03e59f7fe - nenawikiwiki: Adjust wgManageWikiPermissionsAdditionalRights (#2873) * nenawikiwiki: Add edit-content-pages to editor * Update LocalSettings.php * Update LocalSettings.php * Update LocalSettings.php
[21:21:43] <Not-5a0a>	 [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-1
[21:21:45] <Not-5a0a>	 [02mw-config] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbvb3
[21:23:26] <MikeV>	 Looks like editors and edit again!
[21:23:35] <RhinosF1>	 Perfecy
[21:23:43] <MikeV>	 Looks like editors CAN edit again.
[21:25:23] <MikeV>	 Two more questions:  How could I break it by just adding delete rights to editors?   Why do new custom groups end up in "cannot change" where I cannot assign them to users?
[21:26:54] <RhinosF1>	 MikeV: A) It looked like it was set up weord and it was a catalyst B) go to Special:ManageWiki/permissions on the group you want to be able to add it and then group assignments and then tick the box for that group under ‘add to other users’
[21:29:36] <MikeV>	 Thanks RhinosF1 & paladox!  I did not understand the "Group Assignments" tab in ManageWiki.  
[21:41:12] <MikeV>	 Why was (is) edit-content-pages not in the ManageWiki "Assigned Permissions" or "Unassigned Permissions" tab?
[21:43:09] <RhinosF1>	 MikeV: it wont show
[22:02:08] <MikeV>	 I know.  Just wondering why.  I wasted a lot of time looking for it.
[22:10:34] <RhinosF1>	 MikeV: it’s not designed to