[12:35:37] !log tools T223992 rebooting tools-redis-1002 [12:35:41] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [12:35:42] T223992: toolforge: redis issue - https://phabricator.wikimedia.org/T223992 [13:01:13] arturo: it looks like the `tools-db` alias no longer works. Is that intentional? [13:01:28] no, not intentional [13:01:40] could be related to T224000 musikanimal let me take a look [13:01:41] T224000: wmcs-wikireplica-dns: error: circular reference detected - https://phabricator.wikimedia.org/T224000 [13:01:53] okay thanks :) [13:02:52] I do recall reading that `tools-db` is deprecated [13:03:22] well, you can use `tools.db.svc.eqiad.wmflabs` anyway [13:03:34] and that works directly, since `tools-db` is just an alias of that [13:05:10] musikanimal: are you using `tools-db.tools.eqiad.wmflabs` or `tools-db.eqiad.wmflabs`? [13:07:50] I'm using `tools.db.svc.eqiad.wmflabs` [13:08:13] it works fine, I was just using `tools-db` by itself before [13:08:51] oh, `tools-db` directly? not domain or TLD? that's ugly :-S [13:09:22] it is! it's from way back in the way [13:09:52] I think it's fine to remove it, but we should update https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#Steps_to_create_a_user_database_on_tools.db.svc.eqiad.wmflabs [13:10:09] anyway, this works now [13:10:11] https://www.irccloud.com/pastebin/5Mv48r05/ [13:11:30] musikanimal: it already contains a warning about `tools-db` alias being deprecated, right? [13:12:18] probably. It does say it's deprecated at https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#Steps_to_create_a_user_database_on_tools.db.svc.eqiad.wmflabs [13:13:18] some really old tools still use the ugly `tools-db`, but they can be updated [13:13:26] so, no update needed in wikitech, right? [13:13:58] just the "Caution: The DNS alias tools-db is deprecated." , might change it to say it used to exist and no longer works [13:16:01] ok! let me do it [13:16:37] done [16:25:37] wm-bot, help [16:29:19] wm-bot relay test from lucaswerkmeister [16:29:30] okay, that’s working [19:18:06] bd808 a hypothetical for you... if we were to make a namespace in MediaWiki that was only accessible to admins (afaik we haven't done that before?) would that be a huge problem? i.e. would the revisions on that namespace be able to be filtered out from replicas, etc.? [19:37:55] davidwbarratt, MediaWiki does not allow for that [19:38:04] Krenair I know it doesn't [19:38:13] errr... I know it doesn't right now [19:38:14] nor should it allow for that [19:38:18] why not? [19:38:58] afaik, there are access control hooks, so an extension should be able to do this [19:39:19] Because all of mediawiki assumes there is no such thing as different read rights. [19:39:39] so you don't just have to make sure that the page cannot be read, you also have to make sure it cannot be transcluded in any of many ways [19:40:00] in effect, you need a different architecture, and therefore a different wiki software [19:40:23] it is not designed to be a content management system [19:40:34] it is a CMS [19:40:41] it is not designed to protect sensitive data on only certain pages [19:41:00] I don't see why an extension couldn't do that [19:41:05] I certainly wouldnt allow code supporting such a mechanism into any extension I controlled [19:42:01] davidwbarratt: the way an extension could do that is by storing information in a table completely separate from the rest of the MW instance. Which means you're effectively deploying a new wiki, in which case... you might just want to deploy a new wiki. [19:42:19] why would it have to be in a different table? [19:42:49] because any and all other code, core or extension, is at liberty to expose it as they please [19:43:11] I mean they are, but if they aren't checking permissions right now, that is a serious security breach [19:43:24] as that would expose private wiki's content to the world [19:43:38] that's a different situation [19:43:46] private wikis are closed off _as a whole_ [19:43:46] is it? [19:43:52] rather than on a per-page or per-namespace level [19:43:54] it seems to check each title [19:44:11] i.e. a Title::userCan( 'read' ) [19:44:19] otherwise, why do we have that? [19:44:55] that probably should not be something controlled by the title class [19:45:04] it was moved into PermissionManager [19:45:05] but still [19:45:13] we are checking access on read (as far as I can tell) [19:45:16] for every title. [19:45:42] seems like a huge waste if that's impossible [19:46:36] davidwbarratt: https://www.mediawiki.org/wiki/Manual:Preventing_access#Restrict_viewing [19:47:24] davidwbarratt: but note that this only works if unauthorized people cannot edit _any_ pages, cannot use the api at all, etc. [19:48:30] davidwbarratt: because, as noted above, you may be able to block me from reading MyNamespace:MyPage directly, it is unlikely that you can block me from transcluding that page in one of a million ways [19:48:45] davidwbarratt: so, the only supported 'rights managed' solution is to deploy a completely separate wiki [19:49:07] and that wiki can be excluded from replication, as is done for all other private WMF wikis [19:49:35] it's also worth noting that the tasks for this type of restriction in core are wontfix [19:49:35] ok, let's just say for the sake of argument that we allowed MediaWiki (via an extension) to support dynamic permissions on read [19:50:08] i.e. MediaWiki didn't support it, but there was nothing preventing an extension from doing it [19:50:47] unless core supports it, no extension should claim to [19:51:01] it technically does [19:51:18] now if there are bugs with that that should be fixed, that is a different issue [19:51:41] and by saying it technicall does, I mean that it's running a hook to determine access permissions on read [19:51:44] davidwbarratt: I'm not sure what you're trying to argue here [19:51:54] I'm not trying to argue anything [19:52:00] it already technically supports it [19:52:02] the docs have a rather explicit contradiction of that [19:52:13] well then they should be updated against what is in the code [19:52:16] because the code allows it [19:52:47] if the code somehow allows for it right now that is likely by mistake [19:53:09] it doesn't appear to be, someone whitelisted to run the hook when checking for a `read` [19:53:21] but yeah maybe they added that by mistake [19:53:29] but even in that doc you linked, you can whitelist public pages [19:53:38] so I assume this is the mechanism (in reverse) used for that [19:53:55] i.e. I can make a wiki private and whitelist specific pages as being public [19:54:02] davidwbarratt: yes, and, as that page also notes, this only works in very specific circumstances [19:54:03] I'm saying, if you can do that, you can also do it the other way 'round [19:54:09] I understand [19:54:20] but there's nothing "technically" preventing it [19:54:24] davidwbarratt: and that circumstance does not exist in the case of a public wiki with a namespace added [19:54:45] I'm not sure where this dogma is coming from [19:54:50] It is a longstanding principle of 14-15 years that MediaWiki does not allow for this. [19:54:50] no, technically you could rewrite all of mediawiki and end up with twiki, or dokuwiki, or one of the many other solutions that support extensive rights management [19:54:58] why bnot? [19:55:19] well it clearly does allow for it [19:55:47] I have no idea how well it works, but it's technically possible [19:55:54] but this is completely besides my point [19:56:17] I'm really just asking, from an operations perspective, if it would be a problem to filter the revisions table out (for the replicas) by a namespace [19:56:33] I think you'll find the revision table to be merely the tip of the iceberg [19:56:56] _sigh_ [19:56:59] yes, it would be possible to filter the revision table by namespace [19:57:08] thank you [19:57:25] however, I don't see it happening [19:58:23] understood [19:58:29] why would a wikimedia wiki want to use it anyway? [20:05:04] if we wanted to store content/data that would only be visible to permissionsed users. [20:06:36] okay but why would a wikimedia wiki want to do that? [20:12:02] Krenair let's say someone is being an ass-hole and harassing you on our projects, if you were to tell someone (on wiki) that information would be public and the ass-hole could become even more of a problem for you (maybe they start doxxing you, or sending you death threats or stalking you). It would be safer for you to be able to tell admins of a wiki without that information being visible to the ass-hole [20:12:32] we have mechanisms for that [20:12:41] I know [20:13:07] Isnt that what email, otrs are for? [20:13:14] yes [20:13:17] it's jsut an idea I had [20:14:10] I was just exploring the technical feasability of it [20:15:27] it probably is technically feasible to make such a restriction on the revision table [20:15:42] (on the view of it) [20:15:55] but that would probably make the revision table useless due to the performance implications [20:16:07] so not realistic [20:19:15] I would suggest the direction of modifying OTRS (or another system that supports the main use case -- including fine-grained permissions!) to authenticate using MW users & uses MW groups to set those permissions [20:21:31] understood [20:23:18] if it has the performance effect I think it'd have we may as well get rid of the replicas [20:25:01] some of the other tables might be doable using joins, but that likely has the same problem if not worse [20:26:46] comment table might be impossible [20:41:31] sounds like we need to fix the database. :) [21:03:05] davidwbarratt: wow. Lots of discussion in scrollback. I think the part I can add that maybe wasn't said is that redacting that kind of data from the wiki replicas would probably be something that we wanted done in the "santarium" layer that the DBAs maintain if possible. That's a filtering step that happens inside the Prod environment before data gets replicated to the Cloud environment. [21:03:28] I'll stay out of the discussion about if the feature itself is a good idea or not. ;) [21:03:32] oh ok [21:03:55] ok, so from your perspective, it wouldn't be crazy to do that? [21:04:28] errr... if we identified all of the places that needed to be filtered and how to filter them [21:04:35] it is a thing that is done already in some cases. That layer mostly however drops whole tables and columns rather than filtering out rows [21:04:58] ah, I see [21:05:02] most (all?) of the row level filtering is done in the view layer [21:05:27] and that is also possible and done for things like revdel and suporession [21:05:31] *supression [21:05:46] oh ok, yeah I was wondering about that [21:05:53] does the sanitarium do any row-level filtering? [21:06:14] Relatedly, we don't have the page content in the wiki replicas, so if that is all that is being protected it wouldn't effect us there [21:06:33] Krenair: I'm not sure. I don't know that it does, so it may not [21:07:10] I thought it was basically just a big whitelist of tables and columns but how it works and is configured exactly has always been unknown to me [21:09:15] davidwbarratt: the maintain-views.yaml file describes the view level filtering -- https://github.com/wikimedia/puppet/blob/production/modules/profile/templates/labs/db/views/maintain-views.yaml [21:09:47] row level filtering there is done with bits like "if(ar_deleted&4,null,ar_user) as ar_user" [21:10:28] that nulls the ar_user column for a row if the same row has the ar_deleted bits set to do so [21:10:44] this kind of filtering is runtime expensive [21:10:53] but possible [21:11:14] fascinating [21:11:17] thanks! [23:16:29] hi, i would like to know if the puppet module "cgred" is still being used nowadays.. on tools bastions [23:16:42] it was introduced in https://phabricator.wikimedia.org/T131541 [23:17:02] if it's still used we would need a systemd unit file for that [23:17:24] to be able to convert it to systemd::service [23:39:27] mutante: replied on the old task you found, but we are not using it on the Toolforge bastions anymore. a.rturo replaced it with systemd magic when we migrated to Stretch [23:43:30] bd808: yay, thanks! that is the easiest way forward me.. what i am REALLY after is replacing all occurences of base::service_unit and this only had sysvinit.not systemd [23:47:50] ah, makes sense. I think this is something that can just go away. We only have one Trusty server left and it's not using it (labstore1003) [23:53:20] i am uploading a change to delete the module and will add people you mentioned [23:53:30] ack, cool