[11:02:47] !log tools.network-tests created https://toolsadmin.wikimedia.org/tools/id/network-tests [11:02:48] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.network-tests/SAL [11:15:53] !log tools.network-tests started webservice, include a couple of dummy files for download [11:15:55] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.network-tests/SAL [15:58:42] !log admin deploying https://gerrit.wikimedia.org/r/c/operations/puppet/+/664845 to cloudnet servers (T268335) [15:58:46] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [15:58:46] T268335: cloud: neutron l3 agent: improve failover handling - https://phabricator.wikimedia.org/T268335 [19:28:25] Who could i talk to privately regarding some access issues? [19:28:32] toolforge access issues* [19:47:40] Zppix, what issues are you having? [19:49:44] balloons: pm? [21:26:24] !log tools deleted tools-puppetdb-01 since it is unused at this time (and undersized anyway) [21:26:29] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [22:13:56] andrewbogott: I have replied to your comments, see https://phabricator.wikimedia.org/T127717#6839486. It may take a while to read, but in the case you happen to have spare time at the moment, we can have a short conversation on the topic now [22:14:36] otherwise, we can schedule a chat at a later moment [22:14:57] reading! [22:20:12] SPF|Cloud: your diagram is roughly what I was envisioning. I think my main point of ignorance (and worry) is the kibana rbac thing [22:20:24] I'm not sure I'm ready to dive into this now but I will bring it up in our next meeting and see what people think. The safest thing might be to punt for a month or two and hope that the license mess has settled down [22:20:58] 'least privilege' is always a good idea, but I have some concerns with the directions, eg whether to go with production's ELK stack or managing our own [22:22:39] We would definitely want everything internal to cloud-VPS [22:23:06] I understand, it would be great if someone can fork the ELK stuff under a license that does work for us, since I presume ELK is among the best solutions in the market (well, Splunk is better, but that's definitely proprietary :)) [22:23:16] we can still create secure rbac-managed access on things running in a secure project. [22:23:30] Yeah, it seems VERY likely that either the ES folks will back down or someone will fork [22:23:58] (worst case scenario: both) [22:24:14] I guess that's the second-worst-case :( [22:25:41] looking forward to the thoughts of others, let me know. in the meantime, merely sending logs for on-disk storage (rsyslog -> remote -> file or similar, you name it) for WMCS staff is a compromise that works [22:26:46] yeah, it's not a bad idea. We'll have to think over whether that can wait for a full-blown elk stack [22:29:58] sure, my only recommendation is to take into account that 'audit log manipulation is possible' and 'project admins can view their own logs in a nice tool' can be seen as two separate things, reducing the impact of the first thing shouldn't be too hard, the second thing can be a longer-term project [22:30:18] but I guess you already know that :) [22:30:54] thanks for the help, much appreciated to have WMCS support, in order to make this possible [22:41:43] thank you for offering! Feel free to pester me next week if I haven't followed up. [22:45:00] ha, will do