[09:12:58] <arturo>	 !log tools created tools-sgebastion-11 (buster) (T275865)
[09:13:03] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[09:13:03] <stashbot>	 T275865: Toolforge: migrate bastions to Debian Buster - https://phabricator.wikimedia.org/T275865
[09:20:40] <arturo>	 !log tools associate floating IP 185.15.56.66 to tools-sgebastion-10 (T275865)
[09:20:44] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[09:20:45] <stashbot>	 T275865: Toolforge: migrate bastions to Debian Buster - https://phabricator.wikimedia.org/T275865
[09:22:10] <arturo>	 !log tools create DNS A record `login-buster.toolforge.org` pointing to 185.15.56.66 (tools-sgebastion-10) (T275865)
[09:22:14] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[09:24:10] <arturo>	 !log tools allocate & associate floating IP 185.15.56.122 for tools-sgebastion-11, also with DNS A record `dev-buster.toolforge.org` (T275865)
[09:24:18] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[13:55:40] <CristianCantoro>	 Hi all, I have an issue with my project wikicommunityhealth. I have an instance running (called wchm01), I am trying to edit the security groups but I keep getting this error message: "Danger: An error occurred. Please try again later.". Any clue about what could it be generated by?
[13:59:27] <andrewbogott>	 CristianCantoro: this is probably a bug from a recent upgrade.  Do you mind if I add myself to your project so I can see for myself?
[14:01:42] <andrewbogott>	 CristianCantoro: also what rule are you adding to what group?
[14:02:01] <andrewbogott>	 (It's also possible that the backend doesn't like something about your input; it doesn't validate very well)
[14:07:08] <andrewbogott>	 (I need to step away but will follow up if you leave responses here)
[14:14:22] <CristianCantoro>	 andrewbogott: no problem, please go ahead
[14:14:50] <CristianCantoro>	 andrewbogott: I have added a web proxy for port 81 to have a dev deployment of the app
[14:15:01] <CristianCantoro>	 so I need a security group to allow that traffic in
[14:15:48] <CristianCantoro>	 andrewbogott: there is the same setup in another project I am helping with  (wcdo) and it worked fine
[14:17:02] <andrewbogott>	 CristianCantoro: is the error when creating the group or when adding the rule?
[14:18:02] <andrewbogott>	 In group dev_web I see this rule:
[14:18:03] <andrewbogott>	 Ingress 	IPv4 	TCP 	81 	172.16.0.0/21 
[14:19:07] <andrewbogott>	 and just now I was able to add a second rule to that group (for port 82)
[14:20:10] <CristianCantoro>	 andrewbogott: the error is when I edit the security groups for the instance
[14:20:14] <CristianCantoro>	 from the drop-down menu
[14:20:21] <andrewbogott>	 oh!  ok
[14:20:22] <CristianCantoro>	 I click and I get the error above
[14:20:26] <andrewbogott>	 which instance?
[14:20:39] <CristianCantoro>	 the only one there is: wchm01
[14:21:01] <andrewbogott>	 ok, I see the problem now
[14:21:26] <CristianCantoro>	 andrewbogott: :)
[14:23:01] <andrewbogott>	 CristianCantoro: I have added the group via the commandline to get you unblocked; I'll make a bug to investigate the Horizon issue
[14:24:29] <andrewbogott>	 T279667
[14:24:30] <stashbot>	 T279667: Horizon: 'edit security groups' instance menu produces an error - https://phabricator.wikimedia.org/T279667
[14:27:15] <CristianCantoro>	 andrewbogott: Thanks a lot
[16:46:21] <abartov>	 hi. I'm trying to look at some old instances I have in the cloud and am getting persistent "unexpected errors" with no more useful information.  What next?
[16:46:21] <abartov>	 e.g. https://horizon.wikimedia.org/project/instances/1144cb61-f7ab-436f-bd02-003e40667be6/
[16:53:31] <abartov>	 hang on, these are on project "bastion".  Why do I see them at all?
[17:30:49] <bstorm>	 That's a different one
[17:32:39] <bstorm>	 I think we should make a ticket for that as well. We are seeing some interesting bugs in horizon since the upgrade.
[17:33:50] <bstorm>	 andrewbogott: I'm beginning to wonder if there isn't some overarching issue around filtering and policies that changed in the upgrade.
[17:35:24] <Majavah>	 sorting the instance lists seems broken as well, and my javascript console is full of "Uncaught TypeError: jQuery.event.addProp is not a function"
[17:40:06] <bstorm>	 abartov: are you able to add a bit more context to T279697
[17:40:06] <stashbot>	 T279697: Horizon showing instances from the wrong project - https://phabricator.wikimedia.org/T279697
[17:40:15] <bstorm>	 Also Majavah :)
[17:40:57] <bstorm>	 These issues could all be related, or they could be totally different things.
[17:41:06] <bstorm>	 Fun stuff
[17:41:48] <Majavah>	 bstorm: I don't have any issues with seeing things in wrong projects, but unlike previous versions it does not let me click on column headers to sort by them
[17:42:03] <bstorm>	 So maybe I'll split that to another task
[17:42:09] <abartov>	 bstorm: gladly, but I'm not sure what to add.  What would help? 
[17:43:03] <bstorm>	 Well, a comment about exactly what you saw and any errors you saw would. In fact just making sure you are subscribed so we know who to check with when things are fixed is helpful :)
[17:43:43] <bstorm>	 Majavah: do you have time to make a new ticket and add the kanban and Horizon tags?
[17:43:57] <Majavah>	 bstorm: sure, will do in a few moments
[17:44:08] <bstorm>	 Thanks. 
[17:44:56] <andrewbogott>	 bstorm: thanks, I'm looking at some of these today
[17:45:44] <abartov>	 bstorm: okay, subscribed and added screenshots.
[17:45:52] <bstorm>	 Let me know if you want another set of eyes, such as they might be :)
[17:45:59] <bstorm>	 andrewbogott ^
[17:46:11] <andrewbogott>	 thanks
[17:46:52] <bstorm>	 abartov: I don't see the edits on the ticket? https://phabricator.wikimedia.org/T279697
[17:47:12] <bstorm>	 Did it not save?
[17:49:07] <bstorm>	 Subscribed you there
[17:49:16] <bstorm>	 I think I found the right phab account
[17:49:26] <Majavah>	 bstorm: T279699
[17:49:27] <stashbot>	 T279699: Horizon does not allow sorting tables anymore - https://phabricator.wikimedia.org/T279699
[17:49:33] <bstorm>	 Thanks!
[17:49:49] <bstorm>	 That one might be totally different
[17:49:55] <abartov>	 bstorm: whoops, pasted on the wrong ticket. Now fixed.
[17:50:18] <bstorm>	 Cool thanks!
[17:53:03] <Majavah>	 I think that task should be "Can't see instance details", if you have access to any other project it's expected that you have access in bastion but not being able to view instance details (I can't anywhere, tested tools and deployment-prep) is unexpected
[17:53:32] <Majavah>	 "bastion is selected by default" is not a new thing
[17:57:10] <andrewbogott>	 abartov: was bastion your selected project at the time?
[17:57:42] <abartov>	 andrewbogott: yes
[17:57:48] <andrewbogott>	 ah, ok
[17:57:51] <andrewbogott>	 so then Majavah is right
[18:00:55] <andrewbogott>	 I tried to comment on task, please correct me if I misunderstand.
[18:07:23] <bstorm>	 Yeah, that's likely not an error then
[18:08:55] <bstorm>	 Not entirely anyway. 
[18:25:15] <bstorm>	 !log tools cleaned up the deprecated entries in /data/project/.system_sge/gridengine/etc/submithosts for tools-sgegrid-master and tools-sgegrid-shadow using the old fqdns T277653
[18:25:21] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[18:25:22] <stashbot>	 T277653: Toolforge: migrate grid to Debian Buster - https://phabricator.wikimedia.org/T277653
[18:27:41] <bstorm>	 !log toolsbeta cleaned up the deprecated entries in /data/project/.system_sge/gridengine/etc/submithosts for toolsbeta-sgegrid-master and toolsbeta-sgegrid-shadow using the old fqdns T277653
[18:27:45] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL