[06:51:51] !log toolsbeta depool toolsbeta-test-k8s-ingress-1 [06:51:53] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [09:44:16] !log wikidata-dev wb-reconcile: created instance [09:44:18] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikidata-dev/SAL [09:53:04] !log wikidata-dev wb-reconcile: installed MediaWiki + WikibaseReconcileEdit with Ansible; no HTTP server running yet AFAICT [09:53:06] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikidata-dev/SAL [10:22:40] !log wikidata-dev wb-reconcile: fixed the Ansible setup; added wikibase-reconcile-testing.wmcloud.org proxy [10:22:42] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikidata-dev/SAL [10:44:13] !log tools create tools-k8s-ingress-[4-6] T264221 [10:44:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [10:44:17] T264221: Upgrade the nginx ingress controller in Toolforge (and likely PAWS) - https://phabricator.wikimedia.org/T264221 [11:09:12] !log tools deploy helm-based nginx ingress controller v0.46.0 to ingress-nginx-gen2 namespace T264221 [11:09:16] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [11:09:17] T264221: Upgrade the nginx ingress controller in Toolforge (and likely PAWS) - https://phabricator.wikimedia.org/T264221 [11:35:29] !log toolsbeta testing https://gerrit.wikimedia.org/r/c/operations/puppet/+/692875/ [11:35:32] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [12:15:09] !log tools rollback ingress-nginx-gen2 [12:15:15] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [12:29:54] hi, the bullseye VPS image works well -- thanks andrewbogott ! [12:30:30] hm... why are imagemagic jobs getting killed? [12:30:37] is this on the grid engine as well? [12:31:01] a followup question, I'm looking into porting swift to bullseye though in production runs on stretch, I'd like to have two VMs bullseye/stretch side by side; how much work would it be to re-activate the stretch image for 'swift' project only ? would it work out of the box with e.g. ceph andrewbogott ? [12:46:03] godog: I can enable stretch temporarily; what project? [12:47:46] andrewbogott: thank you! the 'swift' project [12:51:23] godog: done [12:52:38] andrewbogott: thank you! appreciate it [12:54:00] sure thing, lmk if it doesn't work :) [14:52:59] !log tools.wikibugs now running on irc.libera.chat via ~/libera directory [14:53:02] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL [15:24:48] andrewbogott: got it working! seems fine, I had to wrestle a little with sudo/ldap etc, what's the "ldap client stack" supposed to be for stretch VMs ? for my use case atm is "classic/sudoldap" and things seems to work, I can sudo passwordless no problem, but wanted to check if that's correct [16:09:33] Hey cloud team. I need to be added as a maintainer of https://cssk.toolforge.org/ (context https://cs.wikipedia.org/wiki/Diskuse_s_wikipedistou:Blahma#Bug_in_https%3A%2F%2Fcssk.toolforge.org%2Fscripts%2Fcssk.js) can anybody help me with that? [16:12:27] Jdlrobson: the official/approved way to get added to a tool is via an existing maintainer. If the tool is abandoned there's a different process for that. [16:12:48] But of course I have not yet looked at the tool -- if it was exclusively maintained by no-longer-at-the-WMF staff then I can just add you :) [16:13:15] https://admin.toolforge.org/tool/cssk looks like the person you were talking to, Blahma, is the sole maintainer. So you should try asking them first. [16:13:41] @AntiComposite @andrewbogott that's the talk page [16:13:50] Blahma says "The extension has since been decommissioned, along with the linked wikiproject, which has lived past its horizon. What would be the best way to disenroll it from all the user scripts that are currently calling it? Feel free to take actions as you wish" [16:14:13] The action I'd like to take is to make that script 404 [16:14:35] which requires access to https://cssk.toolforge.org/ so I can remove the scripts/cssk.js file [16:17:02] Jdlrobson: The best way for you to take actions as you wish is for them to add you. Is there some reason why they won't or can't? [16:19:50] Okay I'll follow up with them again :) Thanks [16:21:47] Jdlrobson: if they don't reply, https://wikitech.wikimedia.org/wiki/Help:Toolforge/Abandoned_tool_policy is the official policy for that [16:23:51] Thanks Majavah it's only 1000 errors a day so not urgent. [16:23:53] Jdlrobson: sorry to stonewall you, this a standard "trying to avoid capricious WMF staff action in community space" thing [16:24:18] andrewbogott: no makes sense. The gadget/user script space is becoming more problematic though [16:24:45] we've had a few cases where bad scripts have caused JS errors for every user hitting our triggers and that becomes a staff issue now we have error logging.. [16:25:11] if anyone is interested I did set up https://phabricator.wikimedia.org/T262493 to get some clarity here [16:26:03] (any input welcomed particularly from the cloud perspective) [16:29:49] Majavah: MacFan4000: did you part? [16:29:55] yes [16:30:02] yep [16:30:16] legoktm or bd808: go in now [16:30:22] or andrewbogott [16:30:40] Let us know when you are in. [16:31:00] we're in [16:31:01] worked [16:32:54] people loading JS from abandoned toolforge tools seems not great from a security POV either. [16:33:55] Jdlrobson: anything specific you're looking from us? [16:46:06] andrewbogott: can I /join -cloud on libera? or should I just hold off and be happy I'm nick squatting [16:46:23] greg-g: feel free [16:51:44] Is wikimedia definitely headed to libera? [16:52:23] seems like it [16:52:34] we've got -cloud setup there [16:52:35] likely yes, but officially there are not yet any decisions [16:52:50] but the meta thread is mostly in favor of moving there [16:53:45] wm-bb, MacFan4000, Majavah WMCS staff will follow wherever the community leads. I'm sitting in channels over there in case people show up but I'm not yet to the point of telling people they /should/ show up there :) [16:54:01] Ok [17:26:04] https://meta.wikimedia.org/w/index.php?title=Wikimedia_Forum&diff=21473927&oldid=21473429&diffmode=source is the most recent statement from the WMGCs [17:26:53] thanks for the update AntiComposite [19:22:26] How long does it take before a newly-added projectadmin can log in to Horizon? [19:22:50] I added someone about 15 minutes ago, but he still reports not being able to log in. [19:32:16] bd808: ^ [19:33:10] ragesoss: able to login to horizon, or able to ssh to an instance in the project? [19:33:24] able to log into horizon. [19:33:28] he is able to ssh into an instance. [19:33:35] Horizon should be functionally immediate. ssh will need puppet to run on the instance [19:33:49] okay, so something else must be wrong? [19:33:59] Horizon does require 2FA for the Developer account [19:34:14] it sounds like he has that set up but I will confirm [19:36:06] He had said he entered the same creds and 2FA token as on wikitech.wikimedia.org but he gets 'Invalid credentials' [19:36:29] ragesoss: they should always be able to login to horizon, btw irrespective of project adminship [19:37:23] chicocvenancio: this made me thing otherwise: https://wikitech.wikimedia.org/wiki/Help:Horizon_FAQ#Accessing_Horizon [19:43:37] * chicocvenancio remembers accessing Horizon before being projectadmin... [19:44:16] he is able to log into wikitech.wikimedia.org with 2FA, but he's still not able to get into horizon. [19:45:49] same username and password? [19:51:29] yes [20:54:25] ragesoss: sorry I wandered away. :( We have had issues in the past with particular usernames and Horizon's auth layer. Most of these have been related to non-ascii characters in the Developer account name and python being goofy. A phab task would be a good next step and then maybe andrewbogott or I can find logs helping to figure out what is busted. [20:54:49] okay [20:55:06] there don't appear to be any non-ascii characters in the name. [20:55:18] I [20:55:26] I'll Phile it tomorrow. [20:59:10] ragesoss: the other thing that /sometimes/ messes up 2fa is if the clock on your phone is a little bit out of sync. Restarting a phone never hurts. [20:59:51] andrewbogott: I've hit that before. but would it work differently between wikitech and horizon? because he can log into wikitech fine with 2FA. [21:01:13] it could if they have different time limits. But it shouldn't make much of a difference [21:01:25] the 2fa validation is actually done on wikitech for Horizon [21:01:29] so it should really behave the same [21:01:44] It's also possible that one or the other site accepts shell name instead of wiki name, or the other way around [21:01:59] Or there's some difference in case sensitivity (I'm just guessing now) [21:02:17] he tried all variants he could think of between his shell name and wikitech username. [21:02:53] we can just work with me setting up the new instances, but we've got no obvious clues as to why login isn't working for him. [21:06:06] what username should I be looking for in the logs? [21:06:23] (advance warning: the logs are never helpful) [21:06:31] Nate Berkopec [21:10:59] ok, I see exactly one failed login in the logs [21:11:03] with no additional info [21:15:15] I can login on Horizon and use it normal, but when I click on "Identity" -> "Roles" I get logged out [21:16:23] mutante: I'd welcome a phab task about that, but in the meantime "don't do that then" :) [21:16:35] clicking some other things either works or tells me I am not authorized. just this one does an actual log out [21:17:27] sure, won't do it then [21:22:06] mutante: it used to be that basically anytime you hit an unauthorized UI Horizon would bounce you and make you re-login. I guess they haven't fixed that one yet... [21:22:42] I'll make a note for when I feel like working on more upstream things (although right now my other auth/policy upstream fixes are getting ignored) [21:22:46] andrewbogott: if it's supposed to be unauthorized, yea, ACK [21:23:01] expected it to just show me my own roles I guess [21:24:41] dont worry about it then, was just trying to check if there was a difference for me using different capitalization and stuff [21:24:45] it's for managing roles, although it should really have a read-only mode for non-admins [21:25:01] gotcha, thanks [21:25:11] Typically Horizon's approach to non-admins is to not know that there's such a thing as a non-admin :( [23:17:55] !log tools.bridgebot Restarting to add libera.chat bridge for #wikimedia-cloud [23:17:57] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.bridgebot/SAL [23:22:13] test from libera network [23:22:22] ack from freenode [23:24:30] why is there a _ in the nick? [23:26:25] !log tools.bridgebot Restart to add libera to #wmhack bridge [23:26:27] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.bridgebot/SAL [23:27:26] MacFan4000: because I forgot to disconnect from the client I was using to setup it's account. And the sfotware that bot uses is not awesome at nick recovery. [23:29:02] bd808: this might be confusing on telegram https://usercontent.irccloud-cdn.com/file/j1vWV7d6/image.png [23:29:18] https://t.me/wmcloudirc/22492 [23:30:22] chicocvenancio: It shows that the message came from outside telegram. On the other ends its the same. I dropped the remote id from the forwarded nick a while ago. [23:30:59] yeah, but when its a single source outside telegram we can be sure where its coming from [23:31:29] its been multi source for at least 6 months :) [23:31:48] it has? what are the other sources? [23:32:18] freenode<->telegram<->chat.wmcloud.org<->libera [23:33:10] I'd like to add a link into zulip too, and possibly discord [23:33:10] it's just that no one uses chat.wmcloud.org [23:35:54] my concern is impersonation attempts are less visible, but its probably not frequent enough that we should care [23:36:28] chicocvenancio: I'm totally willing to hear arguments about what would be "better". with the bot in the middle I'm not sure that we have identity verification anyway [23:37:39] there is a sketchy scripting language that I can use to do fancy things with the nick prefix as messages cross the bridges [23:37:50] It’s unclear how to register on chat.wmcloud.org [23:37:59] Doesn’t look like you can [23:38:04] you have to be invited [23:38:09] Ah [23:38:22] I can invite you if you want [23:38:35] Nah it’s fine [23:39:40] some info at https://diff.wikimedia.org/2020/09/18/introducing-wikimedia-chat/ [23:39:53] but it sounds like registration got closed to invite only? [23:40:12] yeah, without any real documentaion of that anywhere [23:41:20] To be fair to Amir1, the WMCS team gave him a lot of scary advice about running an open chat server [23:42:15] maybe somebody can figure out how to add-on OAuth for account creation during the hackathon :) [23:42:55] Warning: Mattermost is written in Golang [23:43:31] AntiComposite: doesn't stackoverflow cover that language? ;) [23:44:06] * bd808 programs by web search a lot of the time [23:53:36] golang is good actually ;)