[00:23:08] [[Tech]]; ~2025-31563-08; /* I need to create a website on MediaWiki */ new section; https://meta.wikimedia.org/w/index.php?diff=29713962&oldid=29712023&rcid=37611631 [01:00:11] [[Tech]]; BDavis (WMF); /* Noto fonts for a tool */ Reply; https://meta.wikimedia.org/w/index.php?diff=29714113&oldid=29713962&rcid=37611856 [02:35:02] [[Tech]]; Sean.hoyland; /* Noto fonts for a tool */; https://meta.wikimedia.org/w/index.php?diff=29714617&oldid=29714113&rcid=37612464 [07:58:43] [[Tech]]; Sean.hoyland; /* Noto fonts for a tool */; https://meta.wikimedia.org/w/index.php?diff=29715116&oldid=29714617&rcid=37613404 [09:30:35] [[Tech]]; AKlapper (WMF); /* I need to create a website on MediaWiki */ Reply; https://meta.wikimedia.org/w/index.php?diff=29715505&oldid=29715116&rcid=37614036 [21:16:58] volans: Hi volans! :) How are ya. Pardon for ping. [21:17:56] I'm a new dev, not as good as you and the rest of team and aspiring to! :-D Without further ado, I am working on the T250415 [21:17:57] T250415: Homer: add parallelization support - https://phabricator.wikimedia.org/T250415 [21:18:26] However I ran into some setbacks and was curious about testing tips if it works well :D [21:19:17] I am using https://docs.python.org/3/library/concurrent.futures.html , however, I am using ThreadPoolExecutor. Is that bad? [21:19:45] My approach was to create a separate parallel execution uhh "module" let's call it like that [21:20:40] Hook it up to same logger and uhh yeah have each __init__ run get assigned it's own worker and.. Yep. [21:20:57] Ran into some issues and figured it might be best to ask the master behind the homer project :D [21:21:01] Thanks! <3 [21:21:35] FWIW, it's very likely otuside his working hours [21:23:05] Reedy: Whoops sorry. Do you have any advice, please? Much appreciated. Thank you :D [21:23:18] (in regards to the outside working hours part, not homer) [21:23:28] Wait for his working hours? Post on the task? :P [21:23:38] I am not err.. certain about the etiquette how do I put this haha [21:24:01] Reedy: O, waiting for working hours sounds the best :D Task I don't wanna spam since it's nothing very beneficial atm.. [21:24:13] I could throw a WIP on Gerrit too but, why.. I'd just spam it.. It's not fully working [21:24:39] It's one way to get testing run in our infrastructure though [21:24:42] Plus he might see code and say: "yeah this is really bad way of doing it" and uhh yeah [21:25:00] Best way to get an answer on the internet is to post the wrong answer [21:25:17] Reedy: Yes, true.. Would be nice to see netbox too, but I read you need NDA and stuff for that, got serious really quick haha [21:25:56] Getting NDA'd for netbox (etc) isn't a simple process, understandably [21:27:38] Reedy: exactly :p so I wanted to ask volans about advice what to test locally to see if homer works well [21:29:58] like.. the pytest stuff does pass but uhh [21:30:20] yeeah the thing is that i am not very very well versed into actual wide spectrum network config [21:30:29] but i want to learn :-) [21:30:33] If it's not documented (well) how to test locally etc, asking for someone to do that would be a reasonable one [21:31:08] Reedy: Yeah I read all about it on wikitech and on the doc.wikimedia.org/homer [21:31:15] there is some info but I was still not 100% certain [21:31:35] it's not really meant to hold your hand and expects a well versed dev understandably [21:31:59] but alas it's hard only the first time till you learn and i really want to [21:38:53] duude__: learning good :D and we like volunteers! [21:39:19] yay Raine :D thanks [21:39:27] I assume vo.lans will pop up to answer once he's around, so you can just hang around here until then [21:39:44] * volans feels summoned :D [21:39:48] go away! [21:39:49] :D [21:39:49] :O [21:40:11] is it party in the house \o/ [21:40:13] well I'll finish typing anyway :D [21:40:27] o7 [21:40:27] hi duude__, nice to meet you. Thanks for your interest in homer, there is a lot to unpack here. [21:40:45] volans: I'm not your manager, but are you sure you shouldn't do that tomorrow? :D [21:41:28] duude__: I just wanted to finish saying that posting on the task or gerrit isn't spamming when you actually want to help :-) [21:41:48] volans: hello :D thank you nice to meet you too i love italy and tagitelle and now homer :D [21:41:56] lol [21:42:01] tagliatelle* whoops [21:42:49] I'll split my answers to the various topics, lmk if I'm just adding confusion instead of clarity [21:44:09] * Raine may or may not try to re-spill it somewhere more permanent than IRC once it's spilled and soaked up [21:44:35] Raine: Neat, thanks Raine. Yeah I saw on Gerrit that people sometimes open WIP submissions, which can be nicely reviewed and if it's all neat-o after fixing (if needeed) and checks out then merged [21:44:53] Though I'd rather submit something that's already solidly ready to go than uhh WIP and then it gets forgotten lol [21:45:15] Ofc could always on phabricator write "hey my WIP is done can someone test" but uhh yeah Ideally I'd test it fully locally and then say "i'm pretty sure this is finished can someone confirm" :D [21:45:25] [the feature itself] since we've developed the grouping of approval/rejection of a diff running homer is way less annoying because it doesn't need continue interaction. Nevertheless it's still slow and parallelization would be useful, but that's why the priority was moved to "low". [21:47:58] [implementation] as mentioned in https://phabricator.wikimedia.org/T250415#9820426 and https://phabricator.wikimedia.org/T250415#11263494 there is some prior investigation to make sure that the chosen approach will actually work without issues. There wasn't a clear easy winner. If we had already Python 3.14 I'll probably investigate first the InterpreterPoolExecutor but that's probably too far in [21:48:04] the future in terms of debian releases. ... [21:48:06] ... The threadpool approach could work, I'm only worried about some dependency not being thread-safe. [21:49:49] [implementation steps] the whole thing can easily be split into two parts, one that parallelize the generation of the config and the other that parallelize the transport that applies the changes to the network devices [21:50:06] volans: what do you think would it be wise if threadpool approach is chosen and dependencies check out as thread-safe to have it anyway under another branch? [21:51:46] I don't think a separate branch is needed, with gerrit it's wasy to work without additional branches on the server side. I always use a dedicated local branch on my dev env though. [21:52:18] [netbox] you don't need netbox for this, in particular for the transport part. Homer should still work fine without netbox just with yaml files. [21:53:48] [testing] I think that Cathal (commented on the same task) might be able give you some useful advice, he has a local lab of virtual devices where he runs homer IIRC and I don't think it's too complex to create. [21:53:50] Same :D dedi local branch always. True for netbox. Thanks volans. I checked out bullseye and trixie backports atm, no 3.14. Everything still on 3.13.. Rip. [21:54:01] I mean I saw that bookworm is used somewhere too soo.. lel [21:54:26] whoops changed em up [21:54:48] bookworm* -> bullseye* and vice versa :D [21:55:01] yeah, I think that debian forky will have 3.14 or even 3.15 but will be out in 2027 probably [21:55:08] so I wouldn't wait for it :) [21:55:14] :O [21:55:24] at that point i'd just build python 3.14 :D [21:55:45] we don't have to support older version, we can control which OS install so if it was trixie it would be an option [21:55:55] i was always curious why is that not done more often in general in the sre world tbh [21:56:07] esp now that 3.14 adds some more cool stuff like native zstd [21:56:50] oki I will continue then with threadpoolexecutor thank you volans :D [21:56:53] depends where things run, but often stability is more important than bleeding edge :D [21:57:15] volans: true but tbh if you build it yourself.. should be pretty stable if it all passes nicely [21:57:56] but alas when you're doing 100 different things per day, no really time to build it unless it's absolutely necessary [21:58:06] stable is stable and no need to test :D [21:58:12] [wmf internals] I'm currently working on a differnet team and I'm not in charge of the development of homer nor I think I will work on it in the nearby future. So commenting on the task would also let the Infrastructure Foundations team be aware of your intention to work on this [21:58:15] this you have to test which is yet another task.. which is.. not ideal :D [21:58:28] yeah [21:58:40] duude__: hey :) [21:58:52] yes homer can work without Netbox at all just fine [21:58:57] volans: gotcha, thanks a lot for everything volans much appreciated <3 [21:58:59] in this case you define everything in YAML files [21:59:02] topranks: hi!! :D [21:59:04] * duude__ waves hello [21:59:11] how are you doing this evening ^^ [21:59:27] so this is an example repo of where I used homer before without netbox: [21:59:28] https://github.com/topranks/homerlabs/tree/main/labs/evpnlab/homer_public [21:59:36] topranks: nice, nice. do we have any random yaml's perchance for testing :P [21:59:43] i did see one in docs [21:59:57] I'll write more myself but uhh yeah just worried to miss something [22:00:03] alas prob a stupid worry of mine. [22:00:08] yeah the above link has them [22:00:20] woo! let's go. thank you very much :3 [22:00:39] really the only thing to be mindful of is you need to have "devices.yaml", it will look for this file, and also potentially "sites.yaml" and "common.yaml" [22:03:01] basically there is a template directory here which has jinja2 templates for Juniper router configs [22:03:13] and a 'config' directory which has the yaml vars that get used by those templates [22:19:38] gotcha gothca tyvm <3 will message back here tonight or tomorrow if I run into any major obstacles that I can't tackle :D thank you volans and topranks much appreciated <3 [22:19:47] np [22:25:22] * Raine just realised that, as they are in the same timezone as that person with ears everywhere, telling him to go away was perhaps a bit hypocritical [22:37:47] * Raine has the usual amount of self-awareness tonight