[06:28:17] greetings [06:29:09] andrewbogott: on Debian I'm running everything unprivileged, what distro is that ? [07:39:49] morning [07:41:18] o/ [08:43:24] hello! [08:43:36] good morning [08:46:17] dhinus: do you have any other concerns before I deploy https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/997 in prod? [08:49:56] dcaro: no, I forgot to approve it but I'll do now! [08:50:13] ack thanks! [08:51:57] hmpf... the alloy pods don't restart when the config changes, I'll have to restart them one-by-one [08:57:02] are you sure alloy does not reload the running config when it changes? [08:58:10] the process itself you mean? let me check [08:59:31] there's no logs about to, so not sure xd [09:00:20] the drop rate is similar as before (no new spikes yet though) [09:01:01] the number of entries has spiked though, but might be for the few pods I restarted [09:01:28] anyhow, I'll restart them bit by bit just to make sure [09:05:24] yep, the spike seems to be the restart I did, it's going down already [09:27:19] quick lima-kilo fix for macos users: https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/283 [10:01:46] +1ed [10:42:54] quick review for lima-kilo, increasing number of open files (when logs-api added and loki around for some time, they hit the limit again) https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/284 [11:00:30] +1d [11:00:40] quick review for tofu-provisioning: https://gitlab.wikimedia.org/repos/cloud/toolforge/tofu-provisioning/-/merge_requests/93 [13:33:46] quick review for lima-kilo ldap ease, and reboot-reentry [13:33:47] https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/282 [14:17:53] fyi. finally restarted all the alloy pods, we have a bit more than doubled the amount of data we store (throughput wise), but went from dropping ~250logs/s to ~4, now it's a retention vs capacity game :) [14:18:47] nice! [14:27:17] dcaro: when I change something within the lima-kilo repo like playbooks/vars/kind.yaml is there an easier way to get it deployed locally than ./start-devenv.sh keeping the vm or recreating it? [14:28:01] you can run toolforge_run_ansible.sh from within lima-kilo (it accepts tags and tag ranges like `-t start:`) [14:28:20] or when you run `start_devenv.sh`, you can say no to deleting the VM and that will run ansible for everything though [14:28:53] ack, thx [17:48:00] Anyone around? Need a second pair of eyes to look at a cloud-vps quota increase request and + 1 https://phabricator.wikimedia.org/T407733 [18:08:53] Raymond_Ndibe: looks reasonable, I added my +1 in the task [18:09:03] me too xd [18:12:38] * dhinus off [18:16:56] * dcaro off