[17:48:42] mutante: pcc uses the puppet cert name. Some older VMs still have .eqiad.wmflabs certnames, new ones all have eqiad1.wikimedia.cloud [17:48:50] so it depends entirely on when the VM was built [17:51:27] andrewbogott: yea, I was aware of that. that's how I got around the issue until recently. the new part was though that both names were now 404 in compiler [17:51:57] at least for that example in the integration project [17:52:33] mutante: oh, maybe facter needs refreshing? [17:52:35] I just merged it and confirmed noop on the actual machine [17:52:53] ok — if you want to open a task I can investigate later, otherwise we can ignore and see if it comes back :) [17:52:54] yea, maybe it will be gone after the next fact sync [17:53:05] but something seems to have changed somewhere..or timed out [17:53:30] andrewbogott: thank you. I will say.. i create the ticket if I run into it a second time [17:53:39] sounds good! [17:53:40] ignore it for now, but I will find out soon enough [17:58:24] !log tools hard rebooting tools-sgecron-01 following network issues during upgrade to stein T261134 [17:58:28] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [17:58:28] T261134: upgrade cloud-vps openstack to Openstack version 'Stein' - https://phabricator.wikimedia.org/T261134 [18:00:26] oh good [18:00:48] I was about to complain my crontab monitor was complaining that the job is not running [18:06:08] :) [18:08:03] thanks! (back to bed) [20:40:44] !log tools running tc-setup across the grid fleet [20:40:47] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [20:43:43] !log tools running tc-setup across the k8s workers [20:43:45] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [20:56:47] !log tools setting bastions to have mostly-uncapped egress network and 40MBps nfs_read for better shared use [20:56:50] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL