[01:54:25] FIRING: [8x] PuppetConstantChange: Puppet performing a change on every puppet run on puppetmaster1001:9100 - https://puppetboard.wikimedia.org/nodes?status=changed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetConstantChange [03:11:05] FIRING: NetboxAccounting: Netbox - Accounting job failed - https://wikitech.wikimedia.org/wiki/Netbox#Report_Alert - https://netbox.wikimedia.org/core/jobs/ - https://alerts.wikimedia.org/?q=alertname%3DNetboxAccounting [04:11:04] RESOLVED: NetboxAccounting: Netbox - Accounting job failed - https://wikitech.wikimedia.org/wiki/Netbox#Report_Alert - https://netbox.wikimedia.org/core/jobs/ - https://alerts.wikimedia.org/?q=alertname%3DNetboxAccounting [05:54:25] FIRING: [8x] PuppetConstantChange: Puppet performing a change on every puppet run on puppetmaster1001:9100 - https://puppetboard.wikimedia.org/nodes?status=changed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetConstantChange [09:32:05] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, and 2 others: eqsin purged consumers lag - https://phabricator.wikimedia.org/T399221#11012447 (10cmooney) Arelion came back to say they no longer see CRC errrors on their side: ` Please note we are not detecting errors in our interface on Dalla... [09:54:25] FIRING: [8x] PuppetConstantChange: Puppet performing a change on every puppet run on puppetmaster1001:9100 - https://puppetboard.wikimedia.org/nodes?status=changed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetConstantChange [10:07:56] hi, I'm looking a bit at puppetdb - are there recommended ways to get a list of classes applied to hosts? E.g. I'm running pypuppetdb on a puppetdb host to fetch such data [10:40:50] 10SRE-tools, 06Infrastructure-Foundations, 10Spicerack, 10Data-Platform-SRE (2025.07.05 - 2025.07.25), 13Patch-For-Review: Proposal: adding a kafka admin client to spicerack - https://phabricator.wikimedia.org/T399069#11012714 (10brouberol) [10:40:51] 10SRE-tools, 06Infrastructure-Foundations, 10Spicerack, 10Data-Platform-SRE (2025.07.05 - 2025.07.25), 13Patch-For-Review: Proposal: adding a kafka admin client to spicerack - https://phabricator.wikimedia.org/T399069#11012715 (10brouberol) 05Open→03In progress [10:42:24] 10SRE-tools, 06Infrastructure-Foundations, 10Spicerack, 10Data-Platform-SRE (2025.07.05 - 2025.07.25), 13Patch-For-Review: Proposal: adding a kafka admin client to spicerack - https://phabricator.wikimedia.org/T399069#11012730 (10brouberol) a:03brouberol [13:54:25] FIRING: [8x] PuppetConstantChange: Puppet performing a change on every puppet run on puppetmaster1001:9100 - https://puppetboard.wikimedia.org/nodes?status=changed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetConstantChange [15:13:40] federico3: sorry for the late reply, not that I know of. Riccardo is afk today, he may know some good way forward, and maybe jhathaway as well. If you could give us a bit more info about your use case it would be helpful :) [15:15:45] no worries, I found a workaround in the meantime but the usecase is to get the current values of the roles applied in site.pp and the values from the hieradata/hosts/.yaml files [15:19:51] 10netbox, 06Infrastructure-Foundations: Improve Netbox "locations" use - https://phabricator.wikimedia.org/T333948#11013858 (10cmooney) Just to record we discussed this in both the Infra-Foundations meeting and between DC-Ops and netops recently and broadly thee were no objections. So I think we can continue... [15:42:45] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, and 2 others: eqsin purged consumers lag - https://phabricator.wikimedia.org/T399221#11013994 (10cmooney) Not sure how to progress this one. Still see zero packet loss over the link, even running for a longer period (5 mins this time): ` cmoon... [15:45:48] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, and 2 others: eqsin purged consumers lag - https://phabricator.wikimedia.org/T399221#11014023 (10ssingh) > Arelion want to close the ticket as they see no issue. I asked that they don't. Perhaps for now we just leave eqsin depooled and the ci... [15:56:47] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, and 2 others: eqsin purged consumers lag - https://phabricator.wikimedia.org/T399221#11014121 (10cmooney) >>! In T399221#11014023, @ssingh wrote: > I think leaving eqsin depooled given that it is off-peak there and observing this for a few hour... [17:54:25] FIRING: [8x] PuppetConstantChange: Puppet performing a change on every puppet run on puppetmaster1001:9100 - https://puppetboard.wikimedia.org/nodes?status=changed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetConstantChange [21:54:25] FIRING: [8x] PuppetConstantChange: Puppet performing a change on every puppet run on puppetmaster1001:9100 - https://puppetboard.wikimedia.org/nodes?status=changed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetConstantChange