[08:33:22] _joe_: hi, got a suggested change for tlsproxy::envoy to allow limiting the srange on hosts facing public https://gerrit.wikimedia.org/r/c/operations/puppet/+/591000 [08:34:51] <_joe_> mutante: I don't think that change is correct, and it would harm production at the moment [08:35:46] <_joe_> let me check [08:35:51] the default value is $PUBLIC though, as suggested for all sranges on https://phabricator.wikimedia.org/T149804 [08:35:56] thanks [08:36:23] <_joe_> $srange = undef, [08:36:26] <_joe_> is the default [08:40:17] <_joe_> so please let's keep that [08:40:19] _joe_: ack, amended to use undef [08:40:37] <_joe_> mutante: $PUBLIC is definitely not what we want here, too [08:42:36] <_joe_> also I don't find it in modules/base/templates/firewall/defs.erb [08:44:19] maybe it could be $DOMAIN_NETWORKS, but i turned it into undef, yea [08:49:56] <_joe_> mutante: your patch seems correct but it made me wonder [08:50:07] <_joe_> do we really need these hsots to have public IPs? [08:50:40] _joe_: i asked about that and the reason given is: 10:29 cause contint* hosts need to ssh to the WMCS instances and that has to be done over public IPs [08:50:43] <_joe_> if we reach the jenkins web interface via the caches [08:50:50] <_joe_> uh? [08:51:45] otherwise i use CACHES for this stuff when behind ATS, yea [09:04:59] <_joe_> which makes sense but be careful [09:05:11] <_joe_> some services might be called internally [09:07:14] ack [10:46:35] 10serviceops, 10Graphoid, 10Operations, 10Core Platform Team (Icebox): Undeploy graphoid - https://phabricator.wikimedia.org/T242855 (10Jseddon) a:03Seddon [13:56:44] 10serviceops, 10DC-Ops, 10Operations, 10ops-eqiad: scb1001: Memory correctable errors -EDAC- - https://phabricator.wikimedia.org/T250482 (10Cmjohnson) this is a good candidate for a replacement. [14:00:09] 10serviceops, 10DC-Ops, 10Operations, 10ops-eqiad: scb1001: Memory correctable errors -EDAC- - https://phabricator.wikimedia.org/T250482 (10MoritzMuehlenhoff) These servers will be removed from service soon (when the services currently running on it are completely moved to Kubernetes). @akosiaris can best... [15:41:42] 10serviceops, 10Operations: upgrade people.wikimedia.org backend to buster - https://phabricator.wikimedia.org/T247649 (10Dzahn) p:05Triage→03Medium [15:41:47] 10serviceops, 10Operations: replace bromine and vega with buster VMs - https://phabricator.wikimedia.org/T247650 (10Dzahn) p:05Triage→03Medium [15:59:08] I figured it out -- join https://phabricator.wikimedia.org/project/members/4114/ (serviceops-radar) which is a subproject [15:59:24] so that automatically joins you to the, uh, superproject [15:59:27] parent project I guess [16:13:37] rzl: yeah, that's a "feature" of Phabricator. Projects with sub-projects do not directly have members, instead membership is the union of members of the sub-projects. [16:15:28] a similarly funky by inverted thing happens for milestones. They do not have members, but instead inherit the members of their parent project [16:16:41] I think this is all documented in and around https://www.mediawiki.org/wiki/Phabricator/Project_management#Parent_Projects,_Subprojects_and_Milestones [18:40:02] 10serviceops, 10Deployments, 10Patch-For-Review, 10Performance-Team (Radar), and 3 others: Cache of wmf-config/InitialiseSettings often 1 step behind - https://phabricator.wikimedia.org/T236104 (10Jdforrester-WMF) OK, this is deployed, and I made a quick patch and used `SSH_AUTH_SOCK=/run/keyholder/proxy.s... [18:40:17] 10serviceops, 10Deployments, 10Patch-For-Review, 10Performance-Team (Radar), and 3 others: Cache of wmf-config/InitialiseSettings often 1 step behind - https://phabricator.wikimedia.org/T236104 (10Jdforrester-WMF) 05Open→03Resolved a:05Jdforrester-WMF→03Joe [18:40:24] 10serviceops, 10Core Platform Team, 10Performance-Team, 10Release-Engineering-Team-TODO, and 4 others: Define variant Wikimedia production config in compiled, static files - https://phabricator.wikimedia.org/T223602 (10Jdforrester-WMF) [18:40:51] bd808: makes sense, thanks! [20:57:42] 10serviceops, 10Operations: Renew certs for mcrouter on all application servers. - https://phabricator.wikimedia.org/T248093 (10RLazarus) The renewal script works as expected, but the procedure as written caused problems because not every mcrouter host is listed in `/etc/cergen/mcrouter.manifests.d/mediawiki-h...