[10:29:00] 10serviceops, 10Prod-Kubernetes, 10Kubernetes, 10Patch-For-Review: Move helm chart repository out of git - https://phabricator.wikimedia.org/T253843 (10JMeybohm) [10:56:08] <_joe_> hnowlan: do you remember why restbase2009 is down? [10:57:27] _joe_: psu issue it looks like https://phabricator.wikimedia.org/T256863 [10:57:46] <_joe_> ack I asked you before searching phabricator [10:57:52] <_joe_> I hoped you'd remember [10:58:11] <_joe_> I didn't want you to browse phab for me :P [10:58:16] <_joe_> but thanks anyways :) [10:58:36] heh, had the tab open [11:10:00] https://kubeweekly.io -- thought it might be interesting to this audience [13:47:03] How far are we willing to push secrets/general.yaml in Kubernetes service dirs? Having stuff in there from puppet like the puppet_ca_cert in there is great, but is expanding that to other things puppet can tell services something we would consider? [13:47:52] context: I am looking for a nice way to reduce duplication in providing the lists of mw hosts to the API gateway and having them in secrets/general.yaml is one path even if that seems like kinda abusing the concept [13:54:15] hnowlan: shouldn't the API gateway just use the appservers.discovery.wmnet LVS service? [13:54:26] and api.discovery.wmnet and jobrunners.discovery.wmnet and so on? [13:54:43] akosiaris: yeah, probably! :) [13:54:52] :) [13:55:02] although I'd still be curious about your position on the above for academic purposes :D [13:56:49] Depends on the item in question. Overall yes, I 've been wanting to move some stuff in there as well [13:57:30] some things like e.g. url-downloader hostnames make utter sense to be in there and not shipped in the deployment-charts repo [13:57:39] s/hostnames/servicenames/ [13:58:47] generally, things that SRE tends to mess with, but not so much to warrant moving them to some etcd like functionality and need to be exposed as configuration to the services makes sense to be in there [14:32:47] <_joe_> so the answer is [14:33:05] <_joe_> we will just use the local proxy for everything but the api gateway :P [14:33:16] <_joe_> and we will use discovery hostnames as destinations [14:33:34] <_joe_> but my broader, philosophical response is we need an xDS data plane [14:34:59] <_joe_> and on the side of the puppet/k8s interaction - yes stuff we want to control centrally and could be derived by puppet should be stuffed, by puppet, in some yaml files on the deployment servers [14:39:48] 10serviceops, 10OTRS, 10Operations, 10User-notice: Update OTRS to the latest stable version (6.0.x) - https://phabricator.wikimedia.org/T187984 (10akosiaris) Below is a draft plan for the upgrade: [] Obtain a new, Debian Buster host (has already been done, otrs1001) [] Obtain a point in time snapshot of... [14:41:36] 10serviceops, 10LDAP-Access-Requests, 10Operations, 10observability, 10Patch-For-Review: Grant Access to Logstash to Peter(peter.ovchyn@speedandfunction.com) - https://phabricator.wikimedia.org/T249037 (10jcrespo) FYI @AMooney I don't see Peter on our shared records with legal. Sorry for the delay, but t... [14:52:38] 10serviceops, 10OTRS, 10Operations, 10User-notice: Update OTRS to the latest stable version (6.0.x) - https://phabricator.wikimedia.org/T187984 (10jcrespo) As this seems more snapshotting related than databases, I may take care myself of the db preparation needed. If db1077 is freed because of T256120#628... [14:58:47] FYI, we have 40ish codfw mw hosts with PHP 7.0 installed, I'll axe those to reduce the noise in debmonitor [15:11:45] 10serviceops, 10OTRS, 10Operations, 10User-notice: Update OTRS to the latest stable version (6.0.x) - https://phabricator.wikimedia.org/T187984 (10Marostegui) db1077 can be used yep [15:45:51] 10serviceops, 10OTRS, 10Operations, 10User-notice: Update OTRS to the latest stable version (6.0.x) - https://phabricator.wikimedia.org/T187984 (10akosiaris) >>! In T187984#6282101, @jcrespo wrote: > As this seems more snapshotting related than databases, I may take care myself of the db preparation needed... [16:10:20] _joe_: totally agree on the xDS plane, but for a stopgap I'm gonna do some static config to tide us over [16:10:30] what do you mean by local proxy in that sense? [16:11:13] <_joe_> hnowlan: I meant every k8s application will just have an envoy sidecar [16:11:20] ah right [16:18:09] 10serviceops, 10Prod-Kubernetes, 10Kubernetes, 10Patch-For-Review: Move helm chart repository out of git - https://phabricator.wikimedia.org/T253843 (10JMeybohm) [20:14:46] so the Envoy tests are failing on the new builder machine because it can't use IPv6 in the Docker container: "terminating with uncaught exception of type Envoy::Network::SocketBindException: cannot bind '[::1]:0': Cannot assign requested address" [20:15:15] not sure if the difference is that IPv6 was available on the old machine, or IPv6 tests were disabled -- looking [22:21:37] 10serviceops, 10LDAP-Access-Requests, 10Operations, 10observability, 10Patch-For-Review: Grant Access to Logstash to Peter(peter.ovchyn@speedandfunction.com) - https://phabricator.wikimedia.org/T249037 (10KFrancis) @amooney @jcrespo Hi all, I checked with Jim Buatti (Sr. Legal Counsel) to get an update....