[12:41:31] 10serviceops, 10Operations: SRE FY2019 Q4 goal: complete the transition to PHP7 - https://phabricator.wikimedia.org/T219127 (10Joe) [12:46:47] 10serviceops, 10Operations: SRE FY2019 Q4 goal: complete the transition to PHP7 - https://phabricator.wikimedia.org/T219127 (10Joe) [12:49:56] 10serviceops, 10Operations: SRE FY2019 Q4 goal: complete the transition to PHP7 - https://phabricator.wikimedia.org/T219127 (10Joe) [12:50:37] 10serviceops, 10Operations: SRE FY2019 Q4 goal: complete the transition to PHP7 - https://phabricator.wikimedia.org/T219127 (10Joe) [12:50:44] 10serviceops, 10MediaWiki-General-or-Unknown, 10Operations, 10Core Platform Team (PHP7 (TEC4)), and 4 others: Some pages will become completely unreachable after PHP7 update due to Unicode changes - https://phabricator.wikimedia.org/T219279 (10Joe) [12:51:12] 10serviceops, 10Operations: SRE FY2019 Q4 goal: complete the transition to PHP7 - https://phabricator.wikimedia.org/T219127 (10Joe) [13:48:39] fsero: o/ [13:48:39] did you get to look at what might be wrong with tiller? [13:48:50] yes [13:48:58] \o [13:49:11] i was using an script for initializing namespaces that failed and didnt set up tiller [13:49:13] is done now [13:49:19] so you should be able to deploy [13:49:23] k great! trying now... [13:50:34] hmmmm [13:50:35] Error: Get https://10.64.76.1:443/version: x509: cannot validate certificate for 10.64.76.1 because it doesn't contain any IP SANs [13:54:43] fsero: ^ [13:55:29] <_joe_> well [13:55:41] <_joe_> what is 10.64.76.1? a pod? [13:58:28] i dunno, i can't find it in dns or in puppet [13:58:44] brb, restarting compy [14:00:52] 10serviceops, 10Operations: SRE FY2019 Q4 goal: complete the transition to PHP7 - https://phabricator.wikimedia.org/T219127 (10Krinkle) [14:01:53] staging? [14:01:57] yup [14:02:45] CLUSTER=staging scap-helm eventgate-main install -n main -f main/staging-values.yaml stable/eventgate [14:03:55] 10.64.76.1 is the kubernetes API in cluster IP [14:05:05] 10serviceops, 10Operations, 10Services, 10service-runner: Re-evaluate service-runner's (ab) of statsd timing metric for nodejs GC stats - https://phabricator.wikimedia.org/T222795 (10akosiaris) [14:05:13] 10serviceops, 10Operations, 10Services, 10service-runner: Re-evaluate service-runner's (ab) of statsd timing metric for nodejs GC stats - https://phabricator.wikimedia.org/T222795 (10akosiaris) p:05Triage→03Low [14:09:51] ottomata: i know what it is [14:09:57] let me some mins and ill fix it [14:10:01] k thanks! [14:13:25] akosiaris: you around? [14:13:37] who/what sets KUBERNETES_SERVICE_HOST label on tiller deploys? [14:14:09] im going to ammend that manually for now but i guess is nto done manually [14:19:26] fsero: yup, that's correct [14:19:55] we should get rid of that when we get DNS working [14:20:19] ottomata: fixed [14:20:22] please proceed [14:20:44] also the namespace creation part of helmfile [14:21:04] 10serviceops, 10Operations: SRE FY2019 Q4 goal: complete the transition to PHP7 - https://phabricator.wikimedia.org/T219127 (10Krinkle) [14:22:14] 10serviceops, 10Core Platform Team Kanban, 10MediaWiki-General-or-Unknown, 10Operations, and 4 others: Some pages will become completely unreachable after PHP7 update due to Unicode changes - https://phabricator.wikimedia.org/T219279 (10kchapman) CPT needs to review regarding long term fixes for this. [14:23:25] fsero: https://gerrit.wikimedia.org/r/#/c/operations/deployment-charts/+/492269/ [14:23:38] maybe we could use this in the meantime [14:23:49] thanks is working! got an app related problem to fix, but the k8s stuff is working now, thank you! [14:52:00] 10serviceops, 10Operations, 10Phabricator, 10Patch-For-Review, 10Release-Engineering-Team (Watching / External): Reimage both phab1001 and phab2001 to stretch - https://phabricator.wikimedia.org/T190568 (10Dzahn) [15:01:30] 10serviceops, 10Operations, 10Services, 10service-runner: Re-evaluate service-runner's (ab)use of statsd timing metric for nodejs GC stats - https://phabricator.wikimedia.org/T222795 (10akosiaris) [15:04:49] 10serviceops, 10Core Platform Team Kanban, 10MediaWiki-General-or-Unknown, 10Operations, and 4 others: Some pages will become completely unreachable after PHP7 update due to Unicode changes - https://phabricator.wikimedia.org/T219279 (10Anomie) The long-term plan, as I understand it, is that we'll run main... [16:51:51] 10serviceops, 10Beta-Cluster-Infrastructure, 10Release Pipeline, 10Core Platform Team Backlog (Next), and 2 others: Migrate Beta cluster services to use Kubernetes - https://phabricator.wikimedia.org/T220235 (10thcipriani) [17:16:36] 10serviceops, 10Gerrit, 10Operations, 10cloud-services-team: Change /r/p/ to /r/ on all hosts (where https://gerrit.wikimedia.org/r/p/ exists) - https://phabricator.wikimedia.org/T222093 (10Paladox) [19:54:31] 10serviceops, 10Operations, 10Core Platform Team Backlog (Watching / External), 10Patch-For-Review, and 2 others: Use PHP7 to run all async jobs - https://phabricator.wikimedia.org/T219148 (10jijiki) It looks like deployment-prep has an older php7.2 version than production, which is something we should fix... [23:13:57] 10serviceops, 10Gerrit, 10Operations, 10cloud-services-team: Change /r/p/ to /r/ on all hosts (where https://gerrit.wikimedia.org/r/p/ exists) - https://phabricator.wikimedia.org/T222093 (10Paladox) [23:25:55] 10serviceops, 10MediaWiki-Cache, 10Operations, 10Core Platform Team (Security, stability, performance and scalability (TEC1)), and 5 others: Use a multi-dc aware store for ObjectCache's MainStash if needed. - https://phabricator.wikimedia.org/T212129 (10Catrope)