[07:41:01] 10serviceops, 10Analytics, 10EventBus, 10Operations, and 2 others: eventgate-analytics k8s pods occasionally can't produce to kafka - https://phabricator.wikimedia.org/T218268 (10akosiaris) I can not rule out a networking issue but it seems improbable, as after all the logs did make it to logstash. Also th... [10:19:17] cdanis: so I have a look at whether we (can) have metrics for the number of ready pods. Currently the API servers does not expose through /metrics that information, but there is https://github.com/kubernetes/kube-state-metrics which does have that information per https://github.com/kubernetes/kube-state-metrics/blob/master/docs/pod-metrics.md [10:19:25] s/have/had/ [10:21:02] it's looks interesting, albeit somewhat new (first version was for kubernetes 1.9) [11:27:17] should i expet to get Phab notifications for all things tagged with "serviceops" ..without specifically subscribing to individual tickets? [11:27:26] * mutante checks if he is really a "member" [11:28:03] yes, i am, but it feels like i dont get them..hmm [11:32:37] ah, the difference between merely being a member or being a watcher [11:32:56] tbh, if membership means nothing for notifications .. does it mean anything [11:35:43] anyways, made sure to be a watcher now [11:42:41] 10serviceops, 10Scap: Scap2 to use etcd for target servers - https://phabricator.wikimedia.org/T218328 (10jijiki) [12:47:56] 10serviceops, 10ChangeProp, 10Release Pipeline, 10Core Platform Team Kanban (Doing), 10Services (doing): Make change-prop tests undefended of Kafka and Redis - https://phabricator.wikimedia.org/T218396 (10Pchelolo) [12:48:24] 10serviceops, 10ChangeProp, 10Release Pipeline, 10Core Platform Team Kanban (Doing), 10Services (doing): Make change-prop tests independent of Kafka and Redis - https://phabricator.wikimedia.org/T218396 (10Pchelolo) [12:50:30] oh interesting akosiaris, thanks for the pointer [12:51:21] it also points to https://github.com/kubernetes-incubator/metrics-server which is yet another different thing heh [12:51:31] (and new since k8s... 1.8 I think?) [13:03:29] yeah we were targetting that one at some point [13:03:34] but they fill different needs [13:04:17] metrics-server is about utilization metrics like CPU/mem (and pluggable metrics like req/s IIRC) [13:04:53] whereas the we hadn't targeted (did not even know) about the kube-state-metrics one [13:05:04] we should probably work to integrate both [13:28:36] 10serviceops, 10ChangeProp, 10Release Pipeline, 10Core Platform Team Kanban (Doing), 10Services (doing): Make change-prop tests independent of Kafka and Redis - https://phabricator.wikimedia.org/T218396 (10akosiaris) > is it possible to make the stage run an npm or bash command instead of relying on the... [16:48:16] 10serviceops, 10Scap: Define a mediawiki "version" - https://phabricator.wikimedia.org/T218412 (10jijiki) p:05Triage→03Normal [16:58:11] 10serviceops, 10Scap: Define a mediawiki "version" - https://phabricator.wikimedia.org/T218412 (10Dzahn) How about this: We calculate the sha1 sum of each file in /srv/mediawiki/php/cache/gitinfo and then the sha1 sum of the sum of these, like so: `sha1sum /srv/mediawiki/php/cache/gitinfo/* | sha1sum` I tri... [16:58:16] 10serviceops, 10Scap: Define a mediawiki "version" - https://phabricator.wikimedia.org/T218412 (10jijiki) [16:58:24] 10serviceops, 10Operations: Canaries canaries canaries - https://phabricator.wikimedia.org/T210143 (10jijiki) [16:58:48] jijiki: pasted an idea to check the mw "version" https://phabricator.wikimedia.org/T218412#5027492 it seems to work on mwdebug [16:59:24] tx mutante! [17:57:50] 10serviceops, 10Performance-Team, 10Wikimedia-Site-requests, 10Patch-For-Review: Enlarging the default thumb size on Dutch Wikipedia - https://phabricator.wikimedia.org/T215106 (10Jdforrester-WMF) Has this been signed off by SRE and Performance? The last times people wanted to change the default thumb size... [19:38:34] 10serviceops, 10Performance-Team, 10Wikimedia-Site-requests, 10Patch-For-Review: Enlarging the default thumb size on Dutch Wikipedia - https://phabricator.wikimedia.org/T215106 (10Krinkle) In wmf-config, `wgUploadThumbnailRenderMap` is currently set the same for all (public) wikis, as: `lang=php [ 320, 64... [19:50:54] 10serviceops, 10CX-cxserver, 10Citoid, 10Graphoid, and 10 others: Make services swagger specs standard compliant - https://phabricator.wikimedia.org/T218217 (10mobrovac) [20:30:43] 10serviceops, 10Performance-Team, 10Wikimedia-Site-requests, 10Patch-For-Review: Enlarging the default thumb size on Dutch Wikipedia - https://phabricator.wikimedia.org/T215106 (10Framawiki) >>! In T215106#5027521, @Mbch331 wrote: > Someone needs to give it a +2 on Gerrit so it gets merged Precisely no,...