[00:11:30] 10serviceops, 10Recommendation-API, 10Release-Engineering-Team, 10Services, 10Patch-For-Review: Migrate recommendation-api to kubernetes - https://phabricator.wikimedia.org/T241230 (10bmansurov) @akosiaris thanks! The first two points were already done. I've created a chart and uploaded a patch. Would y... [07:49:55] 10serviceops, 10Operations, 10Performance-Team: Increased latency in CODFW API and APP monitoring urls (~07:20 UTC 19 Jan 2020) - https://phabricator.wikimedia.org/T243149 (10elukey) p:05Triage→03Normal [11:00:12] akosiaris: effie: today is a US holiday, no larger team meet today, and ma rk will not make the subteam meeting. do we want to have it anyways? are there topics you want to bring up? [11:05:20] I am ok with not doing the meeting [11:05:49] there is chance I will not be able to make it, and I will know around 18:00 our time [11:08:26] please do put in async meeting notes though :) [11:09:55] I am ok as well. There is also a chance I won't be able to make it anyway [11:09:58] _joe_: ^ [11:10:10] async then and we'll all fill in the notes [11:16:38] 10serviceops, 10Proton, 10Patch-For-Review, 10Product-Infrastructure-Team-Backlog (Kanban): Profile proton memory usage for Helm chart - https://phabricator.wikimedia.org/T238830 (10akosiaris) I 've rerun the benchmark against values of 1,2,3 for `num_workers` `num_workers: 1` diagrams {F31518010} {F315... [11:51:30] 10serviceops, 10Recommendation-API, 10Release-Engineering-Team, 10Services, 10Patch-For-Review: Migrate recommendation-api to kubernetes - https://phabricator.wikimedia.org/T241230 (10akosiaris) >>! In T241230#5815630, @bmansurov wrote: > @akosiaris thanks! The first two points were already done. I've c... [12:24:18] 10serviceops, 10Operations, 10ops-codfw: rack/setup/install new codfw mw systems - https://phabricator.wikimedia.org/T241852 (10jijiki) @Papaul @Joe Do you think it is ok if we rack everything else, have the new servers in rows ABD up and running, then decom some servers in C and rack the new ones? [12:41:42] 10serviceops, 10Operations, 10Parsoid-PHP, 10Wikimedia-production-error: wt2html: Out of memory crashers - https://phabricator.wikimedia.org/T236833 (10ssastry) [12:42:20] 10serviceops, 10Operations, 10Parsoid-PHP, 10Wikimedia-production-error: wt2html: Out of memory crashers - https://phabricator.wikimedia.org/T236833 (10ssastry) 05duplicate→03Open [12:42:48] 10serviceops, 10Operations, 10Parsoid-PHP, 10Wikimedia-production-error: wt2html: Out of memory crashers - https://phabricator.wikimedia.org/T236833 (10ssastry) [13:55:14] 10serviceops, 10Proton, 10Patch-For-Review, 10Product-Infrastructure-Team-Backlog (Kanban): Profile proton memory usage for Helm chart - https://phabricator.wikimedia.org/T238830 (10MSantos) It looks like `num_workers = 2` performed better if we consider failure vs RPS.