[06:50:33] <_joe_> greg-g: more or less, yes :) [09:36:11] I am merging a patch for firejail [09:36:22] (we run thumbor under firejail) [09:36:36] is it ok if I do a rolling restart of the thumbor services? [09:47:56] moritzm godog ^ [09:53:09] jijiki: +1 [09:53:25] tx [10:03:27] sounds good! [13:55:40] 10serviceops, 10Prod-Kubernetes, 10Kubernetes: etcd1004-1006 is unused and idle, use the cluster or kill it. - https://phabricator.wikimedia.org/T212934 (10fselles) [14:14:54] 10serviceops: SRE FY2019 Q3 goal: Increase reach of deployment pipeline - https://phabricator.wikimedia.org/T212935 (10fselles) [14:15:26] cc @all goal taskhttps://phabricator.wikimedia.org/T212935 [14:15:29] https://phabricator.wikimedia.org/T212935 [14:30:54] 10serviceops, 10Operations: SRE FY2019 Q3 goal: Increase reach of deployment pipeline - https://phabricator.wikimedia.org/T212935 (10fselles) [14:40:11] <_joe_> thanks fsero! [15:06:47] 10serviceops, 10Operations, 10Thumbor, 10User-jijiki: Investigate systemd hardening to replace Firejail for Thumbor - https://phabricator.wikimedia.org/T212941 (10jijiki) p:05Triage→03Normal [15:45:18] 10serviceops, 10Operations, 10Thumbor, 10User-jijiki: Stream Thumbor logs to logstash - https://phabricator.wikimedia.org/T212946 (10jijiki) [15:45:34] 10serviceops, 10Operations, 10Thumbor, 10User-jijiki: Stream Thumbor logs to logstash - https://phabricator.wikimedia.org/T212946 (10jijiki) [17:22:39] 10serviceops, 10Operations, 10Thumbor, 10Wikimedia-Logstash, 10User-jijiki: Stream Thumbor logs to logstash - https://phabricator.wikimedia.org/T212946 (10herron) p:05Triage→03Normal Sure! In terms of figuring out how, since the logs are already present in syslog this should be a pretty straightforwa... [17:24:48] 10serviceops, 10Operations: SRE FY2019 Q3 goal: Increase reach of deployment pipeline - https://phabricator.wikimedia.org/T212935 (10herron) p:05Triage→03Normal [17:32:55] 10serviceops, 10Operations, 10Release-Engineering-Team (Watching / External): Increase mwdebugXXXX hosts CPU and memory - https://phabricator.wikimedia.org/T212955 (10greg) [18:14:48] 10serviceops, 10Operations, 10Release-Engineering-Team (Watching / External): Increase mwdebugXXXX hosts CPU and memory - https://phabricator.wikimedia.org/T212955 (10hashar) /proc/cpuinfo reports a single cpu core. Though it is unknown whether uses multiple cores when generate the bytecode cache, it is wor... [18:19:56] 10serviceops, 10Operations, 10Release-Engineering-Team (Watching / External): Increase mwdebugXXXX hosts CPU and memory - https://phabricator.wikimedia.org/T212955 (10hashar) IIRC the cdb files are generated by `rebuildLocalisationCache.php` which is CPU bounds and run with up to 30 parallel tasks. I previo... [18:21:14] 10serviceops, 10Operations, 10Release-Engineering-Team (Watching / External): Increase mwdebugXXXX hosts CPU and memory(?) - https://phabricator.wikimedia.org/T212955 (10greg) [18:56:48] trying to install npm on stretch with puppet..(for testreduce/parsoid testers). until recently that wasn't possible because npm was dropped from stretch. now, as reported by Moritz, we have npm in stretch-backports. but due to dependency errors also gotta use apt::pin and pin about 30 (!sic) node-something packages for that to work. wondering if i should just do "node-*" , it seems wildcards [18:56:54] are fine with apt::pin.. looking at toollabs doing that [19:51:50] this might be better https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/482150/3/modules/testreduce/manifests/init.pp [19:53:13] tries it on the host scandium (stretch, has been sitting there with role:spare for quite some time because of the missing npm) .. which will replace ruthenium (jessie) then for real [19:53:50] refs: https://phabricator.wikimedia.org/T201366 ..be back in a while