[05:50:50] <_joe_> yes, I'm pretty much against having !log actions here, exactly because I think we need to make them as visible as possible to everyone [05:51:17] <_joe_> and no, the fact that people don't talk about anything in #-operations is not a good reason to spread !log out of it [05:51:51] <_joe_> lemme rephrase - as long as your alerts go to #-operations, so should your !log actions [10:05:37] after severe build tweaking it builds! it only took 4 hours [10:22:56] <_joe_> fsero: on your laptop? [10:23:10] <_joe_> if your laptop is a mac, I'm not surprised [10:23:20] <_joe_> given how shitty docker performance is on macs [10:27:05] Is not a mac [10:27:21] But I needed a VM so shitty performance [11:27:43] 10serviceops, 10Operations, 10Core Platform Team Backlog (Later), 10Patch-For-Review, 10Services (next): Migrate node-based services in production to node10 - https://phabricator.wikimedia.org/T210704 (10jijiki) [11:37:33] 10serviceops, 10Multimedia, 10Operations, 10Thumbor: Deploy 3d2png to thumbor servers (stretch) - https://phabricator.wikimedia.org/T216494 (10jijiki) p:05Triage→03Normal [11:38:03] 10serviceops, 10Operations, 10Thumbor, 10Patch-For-Review, 10User-jijiki: Thumbor upgrade to stretch plan - https://phabricator.wikimedia.org/T214597 (10jijiki) [11:38:10] 10serviceops, 10Operations, 10Thumbor, 10Patch-For-Review, and 2 others: Upgrade Thumbor servers to Stretch - https://phabricator.wikimedia.org/T170817 (10jijiki) [11:38:14] 10serviceops, 10Multimedia, 10Operations, 10Thumbor: Deploy 3d2png to thumbor servers (stretch) - https://phabricator.wikimedia.org/T216494 (10jijiki) [11:56:54] 10serviceops, 10Operations, 10Core Platform Team Backlog (Later), 10Patch-For-Review, 10Services (next): Migrate node-based services in production to node10 - https://phabricator.wikimedia.org/T210704 (10elukey) [12:16:12] 10serviceops, 10Gerrit, 10Operations: Gerrit loads very slowly - https://phabricator.wikimedia.org/T215855 (10hashar) From P8071 , it is often helpful to split by queues with `--by-queue` which would shows the number of worker threads and items for each queues. Example showing `ReplicateTo-slaves` and `WorkQ... [12:17:19] 10serviceops, 10Core Platform Team, 10Multimedia, 10Operations, 10Thumbor: Deploy 3d2png to thumbor servers (stretch) - https://phabricator.wikimedia.org/T216494 (10jijiki) [12:30:32] 10serviceops, 10Gerrit, 10Operations: Gerrit loads very slowly - https://phabricator.wikimedia.org/T215855 (10hashar) tools.ci first request was on 2019-02-12T01:12:07 . Maybe it is doing the requests to fast for Gerrit :/ Maybe https://gerrit.wikimedia.org/r/monitoring can give us some hint. Maybe we ca... [15:07:15] 10serviceops, 10Multimedia, 10Operations, 10Performance-Team, 10Thumbor: Deploy 3d2png to thumbor servers (stretch) - https://phabricator.wikimedia.org/T216494 (10kchapman) [15:30:57] 10serviceops, 10Multimedia, 10Operations, 10Performance-Team, 10Thumbor: Deploy 3d2png to thumbor servers (stretch) - https://phabricator.wikimedia.org/T216494 (10Jdforrester-WMF) [15:38:38] 10serviceops, 10Operations, 10ops-codfw, 10User-jijiki: Degraded RAID on thumbor2002 - https://phabricator.wikimedia.org/T214813 (10Papaul) The network cable was was plugged back in after the disk replacement. Should be good now. [15:40:33] 10serviceops, 10User-jijiki: Create dev and testing environments for scap2 - https://phabricator.wikimedia.org/T216518 (10jijiki) [15:42:17] 10serviceops, 10User-jijiki: Create dev and testing environments for scap2 - https://phabricator.wikimedia.org/T216518 (10jijiki) [15:42:21] 10serviceops, 10Scap, 10Release-Engineering-Team (Watching / External), 10User-jijiki: Allow scap sync to deploy gradually - https://phabricator.wikimedia.org/T212147 (10jijiki) [15:53:47] 10serviceops, 10Operations, 10Thumbor, 10Patch-For-Review, 10User-jijiki: Thumbor upgrade to stretch plan - https://phabricator.wikimedia.org/T214597 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by jiji on cumin2001.codfw.wmnet for hosts: ` ['thumbor2002.codfw.wmnet'] ` The log can be foun... [17:16:34] 10serviceops, 10Operations, 10Thumbor, 10Patch-For-Review, and 2 others: Upgrade Thumbor servers to Stretch - https://phabricator.wikimedia.org/T170817 (10jijiki) [17:16:37] 10serviceops, 10Operations, 10ops-codfw, 10User-jijiki: Degraded RAID on thumbor2002 - https://phabricator.wikimedia.org/T214813 (10jijiki) 05Open→03Resolved @Papaul Thank you, it works now:) [17:54:27] 10serviceops, 10MediaWiki-Cache, 10Operations, 10Patch-For-Review, and 3 others: Apply -R 200 to all the memcached mw object cache instances running in eqiad/codfw - https://phabricator.wikimedia.org/T208844 (10jijiki) [19:30:06] 10serviceops, 10Operations, 10Thumbor, 10Patch-For-Review, 10User-jijiki: Thumbor upgrade to stretch plan - https://phabricator.wikimedia.org/T214597 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['thumbor2002.codfw.wmnet'] ` Of which those **FAILED**: ` ['thumbor2002.codfw.wmnet'] ` [20:11:56] 10serviceops, 10Multimedia, 10Operations, 10Performance-Team, 10Thumbor: Deploy 3d2png to thumbor servers (stretch) - https://phabricator.wikimedia.org/T216494 (10jijiki) p:05Normal→03High [20:21:10] 10serviceops, 10Operations, 10Thumbor, 10Patch-For-Review, 10User-jijiki: Thumbor upgrade to stretch plan - https://phabricator.wikimedia.org/T214597 (10Gilles) [21:06:44] 10serviceops, 10Multimedia, 10Operations, 10Thumbor, 10Performance-Team (Radar): Deploy 3d2png to thumbor servers (stretch) - https://phabricator.wikimedia.org/T216494 (10kchapman)