[08:34:36] 10serviceops, 10Machine Learning Platform, 10ORES, 10Operations: ORES redis: max number of clients reached... - https://phabricator.wikimedia.org/T263910 (10jijiki) [12:51:56] 10serviceops, 10Machine Learning Platform, 10ORES, 10Operations: ORES redis: max number of clients reached... - https://phabricator.wikimedia.org/T263910 (10CDanis) "busy web workers" graph, which correlates quite well with the slowdown: https://grafana.wikimedia.org/d/HIRrxQ6mk/ores?viewPanel=13&orgId=1&f... [18:06:38] 10serviceops, 10Machine Learning Platform, 10ORES, 10Operations: ORES redis: max number of clients reached... - https://phabricator.wikimedia.org/T263910 (10calbon) every time it starts at 16:00 [18:10:34] 10serviceops, 10Machine Learning Platform, 10ORES, 10Operations: ORES redis: max number of clients reached... - https://phabricator.wikimedia.org/T263910 (10calbon) {F32364643} [19:41:56] 10serviceops, 10Machine Learning Platform, 10ORES, 10Operations: ORES redis: max number of clients reached... - https://phabricator.wikimedia.org/T263910 (10calbon) It started happening again, I went into each Ores200X box and manually `sudo service uwsgi-ores restart` to restart it.{F32364681}