[05:03:44] <jinxer-wm>	 FIRING: LiftWingServiceErrorRate: ...
[05:03:44] <jinxer-wm>	 LiftWing service has a high rate of non 2/3/400 error code responses - https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing/Alerts#LiftWingServiceErrorRate - https://grafana.wikimedia.org/d/G7yj84Vnk/istio?orgId=1&refresh=30s&var-cluster=codfw%20prometheus/k8s-mlserve&var-namespace=revertrisk&var-backend=revertrisk-language-agnostic-predictor-default.%2A - https://alerts.wikimedia.org/?q=alertname%3DLiftWingServiceErrorRate
[06:44:00] <elukey>	 INFO:root:Unsupported lang: www.
[06:44:01] <elukey>	 ERROR:root:An error has occurred while fetching revisions:  [2378529203] (www). Reason: Cannot connect to host en.wikipedia.org:443 ssl:default [None]
[06:44:23] <elukey>	 so yeah non-well-formed requests
[06:44:48] <elukey>	 ideally we should return HTTP 400 in this case though
[09:03:59] <jinxer-wm>	 FIRING: LiftWingServiceErrorRate: ...
[09:03:59] <jinxer-wm>	 LiftWing service has a high rate of non 2/3/400 error code responses - https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing/Alerts#LiftWingServiceErrorRate - https://grafana.wikimedia.org/d/G7yj84Vnk/istio?orgId=1&refresh=30s&var-cluster=codfw%20prometheus/k8s-mlserve&var-namespace=revertrisk&var-backend=revertrisk-language-agnostic-predictor-default.%2A - https://alerts.wikimedia.org/?q=alertname%3DLiftWingServiceErrorRate
[13:03:59] <jinxer-wm>	 FIRING: LiftWingServiceErrorRate: ...
[13:03:59] <jinxer-wm>	 LiftWing service has a high rate of non 2/3/400 error code responses - https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing/Alerts#LiftWingServiceErrorRate - https://grafana.wikimedia.org/d/G7yj84Vnk/istio?orgId=1&refresh=30s&var-cluster=codfw%20prometheus/k8s-mlserve&var-namespace=revertrisk&var-backend=revertrisk-language-agnostic-predictor-default.%2A - https://alerts.wikimedia.org/?q=alertname%3DLiftWingServiceErrorRate
[13:25:42] <gry>	 wtf
[13:52:24] <wikibugs>	 (03CR) 10CI reject: [V:04-1] build: Updating mediawiki/mediawiki-phan-config to 0.16.0 [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1168437 (owner: 10Libraryupgrader)
[17:03:59] <jinxer-wm>	 FIRING: LiftWingServiceErrorRate: ...
[17:03:59] <jinxer-wm>	 LiftWing service has a high rate of non 2/3/400 error code responses - https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing/Alerts#LiftWingServiceErrorRate - https://grafana.wikimedia.org/d/G7yj84Vnk/istio?orgId=1&refresh=30s&var-cluster=codfw%20prometheus/k8s-mlserve&var-namespace=revertrisk&var-backend=revertrisk-language-agnostic-predictor-default.%2A - https://alerts.wikimedia.org/?q=alertname%3DLiftWingServiceErrorRate
[18:43:44] <jinxer-wm>	 RESOLVED: LiftWingServiceErrorRate: ...
[18:43:44] <jinxer-wm>	 LiftWing service has a high rate of non 2/3/400 error code responses - https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing/Alerts#LiftWingServiceErrorRate - https://grafana.wikimedia.org/d/G7yj84Vnk/istio?orgId=1&refresh=30s&var-cluster=codfw%20prometheus/k8s-mlserve&var-namespace=revertrisk&var-backend=revertrisk-language-agnostic-predictor-default.%2A - https://alerts.wikimedia.org/?q=alertname%3DLiftWingServiceErrorRate
[19:44:16] <wikibugs>	 (03PS2) 10Umherirrender: build: Updating mediawiki/mediawiki-phan-config to 0.16.0 [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1168437 (owner: 10Libraryupgrader)