[00:37:34] <jinxer-wm>	 RESOLVED: DiskSpace: Disk space mwlog2002:9100:/srv 3.521% free - https://wikitech.wikimedia.org/wiki/Monitoring/Disk_space - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&viewPanel=12&var-server=mwlog2002 - https://alerts.wikimedia.org/?q=alertname%3DDiskSpace
[01:04:34] <jinxer-wm>	 FIRING: DiskSpace: Disk space mwlog2002:9100:/srv 3.964% free - https://wikitech.wikimedia.org/wiki/Monitoring/Disk_space - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&viewPanel=12&var-server=mwlog2002 - https://alerts.wikimedia.org/?q=alertname%3DDiskSpace
[01:24:34] <jinxer-wm>	 RESOLVED: DiskSpace: Disk space mwlog2002:9100:/srv 3.849% free - https://wikitech.wikimedia.org/wiki/Monitoring/Disk_space - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&viewPanel=12&var-server=mwlog2002 - https://alerts.wikimedia.org/?q=alertname%3DDiskSpace
[06:34:34] <jinxer-wm>	 FIRING: DiskSpace: Disk space titan2001:9100:/srv 0.8006% free - https://wikitech.wikimedia.org/wiki/Monitoring/Disk_space - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&viewPanel=12&var-server=titan2001 - https://alerts.wikimedia.org/?q=alertname%3DDiskSpace
[06:39:34] <jinxer-wm>	 RESOLVED: DiskSpace: Disk space titan2001:9100:/srv 0.009923% free - https://wikitech.wikimedia.org/wiki/Monitoring/Disk_space - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&viewPanel=12&var-server=titan2001 - https://alerts.wikimedia.org/?q=alertname%3DDiskSpace
[08:00:12] <jinxer-wm>	 FIRING: ThanosCompactHalted: Thanos Compact has failed to run and is now halted. - https://wikitech.wikimedia.org/wiki/Thanos#Alerts - https://grafana.wikimedia.org/d/651943d05a8123e32867b4673963f42b/thanos-compact - https://alerts.wikimedia.org/?q=alertname%3DThanosCompactHalted
[09:06:30] <tappof>	 ^^looking
[09:10:12] <jinxer-wm>	 RESOLVED: ThanosCompactHalted: Thanos Compact has failed to run and is now halted. - https://wikitech.wikimedia.org/wiki/Thanos#Alerts - https://grafana.wikimedia.org/d/651943d05a8123e32867b4673963f42b/thanos-compact - https://alerts.wikimedia.org/?q=alertname%3DThanosCompactHalted
[09:10:25] <jinxer-wm>	 FIRING: SystemdUnitFailed: thanos-compact.service on titan2001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[09:11:01] <jinxer-wm>	 FIRING: ThanosCompactIsDown: Thanos component has disappeared. - https://wikitech.wikimedia.org/wiki/Thanos#Alerts - https://grafana.wikimedia.org/d/0cb8830a6e957978796729870f560cda/thanos-overview - https://alerts.wikimedia.org/?q=alertname%3DThanosCompactIsDown
[09:15:25] <jinxer-wm>	 RESOLVED: SystemdUnitFailed: thanos-compact.service on titan2001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[09:16:01] <jinxer-wm>	 RESOLVED: ThanosCompactIsDown: Thanos component has disappeared. - https://wikitech.wikimedia.org/wiki/Thanos#Alerts - https://grafana.wikimedia.org/d/0cb8830a6e957978796729870f560cda/thanos-overview - https://alerts.wikimedia.org/?q=alertname%3DThanosCompactIsDown
[13:15:38] <jinxer-wm>	 FIRING: StatsvThroughput: StatsV is not ingesting metrics - https://wikitech.wikimedia.org/wiki/Performance.wikimedia.org/Runbook#statsv - https://grafana.wikimedia.org/d/ba06cb37-dfab-40ae-8e61-0710522881e0/statsv - https://alerts.wikimedia.org/?q=alertname%3DStatsvThroughput
[13:20:38] <jinxer-wm>	 RESOLVED: StatsvThroughput: StatsV is not ingesting metrics - https://wikitech.wikimedia.org/wiki/Performance.wikimedia.org/Runbook#statsv - https://grafana.wikimedia.org/d/ba06cb37-dfab-40ae-8e61-0710522881e0/statsv - https://alerts.wikimedia.org/?q=alertname%3DStatsvThroughput