[07:19:23] FIRING: SystemdUnitFailed: wmf_auto_restart_prometheus-mysqld-exporter.service on es2033:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [09:21:54] Hi, could I get a +1 to https://gerrit.wikimedia.org/r/c/operations/puppet/+/1196635 please? restore two nodes with new controllers to the rings, drain the final batch of 3 in codfw. [09:25:57] TY m.arostegui :) [12:16:07] Hello. I'd like to make sure that your team is aware of this: https://phabricator.wikimedia.org/T368098#11280973 - TL;DR The wikibase dumps: https://airflow-test-k8s.wikimedia.org/home?tags=dumps-wikibase will be using your mariadb servers again, for a bit. Hope that's OK with you. [15:22:23] @marostegui @Amir1 if you have a second for a very quick CR https://gerrit.wikimedia.org/r/c/operations/puppet/+/1196696 so I can run the host cloning overnight [15:24:39] federico3, Emperor o/ I noticed that you used cumin1002 recently, if possible I'd ask to migrate to cumin1003 because we'll likely nuke cumin1002 soon [15:24:58] lemme know if there are outstanding issues with 1003, so we'll try to fix them asap :) [15:26:11] elukey: I think I've cleared all my tmux sessions off there, it's just finger memory that means I sometimes hit 1002 not 1003. [15:26:36] perfect [15:27:33] elukey: As an Iranian, please let me when you want nuke anything. Thanks! [15:27:57] federico3: really soon, I take a look [15:29:18] it's just the the run-the-mill ES* provisioning puppet change [15:30:02] a.mir: increasing your xkeyscore points? :D [15:30:57] Always [15:40:03] +1ed the patch [15:40:34] tnx