[04:27:35] I am going to switch s2 primary [04:55:03] jynus_: My next section to migrate to mariadb 10.11 is up to you, from my side, s5 would be ideal, but I can adapt to whatever is easier for you and backups [09:40:59] Amir1: I see you merged https://gerrit.wikimedia.org/r/c/operations/puppet/+/1163846, shall I run update-views on clouddbs? [09:41:22] dhinus: if you have time. It should be basically noop [09:41:52] sure, I was off a couple days but I have time today [09:42:00] Thanks! [09:42:03] I'll let you know if I see anything unexpected [09:42:13] I cataloged wikilove_log table too, so I removed it [09:42:25] ah, wait a sec. I need to make a tiny change [09:42:30] ok [09:45:15] https://gerrit.wikimedia.org/r/c/operations/puppet/+/1167159 [09:45:25] let me merge this now and then we can continue [09:45:55] jynus_: Did you see my previous comment? Maybe I see you have a "_" on your nick so maybe your bouncer is acting funny [09:48:04] dhinus: good to go now. Thanks! [09:49:55] Amir1: ack [09:51:58] s5 is ok [09:55:03] jynus: thanks [11:36:44] btullis: there are some alerts on -operations generated by your maintenance I guess, can you confirm? [11:37:24] Ah, sorry. Yes, that was me. Rebooted db1208 and got distracted. Will sort it now. Thanks. [12:21:56] Amir1: Please place all those new tasks in their appropiate dashboard column, the triage column is getting a bit out of hand [12:22:03] Let's try to keep it tidy [12:22:14] sure! [12:22:33] Thank you! [12:27:41] Amir1: what do we want to do for the remaining hosts in https://phabricator.wikimedia.org/T393990 ? [14:12:26] I rebooted all the apus nodes today, and have now added docs on the process: https://wikitech.wikimedia.org/wiki/Ceph/Cephadm#Rebooting [15:04:25] FIRING: SystemdUnitFailed: wmf_auto_restart_prometheus-mysqld-exporter@x1.service on db2201:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [15:05:11] checking [15:07:35] should be fixed now [15:09:25] RESOLVED: SystemdUnitFailed: wmf_auto_restart_prometheus-mysqld-exporter@x1.service on db2201:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed