[00:17:09] <bstorm>	 !log admin set  --property hw_scsi_model=virtio-scsi and --property hw_disk_bus=scsi on the main stretch image in glance on eqiad1 T275430
[00:17:14] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[00:17:14] <stashbot>	 T275430: Large images cloned to /var/lib/nova/instances/_base filling up disk on hypervisors - https://phabricator.wikimedia.org/T275430
[01:04:33] <bstorm>	 !log tools hard rebooting tools-k8s-worker-76 because it's in a sorry state
[01:04:36] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[03:30:09] <Zppix>	 FYI: some bots may need restarted
[03:47:06] <Zppix>	 If anyone could point me to where I could either privately discuss a matter with a team member or email, that would be appreciated.
[04:34:41] <revi>	 looks like freenode doesn't like a shared black box IP
[04:35:11] <revi>	 (and essentially anyone who doesn't have SASL and start DMing instantly after connection)
[06:50:18] <legoktm>	 revi: wikibugs at least should never DM anyone
[06:50:54] <legoktm>	 or did you mean any bot on the shared IP?
[06:50:57] <revi>	 it turns out: there was a toolforge bot that sends DM, without nickserv auth
[06:51:02] <legoktm>	 ahhh
[06:51:10] <revi>	 and then it got hit by the freenode abusefilter
[06:51:17] <Zppix>	 yeah thats what i wanted to discuss with a WMCS team member privately about
[06:51:20] <legoktm>	 did we already track down which bot?
[06:51:24] <Zppix>	 yes
[06:51:32] <Zppix>	 swtimebot on toolforge
[06:51:33] <revi>	 yes but not gonna shame them publicly
[06:51:36] <revi>	 ouch.
[06:51:39] <legoktm>	 lol
[06:51:54] <Zppix>	 i mean it wouldnt take much to connect the dots
[06:52:01] <revi>	 true, but nonetheless
[06:52:01] <legoktm>	 has anyone reached out to the maintainer yet or should I?
[06:52:04] <Zppix>	 legoktm:  i have
[06:52:09] <legoktm>	 ok, thanks
[06:52:18] <Zppix>	 legoktm:  i've already talked to FN staff, waiting to talk to WMCS
[06:52:26] * revi also did
[06:52:41] <legoktm>	 is there something specific you want from WMCS?
[06:52:44] <revi>	 well, mostly fear of being SULwatcher k-lined again
[06:52:51] <Zppix>	 that i wont disclose publically
[06:52:58] <legoktm>	 I can disable the bot if you want right now if it's causing issues
[06:53:03] <Zppix>	 its not rn 
[06:53:09] <revi>	 I personally think I want to see a new policy requiring auth /meh
[06:53:32] <revi>	 maybe should be required from freenode side but still :-p
[06:54:32] <Zppix>	 i rather it be from WM's side 
[06:55:03] <Zppix>	 i thought at one point we were supposed to get identd 
[06:55:58] <revi>	 we can't kill the non-authed connection, freenode can
[06:55:59] <revi>	 so :-p
[10:54:39] <arturo>	 godog: cloudmetrics grafana just alerted. Is that something you were working on recently?
[10:55:08] <arturo>	 https://grafana-labs.wikimedia.org/ doesn't work
[10:58:57] <godog>	 arturo: I upgraded grafana the other day, but haven't done any operations recently
[10:59:05] <godog>	 recently == last 24h
[10:59:24] <arturo>	 godog: ok thanks
[11:07:10] <arturo>	 !log admin force-reboot cloudmetrics1002, add icinga downtime for 2 hours. Investigating some server issue
[11:07:13] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[12:50:29] <MacFan4000>	 bd808: after being accidentally k-lined twice, wm-bb is offline
[12:51:31] <Zppix>	 it just needs restarted
[16:09:47] <wm-bot>	 !log tools.bridgebot <bd808> Restarted following Freenode k-line
[16:09:49] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.bridgebot/SAL
[18:30:23] <bd808>	 !log tools `sudo wmcs-openstack role remove --user zfilipin --project tools user` T267313
[18:30:28] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[18:30:28] <stashbot>	 T267313: Off-board from Release Engineering - https://phabricator.wikimedia.org/T267313
[18:32:10] <bd808>	 !log bastion `sudo wmcs-openstack role remove --user zfilipin --project bastion user` T267313
[18:32:17] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Bastion/SAL
[20:30:35] <JJMC89[m]>	 Don't know the best place so that WMCS sees this, but https://phabricator.wikimedia.org/T275372#6858373 has feedback about tool deletion instructions / toolsadmin
[20:38:07] <SPF|Cloud>	 andrewbogott: hey, any update regarding https://phabricator.wikimedia.org/T127717?
[20:49:03] <andrewbogott>	 s
[20:49:20] <andrewbogott>	 SPF|Cloud: we talked about it some today, I think Brooke was going to follow up with some thoughts on the ticket.
[20:50:36] <bd808>	 JJMC89[m]: I made a Striker feature request ticket out of it. It's a good idea.
[21:07:24] <SPF|Cloud>	 andrewbogott: ack, thanks :)
[21:07:37] <SPF|Cloud>	 looking forward to the response
[21:51:21] <wm-bot>	 !log tools.quickcategories <lucaswerkmeister> deployed b2f83f22c0 (a11y fix)
[21:51:24] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.quickcategories/SAL