[10:51:26] <arturo>	 !log toolsbeta rebalance load: reallocating toolsbeta-sgewebgrid-lighttpd-0901 from cloudvirt1018 to cloudvirt1003
[10:51:28] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL
[10:53:07] <arturo>	 !log shinken rebalance load: reallocating shinken-02 from cloudvirt1018 to cloudvirt1006
[10:53:08] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Shinken/SAL
[10:53:53] <arturo>	 !log testlabs rebalance load: reallocating abogott-puuppetmaster from cloudvirt1018 to cloudvirt1003
[10:53:54] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Testlabs/SAL
[10:54:31] <arturo>	 !log puppet rebalance load: reallocating puppet-jmm-kernel-jessie from cloudvirt1018 to cloudvirt1003
[10:54:32] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Puppet/SAL
[10:55:00] <arturo>	 !log k8splay rebalance load: reallocating k8s-dzhan from cloudvirt1018 to cloudvirt1003
[10:55:00] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:K8splay/SAL
[10:56:25] <arturo>	 !log wikifactmine rebalance load: reallocating puppetmaster-01 from cloudvirt1018 to cloudvirt1006
[10:56:25] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikifactmine/SAL
[10:57:11] <arturo>	 !log testlabs rebalance load: reallocating util-abogott-jessie from cloudvirt1018 to cloudvirt1006
[10:57:12] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Testlabs/SAL
[10:57:45] <arturo>	 !log webperf rebalance load: reallocating disposable from cloudvirt1018 to cloudvirt1006
[10:57:46] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Webperf/SAL
[11:09:54] <Cyberpower678>	 andrewbogott: every since the VM update, my CPU usage has skyrocketed and I have waits of 32000%
[11:10:45] <arturo>	 Cyberpower678: which VM?
[11:10:54] <Cyberpower678>	 cyberbot-db-01
[11:10:59] <arturo>	 andrew is not around today
[11:12:26] <Cyberpower678>	 arturo: ^^
[11:13:00] * hauskatze gives arturo a glass of Manzanilla
[11:14:29] <arturo>	 Cyberpower678: I could try reallocating your VM to other hypervisor later. It's running on cloudvirt1018
[11:14:34] <arturo>	 which is a bit overloaded right now
[11:14:39] <arturo>	 https://grafana.wikimedia.org/d/aJgffPPmz/wmcs-openstack-eqiad1-hypervisor?orgId=1&var-hypervisor=cloudvirt1018&refresh=30s&from=now-7d&to=now
[11:18:35] <Cyberpower678>	 arturo: standard routine requests to the DB results in essentially a 100% CPU spike.  It's horrible.
[11:19:05] <arturo>	 hauskatze: o/
[11:19:14] <arturo>	 Cyberpower678: thats probably due to iowait
[11:19:18] <Cyberpower678>	 It used to just hang at 10% utilization constantly.
[11:19:43] <Cyberpower678>	 It's severely slowing IABot down unfortunately.
[11:20:09] <arturo>	 cloudvirt1018 has unhealthy RAID which may affect IO as well
[11:21:02] <Cyberpower678>	 :|
[11:21:30] <arturo>	 if you software is doing lots of IO Cyberpower678 may I ask for a stop until I can reschedule the VMs elsewhere?
[11:22:13] <Cyberpower678>	 That would mean shutting down IABot completely.  
[11:22:24] <Cyberpower678>	 As well as the tools depending on it
[11:22:42] <Cyberpower678>	 IABot doesn't do that much IO.
[11:22:50] <arturo>	 ok
[11:32:17] <arturo>	 !log integration depool integration-slave-docker-1040 in preparation for reallocating
[11:32:19] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Integration/SAL
[12:15:14] <arturo>	 !log integration rebalance load, reallocating integration-slave-docker-1040 from cloudvirt1018 to cloudvirt1026
[12:15:15] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Integration/SAL
[12:28:34] <arturo>	 Cyberpower678: so, let me know what you want me to do finally, if reallocate your VMs or now
[12:28:36] <arturo>	 or not*
[14:13:57] <Wurgl>	 jlocal is removed? Is there any alternative?
[14:51:15] <zhuyifei1999_>	 Wurgl: I don't think jlocal is removed. I just checked that it exists on tools-sgecron-01
[14:52:27] <Wurgl>	 tools.persondata@tools-sgebastion-07:~$ jlocal
[14:52:33] <Wurgl>	 -bash: jlocal: command not found
[14:55:44] <Wurgl>	 zhuyifei1999_: Have you seen?
[14:58:13] <jeh>	 looks like jlocal is only available on the sgecron host, installed with https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/production/modules/profile/manifests/toolforge/grid/cronrunner.pp#29
[15:00:22] <Wurgl>	 Okay.
[15:00:46] <Wurgl>	 so it seems to work from within crontab
[15:01:56] <jeh>	 I think that's the intended use, https://wikitech.wikimedia.org/wiki/Help:Toolforge/Grid#Creating_a_crontab
[15:03:01] * jeh is still learning about all the WMF commands
[15:12:45] <Wurgl>	 Debian with WMF is WMFian?
[15:18:23] <zhuyifei1999_>	 (sorry, was away)
[15:20:08] <zhuyifei1999_>	 jeh: yeah exactly :P
[15:21:16] <zhuyifei1999_>	 jlocal is just a wrapper to exec whatever command that follows. what would be the point on the bastion.
[23:12:52] <SMalyshev>	 anybody still around on this Friday afternoon? I need a bump in VCPU quota for my project wikidata-query...
[23:34:26] <legoktm>	 SMalyshev: I think you're supposed to file a bug for that
[23:34:35] <legoktm>	 which...you did :)
[23:39:11] <SMalyshev>	 yep :)