[05:04:38] <wm-bot>	 !log tools.eranbot <samwilson> Ran restart_plagiabot.sh T242291
[05:04:41] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.eranbot/SAL
[05:14:49] <wm-bot>	 !log tools.eranbot <samwilson> Killed and restarted enwiki job 2924844. T242291.
[05:14:55] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.eranbot/SAL
[09:47:48] <Epantaleo>	 hello, I removed a VPS instance from the cloud (project etytree) but its resources haven't been freed I think. THus I cannot create a new instance. 
[09:48:09] <arturo>	 Epantaleo: looking
[09:48:18] <Epantaleo>	 thanks
[09:49:00] <arturo>	 Epantaleo: which quota in concrete? RAM, instances, vCPU...
[09:49:48] <Epantaleo>	 bigram, 8cpus i believe
[09:50:13] <Epantaleo>	 36GB of ram
[09:50:29] <Epantaleo>	 1 instance called etytree-a
[10:32:00] <arturo>	 Epantaleo: I tried a thing. Could you please check if quota looks any different?
[10:32:18] <arturo>	 !log admin ran `root@cloudcontrol1004:~# nova-manage project quota_usage_refresh --project etytree`
[10:32:19] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[10:37:26] <Epantaleo>	 arturo: I still have two instances running, while i should have q
[10:37:27] <Epantaleo>	 1
[10:39:25] <arturo>	 I see
[11:09:25] <arturo>	 Epantaleo: I just created T242332
[11:09:25] <stashbot>	 T242332: CloudVPS: wrong nova quota usage for a project detected - https://phabricator.wikimedia.org/T242332
[11:11:20] <arturo>	 !log admin running `MariaDB [nova_eqiad1]> select * from quota_usages where project_id = 'etytree';` (T242332)
[11:11:22] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[11:11:37] <arturo>	 sorry, wrong c&p
[11:12:11] <arturo>	 !log admin running `MariaDB [nova_eqiad1]> update quota_usages set in_use='0' where project_id='etytree';` (T242332)
[11:12:13] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[11:12:37] <arturo>	 Epantaleo: could you please try again?
[11:18:27] <arturo>	 !log etytree created VM 'arturo-test' to check T242332
[11:18:30] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Etytree/SAL
[11:18:30] <stashbot>	 T242332: CloudVPS: wrong nova quota usage for a project detected - https://phabricator.wikimedia.org/T242332
[11:19:24] <arturo>	 !log etytree deleted VM 'arturo-test' (T242332)
[11:19:25] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Etytree/SAL
[11:26:27] <Epantaleo>	 arturo: problem solved, thanks
[11:27:48] <arturo>	 _+1:
[11:27:54] <arturo>	 👍*
[14:28:59] <chicocvenancio>	 !log paws shutdown unused instances
[14:29:00] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL
[14:54:35] <ironie>	 100-200 Mo for personal Mysql DB on toolserver, it's ok ? I didn't read any note in the documentation about the size
[14:56:20] <Reedy>	 Shouldn't be a problem
[14:56:27] <Reedy>	 Also, toolserver hasn't existed for years ;)
[15:26:41] <Epantaleo>	 hello I  am having problems installing openssl 0.9.8e on the cloud - debian-10.0
[15:30:26] <Reedy>	 isn't that a really old version of openssl?
[15:30:43] <bd808>	 Epantaleo: what sort of problems? apt-get install failing with a message or something else?
[15:31:04] <bd808>	 oh yeah, I'm not sure if Buster has 0.9.x
[15:31:24] <Reedy>	 https://packages.debian.org/buster/openssl is 1.1.1d
[15:31:34] <Epantaleo>	 I need an old version of openssl that i don't seem to be able to get with apt get
[15:31:58] <Epantaleo>	 and when I try from the openssl repo I get errors again
[15:32:09] <Reedy>	 Why do you need an old version?
[15:32:26] <Epantaleo>	 it's needed to install virtuoso opensource
[15:32:37] <Epantaleo>	 no higher version is compatible
[15:32:56] <Epantaleo>	 I had no problems on the previous virtual machine
[15:33:41] <bd808>	 the Jessie vm was using packages that were ~5 years older :)
[15:33:45] <Reedy>	 https://github.com/openlink/virtuoso-opensource/blob/develop/7/README.OpenSSL.md
[15:33:52] <Reedy>	 >Virtuoso can be compiled against OpenSSL 0.9.8i upto OpenSSL v1.0.2p
[15:35:14] <Epantaleo>	 I  installed 1.0.2p
[15:35:26] <Epantaleo>	 but somehow it says I don't have openssl
[15:35:46] <Reedy>	 path issues?
[15:36:41] <Epantaleo>	 I followed instructions here
[15:36:44] <Epantaleo>	 https://www.howtoforge.com/tutorial/how-to-install-openssl-from-source-on-linux/
[15:36:59] <Epantaleo>	 for ubuntu
[15:37:52] <Reedy>	 https://www.howtoforge.com/tutorial/how-to-install-openssl-from-source-on-linux/#configure-openssl-binary would seemingly be the relevant part
[15:38:13] <Epantaleo>	 did that
[15:38:42] <Epantaleo>	 which openssl returns  1.0.2p
[15:38:56] <Reedy>	 FWIW, it seems to be packaged for debian... https://packages.debian.org/buster/virtuoso-opensource
[15:39:05] <Reedy>	 >Install this metapackage for the full suite of packages that make up Virtuoso OSE ("Open-Source Edition").
[15:39:40] <Epantaleo>	 is virtuoso 7 available?
[15:40:38] <Reedy>	 It is in experimental - https://packages.debian.org/experimental/database/virtuoso-opensource-7
[15:40:54] <Epantaleo>	 hmmm
[15:41:07] <Reedy>	 and https://packages.debian.org/experimental/database/virtuoso-opensource
[15:42:15] <Epantaleo>	 I'll try and see what happens thanks
[15:46:14] <Epantaleo>	 can you point me to the exact download link 
[15:46:18] <Epantaleo>	 thanks!
[15:47:21] <Epantaleo>	 does this look right https://packages.debian.org/experimental/amd64/virtuoso-opensource-7/download  ?
[15:47:51] <Epantaleo>	 and then also this: https://packages.debian.org/experimental/all/virtuoso-opensource/download  ?
[17:02:05] <Cyberpower678>	 bd808: just pinging you on the status of getting cyberbot-db-01 moved.  No pressure.
[17:02:23] <Cyberpower678>	 and/or arturo andrewbogott
[17:02:27] <arturo>	 moved to other cloudvirt?
[17:02:31] <Cyberpower678>	 Yes
[17:02:59] <arturo>	 you say ping because this has been already discussed?
[17:03:34] <Cyberpower678>	 bd808 said he will try to get a faster hypervisor but it will be at least several days because of more urgent issues.
[17:04:21] <Cyberpower678>	 It's fine if it can't be done at this time, just wondering where we're at about it/
[17:04:41] <arturo>	 I can handle that
[17:05:27] <arturo>	 Cyberpower678: what is your timezone? do you mind if I do it tomorrow in my EU morning?
[17:05:36] <Cyberpower678>	 I'm EST
[17:05:46] <arturo>	 what is that in UTC?
[17:05:57] <Cyberpower678>	 And I actually prefer that.  There's a process crawling to completion and I want to let it finish first.
[17:06:12] <Cyberpower678>	 arturo: I believe that's UTC-5
[17:06:20] <arturo>	 ok!
[17:06:29] <Cyberpower678>	 I assum you are +1
[17:06:34] <arturo>	 yup
[17:06:45] <Cyberpower678>	 Germany?
[17:06:49] <arturo>	 spain
[17:06:53] <Cyberpower678>	 Oh.
[17:07:18] <Cyberpower678>	 With WMDE being so large and widely known, I just had to guess. :-)
[17:07:42] <arturo>	 fair, I work for the WMF though
[17:09:15] <arturo>	 I'm looking for candidate destinations
[17:09:20] <arturo>	 cloudvirt1014 might be a good one
[17:09:38] <arturo>	 a bit CPU oversubscribed, but not a lot of load otherwise
[17:10:02] <Cyberpower678>	 cyberbot-db-01 is not a CPU hog.  It just needs a fast disk.
[17:10:27] <Cyberpower678>	 Average usage is about 8%
[17:10:36] <arturo>	 cloudvirt1018 may also be a good candidate
[17:12:21] <arturo>	 cloudvirt1029 has an average of disk IO of 0.8% for the last 90 days. Good load avg. That's the good one!
[17:13:47] <arturo>	 ok, will stick a post-it to my laptop to do that tomorrow during my morning!
[17:14:52] <Cyberpower678>	 Great thank you. :-)
[17:17:04] <Cyberpower678>	 arturo: That just leaves resolving the quota issue of the VM, but there's still roughly 2 months to figure that out. :-)
[17:46:28] <bstorm_>	 !log tools refreshing the paws cluster's entire x509 environment T242353
[17:46:31] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[17:46:31] <stashbot>	 T242353: Create new k8s certificate for PAWS cluster - https://phabricator.wikimedia.org/T242353
[18:06:24] <bstorm_>	 !log tools rebooting tools-paws-master-01 T242353
[18:06:27] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[18:06:27] <stashbot>	 T242353: Create new k8s certificate for PAWS cluster - https://phabricator.wikimedia.org/T242353
[18:06:33] <bstorm_>	 !log paws rebooting tools-paws-master-01 T242353
[18:06:34] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL
[18:25:36] <bstorm_>	 !log tools re-joining the k8s nodes to the cluster one at a time to rotate the certs T242353
[18:25:39] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[18:25:40] <stashbot>	 T242353: Create new k8s certificate for PAWS cluster - https://phabricator.wikimedia.org/T242353
[18:26:01] <bstorm_>	 !log tools re-joining the k8s nodes OF THE PAWS CLUSTER to the cluster one at a time to rotate the certs T242353
[18:26:03] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[22:06:11] <bd808>	 Toolforge has a new Kubernetes cluster that is ready for more beta tester/early adopters! -- https://lists.wikimedia.org/pipermail/cloud-announce/2020-January/000247.html
[23:35:05] <bstorm_>	 !log tools depooled tools-sgeexec-0939 because it isn't acting right and rebooting it
[23:35:07] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[23:52:40] <bstorm_>	 !log tools.fountain restarted webservice because it didn't cleanly depool during toolforge maintenance T242385
[23:52:42] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.fountain/SAL
[23:52:42] <stashbot>	 T242385: "503 Service Unavailable" error on Fountain tool - https://phabricator.wikimedia.org/T242385