[00:13:42] !log tools.whois restarted tool to fix a brief error caused during bug testing of the Gridengine system [00:13:43] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.whois/SAL [02:15:08] !log tools.eranbot Restarted web service and enwiki job. T242291. [02:15:11] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.eranbot/SAL [06:33:31] !log library-upgrader creating upgrader-06 instance, large flavor this time [06:33:33] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Library-upgrader/SAL [06:53:21] !log library-upgrader disabling all services on upgrader-05 [06:53:23] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Library-upgrader/SAL [06:59:02] !log library-upgrader upgrader06: enabling buster-wikimedia/thirdparty/kubeadm-k8s apt repo and installing docker-ce [06:59:04] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Library-upgrader/SAL [06:59:49] !log library-upgrader upgrader06: adduser libup && adduser libup docker [06:59:49] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Library-upgrader/SAL [07:52:40] !log library-upgrader switched over libraryupgrader2.wmflabs.org to point to upgrader06 [07:52:42] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Library-upgrader/SAL [09:34:48] !log library-upgrader shutdown upgrader-05 instance [09:34:49] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Library-upgrader/SAL [09:46:11] !log cyberbot moving cyberbot-db-01 from cloudvirt1009 to cloudvirt1029 to try a faster hypervisor [09:46:13] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Cyberbot/SAL [10:53:53] how does the new kubernetes cluster (https://wikitech.wikimedia.org/wiki/News/2020_Kubernetes_cluster_migration) relate to the toolforge.org domain? [10:54:09] I’m not very interested in the new cluster by itself, but I am very interested in having per-tool subdomains [10:54:27] (right now *.toolforge.org seems to be equivalent to tools.wmflabs.org, which doesn’t seem especially useful) [10:55:59] Lucas_WMDE: the new domain is something we will try to introduce this Q. The new cluster has been crafted in a way that should allow us to easily introduce the new domain [10:57:01] ok [10:58:26] thanks! [10:59:00] let us know how it goes when you try the new k8s cluster [11:44:15] hello, where can I find instructions on how to scp between two instances in the Wikimedia Cloud Services? [11:45:32] Epantaleo: in general, we don't have mechanisms for doing that, because you would need to go through a SSH bastion first. However, I have an alternative: [11:46:38] try something like `ssh myinstance.eqiad.wmflabs 'cat myfile' | ssh myinstance2.eqiad.wmflabs 'cat > myfile'` in your laptop [11:47:10] you could also transfer them via the bastion [11:47:48] yes, you could do something like that in the bastion as well [11:48:53] is it ok to do it for large files (> 1Gb)? [11:49:32] the example I pasted doesn't use local storage in the machine running the original ssh command! [11:49:38] so, yes it should be fine Epantaleo [11:53:36] I run the command inside the `` tags, and got: Host key verification failed [12:01:26] arturo: I get "Host verification failed" when running that comand on my computer [12:02:23] can you paste your command? [12:03:06] oh ok I run it including the `` tags and worked... thanks [12:08:13] the cat redirection in the second command may be in the opposite direction, < [13:04:41] !log cyberbot cyberbot-db-01 is now on cloudvirt1029 [13:04:42] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Cyberbot/SAL [13:27:24] !log admin cloudvirt1009: virsh undefine i-000069b6. This is tools-elastic-01 which is running on cloudvirt1008 (so, leaked on cloudvirt1009) [13:27:26] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [14:56:18] arturo: was the migration successful? [15:03:02] Cyberpower678: looks like it was, it's active on the new hypervisor [15:12:58] jeh: and it looks good now. [15:30:37] !log tools git stash-ing local puppet changes in hopes that arturo has that material locally, and it doesn't break anything to do so [15:30:40] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [15:35:10] !log tools depooling and rebooting tools-worker-1016 because it still had the leftover mount problems [15:35:12] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [15:45:49] !log tools depooled tools-paws-worker-1013 to reboot because I think it is the last tools server with that mount issue (I hope) [15:45:52] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [16:15:54] !log tools.ftl restarted the tool to clean up loads of uninterruptible and zombie perl processes [16:15:56] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.ftl/SAL [20:14:41] hi all, Brian in Berkeley with OSGeo dot org.. I want to say "congratulations" on the announce of "2020 Kubernetes cluster" [20:15:21] I copied the announcement to OSGeo SAC channel, for more exposure there [21:05:52] !log toolsbeta updated toollabs-webservice package to 0.55 for testing [21:05:54] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [23:31:33] !log tools updated toollabs-webservice package to 0.56 [23:31:35] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL