[02:19:36] <bd808>	 !log tools.docker-registry Hard restart of webservice to pick up redirect logic (T242719)
[02:19:38] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.docker-registry/SAL
[02:19:39] <stashbot>	 T242719: https://tools.wmflabs.org/{toolname} no longer redirects to https://tools.wmflabs.org/{toolname}/ on new k8s cluster - https://phabricator.wikimedia.org/T242719
[03:58:24] <DSquirrelGM>	 why do I get the feeling I ended up being the trigger for that user talk page edit on wikitech :|
[04:06:10] <DSquirrelGM>	 anyway, to the topic I was going to ask about - Is there any way to delete duplicate or unused tool records created from toolsadmin and shown there and in hay's tool listing, and if so, where are they actually stored? Need to do some cleanup on https://toolsadmin.wikimedia.org/tools/id/squirrelnest-upf and https://tools.wmflabs.org/admin/tools
[04:08:22] <bd808>	 DSquirrelGM: T198114
[04:08:22] <stashbot>	 T198114: Allow tool maintainers to delete toolinfo via ToolsAdmin - https://phabricator.wikimedia.org/T198114
[04:08:23] <DSquirrelGM>	 tried adding the toolinfo.json file to override that, but it doesn't seem to have had any effect
[04:09:56] <DSquirrelGM>	 so - not implemented yet then?
[07:46:08] <Erutuon>	 so I've got a phantom directory in templatehoard that refuses to be unlinked because it refuses to admit it's a directory. it seems to be a hard link to another directory that I created by mistake.
[07:47:42] <hare>	 zhuyifei1999_: have you ever thought about scaling up video2commons?
[07:47:56] <Erutuon>	 the path is ~/www/static/dump/20200120/20200201/; the same directory is found at ~/www/static/dump/20200201/, the correct location
[07:48:02] <zhuyifei1999_>	 hare: scale to?
[07:48:21] <zhuyifei1999_>	 Erutuon: toolforge?
[07:48:28] <zhuyifei1999_>	 which tool?
[07:48:30] <Erutuon>	 yes, the templatehoard tool
[07:48:50] <hare>	 zhuyifei1999_: just in general, increasing its capacity. i would be happy to help with that
[07:49:10] <zhuyifei1999_>	 it uses three transcoding instances to serve the requests
[07:49:40] <hare>	 how much CPU?
[07:50:13] <zhuyifei1999_>	 every instance has 16 cores
[07:50:30] <zhuyifei1999_>	 I usually have one instance on standby in case Ineed to do maintenance
[07:50:45] <zhuyifei1999_>	 https://tools.wmflabs.org/nagf/?project=video
[07:51:17] <zhuyifei1999_>	 Erutuon: I see a symlink
[07:51:20] <zhuyifei1999_>	 07:49:46 0 ✓ zhuyifei1999@tools-sgebastion-08: ~$ ls ~tools.templatehoard/www/static/dump/20200120/20200201 -ld
[07:51:20] <zhuyifei1999_>	 lrwxrwxrwx 1 tools.templatehoard tools.templatehoard 80 Feb  3 07:31 /data/project/templatehoard/www/static/dump/20200120/20200201 -> /mnt/nfs/labstore-secondary-tools-project/templatehoard/www/static/dump/20200201
[07:51:38] <zhuyifei1999_>	 you can rm that symlink
[07:51:48] <hare>	 zhuyifei1999_: I take it you are not in need of more cores?
[07:52:38] <zhuyifei1999_>	 hare: as far as I can tell, no
[07:52:55] <Erutuon>	 zhuyifei1999_: thanks, done.
[07:52:59] <zhuyifei1999_>	 are you seeing some significant latency or something?
[07:53:11] <hare>	 nah, I'm just contemplating future projects
[07:53:24] <Erutuon>	 not sure how I got confused there
[07:53:32] <hare>	 growing the computational capacity of the wikimedia movement is a goal of mine
[07:53:44] <zhuyifei1999_>	 I see
[07:54:14] <hare>	 Another idea is to rent out a GPU cloud
[07:54:32] <hare>	 IIRC cloud services is missing that in particular
[07:54:43] <zhuyifei1999_>	 yeah, if the transcodes could use some GPU power that might be a real speed up
[08:01:05] <arturo>	 morning
[08:32:57] <Epantaleo>	 hello, after a few trials I can say I can't move my tool from jessie to stretch or buster
[08:33:38] <Epantaleo>	 is there any way I can keep my tool on jessie until a debian package of virtuoso opensorce 7 is released?
[08:33:54] <Epantaleo>	 otherwise I will be forced to shut it down
[08:46:21] <arturo>	 Epantaleo: sorry I forgot, what is your tool?
[08:48:14] <Epantaleo>	 etytree
[08:51:07] <arturo>	 oh, a CloudVPS project
[08:52:06] <arturo>	 the problem with Jessie is that we won't support it past April 2020 because is not going to have security support upstream (Debian) Epantaleo 
[08:52:36] <Epantaleo>	 I understand
[08:52:45] <Epantaleo>	 is there an alternative?
[08:52:53] <arturo>	 can I take a look?
[08:53:00] <Epantaleo>	 sure
[08:53:16] <arturo>	 what is the webpage/repo/url/whatever of this software?
[08:53:20] <Epantaleo>	 what would you like to do?
[08:53:52] <Epantaleo>	 I cannot install virtuoso-opensource-7
[08:54:05] <Epantaleo>	 https://github.com/openlink/virtuoso-opensource/
[08:54:22] <Epantaleo>	 i can install 6.1 but it cannot manage my data
[08:54:37] <Epantaleo>	 virtuoso-opensource-6.1
[08:54:38] <arturo>	 what is jessie current version?
[08:55:03] <Epantaleo>	 of virtuoso?
[08:55:15] <Epantaleo>	 virtuoso-opensource-7
[08:56:35] <Epantaleo>	 (the default on github)
[08:59:26] <arturo>	 so the thing is, jessie is running v7, and you would like to have v7 too in buster or stretch
[08:59:36] <Epantaleo>	 correct
[09:00:02] <arturo>	 how do you usually install it? puppet, deb packages, source code directly?
[09:00:22] <Epantaleo>	 deb packages or source code
[09:00:36] <arturo>	 and where do you get deb packages from?
[09:00:49] <Epantaleo>	 there is an experimental debian package
[09:01:19] <Epantaleo>	 of v7 but coudn't manage to install it as some dependencies where missing
[09:01:22] <Epantaleo>	 were
[09:01:47] <arturo>	 in the etytree-a.etytree.eqiad.wmflabs server?
[09:02:02] <Epantaleo>	 yes
[09:02:04] <Epantaleo>	 https://packages.debian.org/experimental/virtuoso-opensource-7
[09:02:24] <Epantaleo>	 I tried on a different VM
[09:03:44] <arturo>	 let me try!
[09:07:33] <arturo>	 I think this might require just a rebuild for buster
[09:07:42] <arturo>	 I can try that, but I'm not sure where to put the resulting package
[09:08:05] <Epantaleo>	 what do you mean?
[09:08:39] <arturo>	 if I rebuild the deb package for Debian Buster (so dependencies are correct), where should I host this new deb package?
[09:08:43] <arturo>	 I'm not sure
[09:09:13] <Epantaleo>	 I'm not sure I understand the question. in the file system of etytree-a
[09:09:18] <Epantaleo>	 doesn't matter where
[09:09:39] <arturo>	 oh, of course, but if we lost the VM for whatever reason, we will lost the package :-P
[09:10:00] <arturo>	 lose*?
[09:10:03] <Epantaleo>	 do you have an alternative?
[09:11:48] <arturo>	 no, I'm thinking on it
[09:11:53] <Epantaleo>	 :)
[09:13:47] <Epantaleo>	 should I open a task so that we can keep in toush even if I logout of this?
[09:13:49] <Epantaleo>	 touch
[09:16:12] <arturo>	 ok
[09:16:15] <arturo>	 thanks!
[09:16:23] <arturo>	 right now I'm building the debian package for Buster
[09:16:27] <Epantaleo>	 oh, thank you!
[09:16:35] <arturo>	 the source code triggers more or less every single warning from GCC
[09:22:38] <Epantaleo>	 task created https://phabricator.wikimedia.org/T244111
[09:31:14] <arturo>	 cool
[09:31:19] <arturo>	 the package is taking a while to build...
[09:38:06] <arturo>	 !log tools tools-prometheus-01: systemctl stop prometheus@tools. Another try to migrate data to tools-prometheus-{03,04} (T238096)
[09:38:09] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[09:38:10] <stashbot>	 T238096: Toolforge: prometheus: refresh setup - https://phabricator.wikimedia.org/T238096
[09:38:38] <arturo>	 Epantaleo: ok! the package finished the building successfully, apparently
[09:38:56] <arturo>	 will scp it to /root in the etytree-a server
[09:40:47] <Epantaleo>	 great
[09:43:42] <arturo>	 are you in the middle of the night or are you in a timezone similar to mine?
[09:45:03] <Epantaleo>	 it's morning here
[09:46:37] <Epantaleo>	 i'm not a debian expert
[09:46:45] <Epantaleo>	 any tips on how to proceed?
[09:47:42] <Epantaleo>	 i'm at a super beginner level :)
[09:57:34] <arturo>	 !log etytree installed virtuoso-opensource-7 on the etytree-a server by hand (see T244111)
[09:57:38] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Etytree/SAL
[09:57:39] <stashbot>	 T244111: request to build virtuoso-opensource-7 debian package for buster - https://phabricator.wikimedia.org/T244111
[09:57:43] <arturo>	 Epantaleo: try now, the software should be installed now
[10:06:43] <Epantaleo>	 checking it out thanks
[10:06:50] <Epantaleo>	 which password did you set?
[10:07:22] <Epantaleo>	 during installation
[10:12:23] <arturo>	 I didn't set one
[10:12:38] <arturo>	 the default is probably something like 'dba' or the like
[10:13:03] <arturo>	 you can try reconfiguring the package to see the password prompt again
[10:14:28] <arturo>	 something like `dpkg-reconfigure virtuoso-opensource-7`
[10:14:41] <arturo>	 I don't remember which package had the debconf prompt for the password
[10:37:06] <Epantaleo>	 I'm not sure what's wrong but I launched virtuoso and i should be able to see it on localhost:8890
[10:37:10] <Epantaleo>	 w3m http://localhost:8890/
[10:37:15] <Epantaleo>	 but i don't see anything
[10:37:58] <arturo>	 do you see anything listening in that port?
[10:38:04] <arturo>	 try `ss -putanl`
[10:38:04] <Epantaleo>	 i set a DNS proxy at port 8890 at http://etytree-virtuoso.wmflabs.org/
[10:39:13] <Epantaleo>	 peer address:port 0.0.0.0:*
[11:06:19] <Epantaleo>	 arturo: virtuoso says "HTTP/WebDAV server online at 8890" though
[11:06:28] <Epantaleo>	 not sure what's going on
[11:16:03] <Epantaleo>	 did you also install the vad packages?
[11:21:20] <arturo>	 I didn't Epantaleo 
[11:23:55] <arturo>	 let me install them
[11:27:22] <arturo>	 Epantaleo: installed them! try again please
[11:29:29] <Epantaleo>	 i can see that thank you
[11:29:41] <Epantaleo>	 howerver stil i cannot see virtuoso on port 8890 :(
[11:31:22] <Epantaleo>	 oh ok suddenly it works
[11:31:25] <Epantaleo>	 thanks!!
[11:33:10] <arturo>	 🎉
[12:42:23] <Epantaleo>	 arturo: did you do any additional steps than those reported in the task?
[12:42:31] <Epantaleo>	 just for future reference
[12:42:37] <Epantaleo>	 thanks a lot again!
[12:43:11] <arturo>	 I don't think so
[12:48:19] <arturo>	 !log tools shutdown tools-prometheus-01 and tools-prometheus-02, after fixing the proxy `tools-prometheus.wmflabs.org` to tools-prometheus-03, data synced (T238096)
[12:48:22] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[12:48:22] <stashbot>	 T238096: Toolforge: prometheus: refresh setup - https://phabricator.wikimedia.org/T238096
[14:12:03] <arturo>	 !log tools move tools-prometheus-04 from cloudvirt1022 to cloudvirt1013
[14:12:05] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[16:57:19] <wm-bot>	 !log tools.integraality <jeanfred> Deploy latest from Git master: 3b2867f, 10f67fd, 2e6a669 (T244030), 279c92b, de6308e
[16:57:23] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.integraality/SAL
[21:26:12] <bd808>	 !log tools.copyvios Moved uwsgi.log archives to $HOME/logs to clean up $HOME a bit
[21:26:21] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.copyvios/SAL
[21:29:09] <bd808>	 !log tools.copyvios Attempting migration to 2020 Kubernetes cluster (T244107)
[21:29:11] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.copyvios/SAL
[21:29:11] <stashbot>	 T244107: Copyvios tool webservice failed to start on new Kubernetes cluster - https://phabricator.wikimedia.org/T244107
[21:34:06] <bd808>	 !log tools.copyvios Now running on 2020 Kubernetes cluster (T244107)
[21:34:09] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.copyvios/SAL
[22:26:11] <bd808>	 !log tools.countcounttest Deleted "interactive" pod on 2020 Kubernetes cluster that seems to be an experiment in starting a Pod manually.
[22:26:13] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.countcounttest/SAL
[22:33:10] <bd808>	 !log tools.autodesc webservice shows 1820 restarts in last 17 days. Latest logged reason is OOM.
[22:33:11] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.autodesc/SAL
[22:35:31] <bd808>	 !log tools.autodesc Restarted webservice with a higher default memory limit
[22:35:32] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.autodesc/SAL
[22:57:15] <Zppix>	 bd808: quick pm?
[22:57:24] <bd808>	 sure