[03:44:51] <DMaza>	 hello, I'm making some performance updates to https://tools.wmflabs.org/interaction-timeline/ and I'm gonna start caching some db queries. I have two options here.. Filesystem or Redis
[03:45:38] <DMaza>	 do you guys rather have me using one over the other? or is it all the same to you
[03:46:19] <legoktm>	 DMaza: redis. filesystem is on NFS which is rather slow and not good for storing data
[03:46:37] <DMaza>	 thank legoktm, redis it is
[03:46:44] <DMaza>	 *thanks
[08:37:14] <legoktm>	 !screen
[08:37:18] * legoktm pokes wm-bot5
[08:40:28] <legoktm>	 !screen is $ script /dev/null (https://wikitech.wikimedia.org/wiki/Screen#Troubleshooting)
[08:40:29] <wm-bot5>	 Key was added
[11:06:11] <revi>	 https://tools.wmflabs.org/meta/stewardry/enwiki?checkuser=1 I think this is just same version of yesterday guc error?
[11:06:51] <revi>	 oh
[11:06:51] <revi>	 nvm
[11:06:59] <revi>	 was half-sleeping
[11:15:31] * arturo just had a micro heart attack
[11:15:52] <arturo>	 revi: I see `atal error: Unsupported operand types in /mnt/nfs/labstore-secondary-tools-project/meta/git/wikimedia-contrib/tool-labs/stewardry/index.php on line 178`
[12:43:22] <paladox>	 arturo: what’s a micro heart attack?
[12:45:17] <arturo>	 paladox: it was a joke, I had an heart attack when revi reported an issue which could be related to yesterday outage xD
[12:45:55] <paladox>	 Oh I see
[12:46:03] <paladox>	 Thanks for explaining :)
[12:49:00] <arturo>	 I'm with chasemp now, nss/ldap/pam packages give me anxiety @_@
[12:55:25] <paladox>	 Oh
[13:47:33] <arturo>	 !log tools deploying more apt pinnings: https://gerrit.wikimedia.org/r/#/c/416934/
[13:47:38] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[14:00:48] <Nudin_WMDE>	 Technical Advice IRC meeting starting in 2 hours in channel #wikimedia-tech, hosts: @addshore & @Christoph_Jauera_(WMDE) - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting
[14:08:12] <arturo>	 !log tools just merged NFS package pinning https://gerrit.wikimedia.org/r/#/c/416943/
[14:08:16] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[18:30:57] <madhuvishy>	 !log tools Killed php-cgi job run by user 51242 on tools-webgrid-lighttpd-1413
[18:31:01] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[19:34:15] <paladox>	 andrewbogott im getting these errors on the puppet master
[19:34:27] <paladox>	 https://phabricator.wikimedia.org/P6818
[19:34:52] <andrewbogott>	 paladox: is that a stretch box?
[19:35:04] <paladox>	 andrewbogott yes
[19:35:31] <andrewbogott>	 paladox: ok — local puppetmaster on stretch isn't really a thing that we keep track of since (as far as I know) that's the only one
[19:35:37] <andrewbogott>	 it probably needs a new hiera file someplace
[19:35:56] <paladox>	 andrewbogott oh, it was supported from last week
[19:36:07] <andrewbogott>	 it's a side effect of https://gerrit.wikimedia.org/r/#/c/416850/ — best to create a ticket
[19:36:17] <andrewbogott>	 paladox: just because it works sometimes doesn't mean it's supported :)
[19:36:19] <paladox>	 andrewbogott i upgraded when godog said i could :)
[19:37:51] <paladox>	 andrewbogott yep, though i did ask last week before upgrading, which i was told i could :)
[19:38:40] <paladox>	 bd808 what does the !help command do?
[19:39:07] <bstorm_>	 It's generally to page the on-call team member
[19:39:13] <paladox>	 ah thanks
[19:39:21] <bstorm_>	 s/page/notify
[19:39:27] <bstorm_>	 it doesn't really page them per se :)
[19:41:46] <paladox>	 oh :)
[19:46:32] <chicocvenancio>	 I'm notified as well, paladox, but that is expected and good
[19:46:43] <paladox>	 thanks :)
[19:55:23] <paladox>	 andrewbogott will be fixed in https://gerrit.wikimedia.org/r/#/c/417012/1/
[20:09:07] <Sagan>	 !log rcm Xenon: Running update
[20:09:09] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Rcm/SAL
[20:09:11] <Sagan>	 !log rcm Neon: Running update
[20:09:12] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Rcm/SAL
[20:09:18] <Sagan>	 !log rcm Oxygen: Updating packages
[20:09:19] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Rcm/SAL
[20:09:27] <Sagan>	 !log rcm Tin: Updating packages
[20:09:28] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Rcm/SAL
[20:09:56] <chasemp>	 Sagan: I detect a theme in naming there :)
[20:10:14] <Sagan>	 :)
[20:10:34] <Sagan>	 it's mostly that it is easy for me to check when I did that last time ;)
[20:42:43] <chicocvenancio>	 !log tools killed io intensive recursive zip of huge folder
[20:42:49] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[20:44:00] <chicocvenancio>	 Freddy2001: hey, that ^ was your zip process
[20:44:51] <chicocvenancio>	 zipping a small folder probably won't affect anyone, but the zipfile is already 15gb and it maxed io for the bastion
[20:45:19] <chicocvenancio>	 sending that to the grid will make sure less users are affected
[20:46:20] <chasemp>	 chicocvenancio: another tip of limited utility if too big for local tmp usage esp is to zip onto scratch which has a separate IO quota and a separate backend
[20:46:38] <chasemp>	 obv breaks down too if too many ppl are doing it but generally useful
[20:46:57] <chicocvenancio>	 chasemp: won't it still leave the tempfile in $HOME?
[20:47:46] <chasemp>	 that may mean it's of very limited utlity then yes :)
[20:48:14] * chicocvenancio is reading man zip
[20:48:17] <Freddy2001>	 hi chicocvenancio 
[20:48:59] <chicocvenancio>	 `--temp-path` or `-b` can change tempfile location
[20:49:01] <Freddy2001>	 i want move some files from nfs to an seperate instance
[20:49:18] <chicocvenancio>	 Cloud VPS?
[20:50:03] <Freddy2001>	 yes
[20:51:01] <chicocvenancio>	 Freddy2001: do you know how big the cabal folder is?
[20:51:02] <zhuyifei1999_>	 Freddy2001: rsync / scp
[20:51:25] <Freddy2001>	 it was 19 gb, you cancelled it just before it has finished
[20:51:49] <chicocvenancio>	 weirdly, du shows me 146mb for that
[20:51:58] <Freddy2001>	 lol?
[20:52:23] <Freddy2001>	 i want to keep it in eqiad, because my internet connection is not that fast
[20:52:35] <chicocvenancio>	 yeah, that makes sense, with 19GB
[20:52:47] <Freddy2001>	 so i can not use scp to download it on my local machine and upload it to cloud vps
[20:53:16] <zhuyifei1999_>	 you can still try rsync within cloud
[20:53:21] <chicocvenancio>	 scp between them should be possible if you setup the security group for the instance
[20:53:37] <Freddy2001>	 zhuyifei1999_, how?
[20:53:42] <chicocvenancio>	 (I actually think the default security group allows it)
[20:54:24] <zhuyifei1999_>	 I'll check the manual. Done it before but that was long time ago
[20:54:33] <chicocvenancio>	 scp cabal YOURINSTANCE:/desired/path
[20:54:34] <zhuyifei1999_>	 I gtg in a min, will be back in ~20 mins
[20:55:33] <Freddy2001>	 thanks, chicocvenancio, i will try it
[20:55:36] <chicocvenancio>	 if you haven't changed the default security group and have agent forwarding enabled it will work from your user
[20:55:39] <chicocvenancio>	 Freddy2001: ^
[20:55:51] <Freddy2001>	 fitst i need to import my ssh keys, right?
[20:56:35] <chicocvenancio>	 how are you connecting to WMCS?
[20:56:42] <chicocvenancio>	 windows, linux, mac?
[20:56:55] <chicocvenancio>	 putty, ssh client, other?
[20:57:29] <Freddy2001>	 ssh on bash
[20:58:07] <Freddy2001>	 i am using ubuntu
[20:58:43] <chicocvenancio>	 ok, do you use ssh-agent?
[20:59:49] <Freddy2001>	 i just have my keys in .ssh/ and on wikitech
[21:00:13] <Freddy2001>	 connecting via ssh bastion.wmflabs.org
[21:01:41] * chicocvenancio is chekcing something
[21:02:40] <Freddy2001>	 sure
[21:08:56] <zhuyifei1999_>	 Freddy2001: ubuntu uses ssh agent by default
[21:09:11] <Freddy2001>	 so what do i need to do?
[21:10:11] <chicocvenancio>	 try to send the command from tools-bastion-03
[21:10:17] <zhuyifei1999_>	 I do not recommend forwarding your agent, as it allows a root to use your keys while you are connected
[21:11:04] <chicocvenancio>	 zhuyifei1999_: copying the keys to his .ssh folder does the same, no?
[21:11:05] <zhuyifei1999_>	 but if you feel the thing worth the risk, the way to forward the agent is ssh -A
[21:11:13] <Freddy2001>	 <chicocvenancio> try to send the command from tools-bastion-03 → i did, but it failed with permission denied (publickey)
[21:11:44] <zhuyifei1999_>	 chicocvenancio: no, that is permanent
[21:13:05] <zhuyifei1999_>	 if the agent is forwarded it’s much more difficult to obtain the key itself
[21:13:39] <Freddy2001>	 same here with -A 
[21:13:44] <Freddy2001>	 Permission denied (publickey).lost connection
[21:14:16] <zhuyifei1999_>	 if someone malicious gets the key, they can impersonate at any time, but with agent, impersonate only when connected
[21:15:31] <zhuyifei1999_>	 Freddy2001: if you decide to you ssh agent forward method, you need to ssh to the bastion with -A
[21:15:47] <zhuyifei1999_>	 that forwards the agent to the bastion
[21:16:49] <zhuyifei1999_>	 I recommend starting a rsync daemon and adjusting the firewall at the file destination instead
[21:17:03] <Freddy2001>	 zhuyifei1999_, i tried -A, but it failed
[21:17:20] <zhuyifei1999_>	 ssh to bastion?
[21:18:00] <chicocvenancio>	 maybe it is proxying through bastion.wmflabs even to tools-bastion-03?
[21:18:18] <chicocvenancio>	 but rsync is definitely safer and better
[21:21:36] <Freddy2001>	 okay
[21:21:54] <zhuyifei1999_>	 if someone feel adventurous they could also try to build HBA (host-based authentication)for ssh
[21:23:01] <zhuyifei1999_>	 that toolforge internally does that, so tools can put crontabs on tools-cron-01
[21:25:18] <zhuyifei1999_>	 Freddy2001: anyways, if you try to go the agent-forwarding method, please show how you connected to toolforge bastion and how you are attempting to connect to the receiver
[21:41:56] <chasemp>	 tools-bastion-03 is directly exposed so you can ssh directory to it via login.tools.wmflabs.org (I believe) doing that w/ forwarding if firewalls are opened up would let you scp something to anotehr instance in another project
[21:58:00] <chicocvenancio>	 chasemp: thats what I usually do, but I trust the roots more than I should
[21:58:25] * bd808 has shifty eyes
[22:01:42] * zhuyifei1999_ forwards agent if I commit to some huge repos like ops/puppet, and don't have time to clone the repo to local
[22:06:41] <Krinkle>	 Hm.. https://tools.wmflabs.org/?tool=meetbot and https://tools.wmflabs.org/?tool=guc used to show which developers maintain the tool.
[22:06:48] <Krinkle>	 Looks like that isn't listed there anymore.
[22:07:03] <Krinkle>	 Is there a different public  source to view it? the "manage maintainers" link requires login.
[22:08:33] <chasemp>	 Krinkle: yeah good question, I'm not sure what the best view for maintainers is atm either
[22:08:40] <chasemp>	 bd808: ^ :)
[22:09:34] <madhuvishy>	 Krinkle: https://tools.wmflabs.org/admin/tools
[22:10:03] <bd808>	 the missing maintainers is a bug...
[22:10:13] <bd808>	 there's an open task about it somewhere...
[22:11:37] <bd808>	 T186571
[22:11:38] <stashbot>	 T186571: Toolforge search results does not show all maintainers - https://phabricator.wikimedia.org/T186571
[22:12:10] <bd808>	 there is some db table that is out of sync I guess. I haven't really looked into it
[22:12:56] <bd808>	 the "better" view is https://toolsadmin.wikimedia.org/tools/id/guc
[23:20:39] <halfak>	 !log wikilabels staged wikilabels-wmflabs-deploy:67127b
[23:20:42] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikilabels/SAL
[23:22:53] <halfak>	 !log wikilabels deployed wikilabels-wmflabs-deploy:67127b
[23:22:55] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikilabels/SAL