Fork me on GitHub

Wikimedia IRC logs browser - #wikimedia-cloud

Filter:
Start date
End date

Displaying 103 items:

2021-02-23 10:48:59 <arturo> !log admin installing linux-image-amd64 from buster-bpo 5.10.13-1~bpo10+1 in cloudnet1004 (T271058)
2021-02-23 10:49:05 <stashbot> Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
2021-02-23 10:49:05 <stashbot> T271058: cloudnet1004/cloudnet1003: network hiccups because broadcom driver/firmware problem - https://phabricator.wikimedia.org/T271058
2021-02-23 10:49:46 <arturo> !log admin rebooting clounet1004 into new kernel from buster-bpo (T271058)
2021-02-23 10:49:58 <stashbot> Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
2021-02-23 13:05:45 <godog> hi, reading https://wikitech.wikimedia.org/wiki/Help:Using_a_web_proxy_to_reach_Cloud_VPS_servers_from_the_internet am I right in concluding that the web proxy will talk to its backends via http only and not https, e.g. if I use 443 as the port ?
2021-02-23 13:06:58 <Majavah> godog: currently yes
2021-02-23 13:07:30 <godog> thank you Majavah
2021-02-23 13:07:55 <Majavah> T274386 is about adding https support, if you want to subscribe to it
2021-02-23 13:07:56 <stashbot> T274386: Allow end to end encryption through the shared web proxy - https://phabricator.wikimedia.org/T274386
2021-02-23 13:10:01 <godog> definitely interested, subscribed! thanks for the pointer
2021-02-23 16:06:15 <andrewbogott> godog: if you want full end-to-end today your best bet is to get a floating IP for your project and manage things yourself. Setting up acme-chief in your project to manage your own cert is a fair bit of trouble but it works and we're doing it in several projects.
2021-02-23 17:03:34 <godog> andrewbogott: thank you, yeah that's an attractive option. manage things myself I'm assuming you mean e.g. wildcard dns records and all that, what would be the process/procedure for requesting ? or self-service ?
2021-02-23 17:06:30 <andrewbogott> godog: you can make a quota request for an IP: https://phabricator.wikimedia.org/project/view/2880/
2021-02-23 17:06:41 <andrewbogott> floating ip docs: https://wikitech.wikimedia.org/wiki/Help:Manage_floating_IP_addresses_assigned_to_Cloud_VPS_instances#Floating_IP_addresses
2021-02-23 17:07:02 <andrewbogott> acme-chief in cloud-vps docs: https://wikitech.wikimedia.org/wiki/Acme-chief/Cloud_VPS_setup
2021-02-23 17:07:22 <andrewbogott> ^ that last leaves a lot to be desired but the docs can only be improved by having a new person try to follow them :)
2021-02-23 17:08:07 <godog> hehehe
2021-02-23 17:08:48 <godog> ok I'll start with the floating ip request
2021-02-23 17:11:43 <godog> re: dns bits, if initially I'd need "just" a wildcard A record like *.project.wmcloud.org -> my floating ip is that self-service or I need to bug someone ?
2021-02-23 17:18:47 <Tomodachi94> hi
2021-02-23 17:19:30 <Tomodachi94> I keep getting an error upon running "ssh Tomodachi94@tools-login.wmflabs.org
2021-02-23 17:19:30 <Tomodachi94> ":
2021-02-23 17:19:31 <Tomodachi94> The authenticity of host 'tools-login.wmflabs.org (185.15.56.48)' can't be estab
2021-02-23 17:19:31 <Tomodachi94> lished.
2021-02-23 17:19:32 <Tomodachi94> ECDSA key fingerprint is SHA256:8fLy4F9XDYdR/uHihWoPihKDhPaxCh0au/paSdGB7K8.
2021-02-23 17:19:32 <Tomodachi94> Are you sure you want to continue connecting (yes/no)? yes
2021-02-23 17:19:33 <Tomodachi94> Warning: Permanently added 'tools-login.wmflabs.org,185.15.56.48' (ECDSA) to the
2021-02-23 17:19:33 <Tomodachi94>  list of known hosts.
2021-02-23 17:19:34 <Tomodachi94> Tomodachi94@tools-login.wmflabs.org: Permission denied (publickey,hostbased).
2021-02-23 17:20:02 <Tomodachi94> I have the key in .ssh/rsa-id
2021-02-23 17:21:37 <Tomodachi94> !help
2021-02-23 17:21:38 <wm-bot> If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-kanban
2021-02-23 17:22:56 <andrewbogott> Tomodachi94: My first thought is that that it's unusual for a username to have a capital letter in it; is it possible you've confused your on-wiki name with your shell name?
2021-02-23 17:23:41 <Tomodachi94> andrewbogott perhaps, let me try it an all-lowercase user
2021-02-23 17:23:48 <Majavah> according to https://ldap.toolforge.org/user/Tomodachi94 the shell username is `tomodachi94`, maybe try using that (lowercase)
2021-02-23 17:23:49 <Tomodachi94> *as an
2021-02-23 17:24:22 <Tomodachi94> $ ssh tomodachi94@tools-logit.wmflabs.org
2021-02-23 17:24:23 <Tomodachi94> omodachi94@tools-login.wmflabs.org: Permission denied (publickey,hostbased).
2021-02-23 17:24:46 <Tomodachi94> * tomodachi94@tools-login.wmflabs.org: Permission denied (publickey,hostbased).
2021-02-23 17:25:02 <andrewbogott> yeah, the server doesn't like your key
2021-02-23 17:25:10 <andrewbogott> you can see more about what's happening if you do ssh -vvv
2021-02-23 17:25:32 <Tomodachi94> welp that spammed my console
2021-02-23 17:25:49 <Tomodachi94> OH
2021-02-23 17:25:59 <Tomodachi94> it is looking for "id_rsa", not "id-rsa"
2021-02-23 17:26:05 <Majavah> also use a pastebin service instead of pasting multiple lines here, otherwise the network might kick you off
2021-02-23 17:26:30 <Tomodachi94> oh, thanks for the tip
2021-02-23 17:26:50 <Tomodachi94> it still doesn't like my key D:
2021-02-23 17:27:46 <Tomodachi94> im going to regen my key i guess
2021-02-23 17:29:10 <andrewbogott> Tomodachi94: if you're not sure what key is getting used you can force your local ssh client to use a particular key with -i
2021-02-23 17:29:29 <andrewbogott> like ssh -i /path/to/my/private/key <username>@<hostname>
2021-02-23 17:30:11 <Tomodachi94> oh, that worked andrewbogott
2021-02-23 17:30:17 <andrewbogott> nice
2021-02-23 17:31:13 <RhinosF1> Tomodachi94: you might wanna create a file in .ssh/config that saves the path to key and username for you
2021-02-23 17:31:56 <andrewbogott> thank you for helping out, RhinosF1 and Majavah !
2021-02-23 17:32:21 <Tomodachi94> thank you andrewbogott RhinosF1 and Majavah
2021-02-23 17:32:52 <RhinosF1> Tomodachi94: you can also use login.toolforge.org now I believe
2021-02-23 17:33:05 <RhinosF1> andrewbogott: np
2021-02-23 19:52:50 <brawer> Hi all, can toolforge kubernetes jobs attach Cinder volumes? https://kubernetes.io/blog/2020/02/07/deploying-external-openstack-cloud-provider-with-kubeadm/
2021-02-23 19:57:35 <andrewbogott> brawer: I wouldn't expect that to work, although I'm interested in hearing about what you find if you try.
2021-02-23 20:24:23 <bd808> brawer: today the answer to your question is no. But having cinder backed persistent volumes for Toolforge tools is something that we would like to enable in the future.
2021-02-23 20:24:47 <brawer> Thanks for the quick reply! Filed https://phabricator.wikimedia.org/T275555
2021-02-23 20:24:50 <bd808> We had to get cinder first before anyone could work on exposing it to Kubernetes. :)
2021-02-23 20:36:00 <andrewbogott> !log admin adding r/o access to the eqiad1-glance-images ceph pool for the client.eqiad1-compute for T275430
2021-02-23 20:36:04 <stashbot> Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
2021-02-23 20:36:04 <stashbot> T275430: Large images cloned to /var/lib/nova/instances/_base filling up disk on hypervisors - https://phabricator.wikimedia.org/T275430
2021-02-23 20:44:09 <wm-bot> !log tools.adhs-wde <root> Deleted "test" deployment and related pod stuck in CrashLoopBackoff.
2021-02-23 20:44:14 <stashbot> Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.adhs-wde/SAL
2021-02-23 20:47:57 <wm-bot> !log tools.lekhaki <root> Deleted deployment.apps/lilywhite.bot which was spawning pods into CrashLoopBackoff due to missing /data/project/lekhaki/tool-lekhaki/main.js entrypoint file.
2021-02-23 20:48:00 <stashbot> Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lekhaki/SAL
2021-02-23 20:55:13 <wm-bot> !log tools.wikiloop <root> Stopped webservice. Pod in CrashLoopBackoff and restarting did no seem to help.
2021-02-23 20:55:15 <stashbot> Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikiloop/SAL
2021-02-23 20:57:38 <brawer> What’s the current advice for writing tools in compiled languages? Currently I’m statically cross-compiling the binary on my laptop, and then scp that to dev.toolforge.org, from where I move it into the role account’s home directory. Deployment would be simpler if I could push images to the Docker registry, as in `docker image push
2021-02-23 20:57:38 <brawer> docker-registry.toolforge.org/user/tool:v1.2.3`. Then, I could also package libraries and static files into the image. However, that doesn’t seem supported.?(Maybe for security reasons? But since I can scp my binaries to the bastion server, or log into the bastion to fetch my executables off the internet, allowing developers to push their images
2021-02-23 20:57:39 <brawer> to the Docker registry wouldn’t be any less secure?) Anyhow, how do people currently ship their code? Apologies for my newbie question.
2021-02-23 21:03:40 <bd808> brawer: most Toolforge tools are in interpreted languages rather than compiled. For compiled things, what you have worked out would seem reasonable to me. The other option is compiling locally on Toolforge using either the grid engine or a custom submitted Kubernetes task.
2021-02-23 21:04:47 <bd808> T194332 is the epic for next generation Toolforge things which will not allow pushing arbitrary containers, but is hoped to provide a good solution for compiling and deploying.
2021-02-23 21:04:47 <stashbot> T194332: [Epic] Make Toolforge a proper platform as a service with push-to-deploy and build packs - https://phabricator.wikimedia.org/T194332
2021-02-23 21:08:10 <legoktm> brawer: https://wikitech.wikimedia.org/wiki/User:Legoktm/Rust_on_Toolforge is how I'm currently doing it for Rust projects
2021-02-23 21:11:58 <brawer> Thank you!
2021-02-23 21:20:08 <wm-bot> !log tools.simplewd <root> Hard stop/start cycle. Pod in CrashLoopBackOff with average restart every 5 minuted for the last 2 months.
2021-02-23 21:20:12 <stashbot> Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.simplewd/SAL
2021-02-23 22:22:36 <wm-bot> !log tools.citationhunt <root> Clean up completed pods
2021-02-23 22:22:39 <stashbot> Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.citationhunt/SAL
2021-02-23 22:25:31 <wm-bot> !log tools.citationhunt <root> Deleted Error state cronjob pods
2021-02-23 22:25:34 <stashbot> Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.citationhunt/SAL
2021-02-23 22:31:06 <daemon> /!\ this chat has moved to irc.crimeircd.net #pp /!\
2021-02-23 22:31:27 <hsiktas> /!\ this chat has moved to irc.crimeircd.net #pp /!\
2021-02-23 22:34:17 <mutante> lol?
2021-02-23 22:35:53 <Majavah> mutante: spambots, similar ones were going across freenode few weeks ago
2021-02-23 22:36:15 <mutante> yea, they got k-lined
2021-02-23 22:39:03 <wm-bot> !log tools.jb <root> Stopping jdk11 webservice in CrashLoopBackOff caused by missing extra arguments to tell the pod what to run.
2021-02-23 22:39:06 <stashbot> Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.jb/SAL
2021-02-23 22:40:26 <bstorm> !log cloudvirt-canary rebuild the canary for 1028 after image changes and all is well T275430
2021-02-23 22:40:29 <stashbot> Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Cloudvirt-canary/SAL
2021-02-23 22:40:30 <stashbot> T275430: Large images cloned to /var/lib/nova/instances/_base filling up disk on hypervisors - https://phabricator.wikimedia.org/T275430
2021-02-23 22:43:55 <bstorm> !log admin set --property hw_scsi_model=virtio-scsi and --property hw_disk_bus=scsi on the main buster image in glance on eqiad1 T275430
2021-02-23 22:43:59 <stashbot> Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
2021-02-23 23:06:54 <bstorm> !log tools draining tools-k8s-worker-55 to clean up after dumps changes T272397
2021-02-23 23:07:00 <stashbot> Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
2021-02-23 23:07:00 <stashbot> T272397: cloud: drop NAT exception for dumps NFS - https://phabricator.wikimedia.org/T272397
2021-02-23 23:11:46 <bstorm> !log tools draining a bunch of k8s workers to clean up after dumps changes T272397
2021-02-23 23:11:50 <stashbot> Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL

This page is generated from SQL logs, you can also download static txt files from here