[14:30:25] <wm-bot>	 !log tools.wd-image-positions <lucaswerkmeister> deployed ff77d74e3f (add somevalue depicts statements)
[14:30:28] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wd-image-positions/SAL
[17:16:52] <mpeel>	 Hi, does anyone have a bit of time to help me with the correct setup to run a python script querying the commonswiki database, please? I have the python code and query working, it’s just the server setup I’m not so familiar with.
[17:38:55] <mpeel>	 !help if possible please. :-) I’ve got as far as test runs on the login node now, but am struggling to get the code into a tool to run it through the grid engine.
[17:38:55] <wm-bot>	 If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-kanban
[17:40:30] <andrewbogott>	 mpeel: I don't know a whole lot but I can try to help.  Is this a one-off job you want to dispatch, or a service that needs to run indefinitely?
[17:41:18] <mpeel>	 andrewbogott: thanks. :-) It’s a batch job I want to run every day or two.
[17:41:55] <andrewbogott>	 but you'll be running it by hand each time?
[17:42:30] <mpeel>	 at the moment, yes. It can probably be automated further, but that’s something for the future.
[17:42:37] <andrewbogott>	 got it, ok
[17:42:47] <andrewbogott>	 so this sounds like
[17:42:47] <andrewbogott>	 https://wikitech.wikimedia.org/wiki/Help:Toolforge/Grid#Submitting_simple_one-off_jobs_using_'jsub'
[17:42:55] <andrewbogott>	 are you already trying to follow that guide and running into trouble?
[17:43:29] <mpeel>	 I haven’t got that far yet. The code is in my user home directory only right now.
[17:44:23] <mpeel>	 I can’t figure out how to copy from my home directory to a tool account.
[17:44:28] <andrewbogott>	 It's really simple, basically just 'jsub </path/to/script>'
[17:44:54] <mpeel>	 I’ve set up https://toolsadmin.wikimedia.org/tools/id/pibot . I can check out that git repository, but not commit to it.
[17:45:08] <andrewbogott>	 ok hang on...
[17:45:13] <andrewbogott>	 you're logged into a toolforge bastion right?
[17:45:17] <andrewbogott>	 And have your code there?
[17:45:34] <mpeel>	 I’m logging into tools-login.wmflabs.org
[17:45:50] <mpeel>	 I can also ftp into it, so I have my code in /home/mpeel/ at the moment.
[17:48:31] <Krenair>	 Yeah so you just need to get the files from your account to the tool account
[17:49:01] <mpeel>	 yup
[17:49:02] <Krenair>	 sounds like a standard permissions exercise? either the tool needs to be able to read the files out of your area to do the copy, or you need to be able to write the files into the tool's area to do it
[17:49:42] <Krenair>	 or you push it elsewhere and pull down into the tool
[17:49:45] <mpeel>	 ah, that’s OK? I wasn’t sure if I had to use the git repository to store the files.
[17:49:52] <Krenair>	 oh you probably should
[17:50:09] <Krenair>	 your repo is empty
[17:50:21] <Krenair>	 were you struggling to push to it?
[17:51:00] <mpeel>	 “fatal: unable to access 'https://phabricator.wikimedia.org/source/tool-pibot.git/': The requested URL returned error: 403”
[17:51:09] <mpeel>	 when I try to ‘git push'
[17:51:22] <mpeel>	 but if I can just store the code in the tool homedirectory, pico seems to work.
[17:51:37] <Krenair>	 mm, it's supposed to be open source I think
[17:51:50] <Krenair>	 don't think you *have* to use the provided git repo, it could be github or whatever
[17:52:06] <Krenair>	 let's see if I can deal with the permissions on this
[17:52:56] <Krenair>	 it looks like it's editable/pushable by Mike_Peel on phab
[17:53:31] <Krenair>	 let's see if I can put stuff there as an admin
[17:54:11] <Krenair>	 Oh right
[17:54:13] <Krenair>	 pushing via HTTPS
[17:54:34] <Krenair>	 mpeel, obviously don't paste your credentials themselves, but what type of credentials are you trying to use here?
[17:54:48] <Krenair>	 I'm getting prompted like this
[17:54:49] <Krenair>	 Password for 'https://Krenair@phabricator.wikimedia.org': 
[17:55:15] <Krenair>	 Which interestingly enough does not accept my LDAP password.
[17:55:52] <Krenair>	 There's a thing in Phab's Settings about a VCS Password, need to be in high-security mode to get into it
[17:57:46] <Krenair>	 With that set up:
[17:57:47] <Krenair>	 To https://phabricator.wikimedia.org/source/tool-pibot.git
[17:57:48] <Krenair>	  * [new branch]      alex-testing -> alex-testing
[17:58:09] <andrewbogott>	 mpeel: sorry I lost the thread, my phone is buzzing like crazy and I'm distracted
[17:58:53] <hauskatze>	 Krenair: I think either https or ssh push on diffusion was bugged somehow. Saw a task about it but I don't remember the number
[17:59:29] <hauskatze>	 another reason not to use differential/diffusion (/me hides)
[18:00:06] <mpeel>	 I’m using my https://phabricator.wikimedia.org/p/Mike_Peel/ account.
[18:01:01] <Krenair>	 And when it prompts for password, the password you give it, do you try the one set up from Phab's VCS Password settings, or one used for some other system?
[18:01:17] <mpeel>	 I have all my code on bitbucket, I guess I could just use that.
[18:01:22] <Krenair>	 sure.
[18:01:34] <mpeel>	 The same password I use to log into phabricator.
[18:01:41] <Krenair>	 if you've already got one, makes sense to just pull that one down into the tool
[18:01:48] <Krenair>	 mpeel, so your LDAP password then?
[18:01:56] <mpeel>	 probably, yes.
[18:02:00] <Krenair>	 I tried that and it doesn't work, presumably because of 2FA etc.
[18:02:08] <Krenair>	 Oh no
[18:02:10] <Krenair>	 Looking at your user
[18:02:14] <Krenair>	 You actually do not have LDAP associated
[18:02:24] <Krenair>	 you're logging in via CA integration with MediaWiki...
[18:02:35] <Krenair>	 yeah it definitely won't be able to do password auth through that.
[18:02:43] <mpeel>	 ah, that’s not obvious!
[18:02:54] <Krenair>	 think that's an OAuth thing
[18:03:10] <Krenair>	 I mean, you don't put your password directly into Phabricator for that, do you?
[18:03:30] <Krenair>	 It sends you to MediaWiki and you log in there if you aren't already logged in
[18:03:39] <mpeel>	 I’m not sure. I know it’s a different password than my mediawiki login.
[18:04:08] <mpeel>	 but ldap login doesn’t work. odd.
[18:04:18] <Krenair>	 uhhh.
[18:04:22] <Krenair>	 really?
[18:04:31] <Krenair>	 I think LDAP and MediaWiki are the only two options...
[18:04:46] <mpeel>	 ah, no, sorry, the different password is for toolsadmin.
[18:05:17] <Krenair>	 right
[18:05:19] <Krenair>	 anyway
[18:05:32] <Krenair>	 to push to a Diffusion repo you'd need to go and set up a VCS password
[18:05:47] <Krenair>	 but in your case, you already have a Bitbucket repo
[18:05:50] <Krenair>	 which is probably all you need
[18:06:12] <mpeel>	 OK, so I can put the code on there, and then git pull it into the tool account?
[18:06:16] <Krenair>	 yeah.
[18:25:41] <mpeel>	 Krenair: andrewbogott Thanks for your help, my code’s on the system now, and I’m testing running it using jsub now!
[18:25:49] <Krenair>	 great
[18:30:21] <andrewbogott>	 cool!
[20:49:26] <mpeel>	 “pymysql.err.OperationalError: (2013, 'Lost connection to MySQL server during query’)” - ah well. :(
[20:49:47] <mpeel>	 (see https://phabricator.wikimedia.org/T246970 for the saga, if you’re interested)
[20:59:16] <RhinosF1>	 !log tools.zppixbot-test tools.zppixbot-test@tools-sgebastion-07:~/.sopel$ kubectl scale --replicas=0 deployment.apps/zppixbot-test
[20:59:18] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot-test/SAL
[21:00:28] <RhinosF1>	 !log tools.zppixbot-test switch that to sopeltest.bot
[21:00:28] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot-test/SAL
[21:01:54] <RhinosF1>	 !log tools.zppixbot-test rename starter.sh to starter-old and create starter-new - move zppixbot-test to use it for the deployment
[21:01:56] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot-test/SAL
[21:14:29] <RhinosF1>	 !log tools.zppixbot-test tools.zppixbot-test tools.zppixbot-test@tools-sgebastion-07:~/.sopel$ kubectl scale --replicas=1 deployment.apps/sopeltest.bot
[21:14:32] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot-test/SAL
[21:18:26] <RhinosF1>	 !help it's now got the status CrashLoopBackOff and kubectl logs shows noothing
[21:18:26] <wm-bot>	 If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-kanban
[21:20:26] <wm-bridgebot>	 [telegram] <bd808> RhinosF1: try ‘kubectl get event’
[21:23:04] <RhinosF1>	 @bd808: here's the output https://www.irccloud.com/pastebin/eyzxkYna/
[21:23:20] <RhinosF1>	 I see it
[21:26:48] <RhinosF1>	 !log tools.zppixbot-test tools.zppixbot-test@tools-sgebastion-07:~/k8s$ take /data/project/zppixbot-test/k8s/starter-new.sh - I hate forklift's file handling at times
[21:26:49] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot-test/SAL
[21:27:53] <RhinosF1>	 @bd808: I should have fixed it and should know by now that ForkLift can be a jerk
[21:28:02] <RhinosF1>	 (that's my sftp client)
[21:29:06] <RhinosF1>	 nope, I see 45s         Warning   Failed              pod/sopeltest.bot-6b45cd8d5-vkc86     Error: failed to start container "sopeltest": Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"/data/project/zppixbot-test/k8s/starter.sh\": stat /data/project/zppixbot-test/k8s/starter.sh: no such file or directory": unknown 
[21:29:24] <RhinosF1>	 which makes 0 sense as it should be looking at starter-new.sh
[21:29:59] <RhinosF1>	 sopelbot.yaml clearly has been changed to           command: [ "/data/project/zppixbot-test/k8s/starter-new.sh", "bash" ]
[21:30:17] <RhinosF1>	 unless for some reason changing sopelbot.yaml requires rebooting the deployment
[21:34:46] <RhinosF1>	 @bd808: should I re-create the deployment?
[21:36:48] <wm-bridgebot>	 [telegram] <bd808> RhinosF1: the yaml on disk is not automatically read by Kubernetes. You must update the running config in the cluster yourself. So, yes.
[21:47:56] <RhinosF1>	 @bd808: 10s         Warning   Failed              pod/sopeltest.bot-6876f8c6b4-wrcq5    Error: failed to start container "sopeltest": Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"/data/project/zppixbot-test/k8s/starter-new.sh\": permission denied": unknown
[21:48:11] <RhinosF1>	 4 -rw-r--r--  1 tools.zppixbot-test tools.zppixbot-test  611 May 30 21:03 starter-new.sh shows in ls -lsa
[21:49:13] <wm-bridgebot>	 [telegram] <bd808> Needs chmod a+x starter-new.sh
[21:49:50] <wm-bot>	 !log tools.zppixbot-test <rhinosf1> chmod a+x starter-new.sh
[21:49:52] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot-test/SAL
[21:50:31] <RhinosF1>	 that's looking better
[21:52:27] <RhinosF1>	 @bd808: Yey! We're back.
[21:52:48] <RhinosF1>	 !log tools.zppixbot-test Maint Complete!
[21:52:49] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot-test/SAL
[21:53:26] <Reedy>	 You really don't need to keep pinging him
[21:54:44] <RhinosF1>	 I'm off for the night anyway now that's resolved