[10:14:25] !help Can I use cronjob on a html file? [10:14:25] Adithyak1997: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [10:16:20] hi Adithyak1997. HTML is a markup language https://en.wikipedia.org/wiki/Markup_language, not a programming one. [10:16:50] so the cronjob will not be able to execute anything with such a file. [10:53:00] !help I would like to create a cronjob that runs every 30 minutes. I have the idea about the syntax. But I have doubt whether I need to used jstart or jsub [10:53:00] Adithyak1997: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [13:16:08] !help I have forked a repository in github. I would like to know how I can commit my change to reflect it in toolforge [13:16:08] Adithyak1997: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [13:42:06] Adithyak1997: login with SSH, run `become your_tool_name`; `cd` into the directory/repository and run `git pull` [13:42:19] Adithyak1997: there's no automated polling in toolforge [13:46:22] Ok [13:48:25] But what does directory/repository mean? Is that the location in github website? [15:14:36] How can I edit html files for a tool that is hosted in toolforge? [15:14:45] !help [15:14:45] Adithyak1997: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [15:21:58] Adithyak1997: i would say start at https://tools.wmflabs.org/admin/tools and identify the tool. Then see who is listed as Maintainer of it. See if you can find them on IRC or wiki and ask. next thing you make a user on wikitech.wikimedia.org and then you can use it on https://toolsadmin.wikimedia.org/auth/login/?next=/tools/membership/apply to request access to toolforge [15:23:09] I actually have been added as a co-maintainer for the tool Firefly tools. All the things mentioned above by you have been done by me. [15:23:30] I am using windows 10. I am using putty to edit files other than html [15:23:34] ah, ok. then i misunderstood [15:23:34] Have you read through this material? https://wikitech.wikimedia.org/wiki/Help:Getting_Started [15:24:02] Yes [15:24:06] There is a lot of basic stuff linked in that may clear up skme questions [15:24:44] Ok. Have you been able to log in via ssh yet? [15:24:51] Yes [15:25:04] I have edited cron job in that [15:25:15] Ok. Cool [15:25:47] Through forking, I was able to edit html files [15:26:12] But I couldn't add them to the tool present in toolforge [15:26:29] git commands described on githib should work on there, ideally run as your tool [15:27:00] Are you working as your tool? [15:27:05] But the git pull command is showing a permission denied error as I am not an owner or maintainer of that tool [15:27:35] I didn't understand. If you mean am I working on my tool, yes I am working [15:27:35] Running `become whateveryourtoois` [15:27:43] Done [15:27:43] No [15:28:02] That command changes to an account named after your tool [15:28:15] I have done that [15:28:24] It also changes folder, ok [15:29:14] Your files should be there. Your git credentials need to be entered correctly. Toolforge doesn't affect that [15:29:32] Waiit [15:29:37] Ok [15:30:29] if git pull doesn't work because of local permissions, you might be working with files your account made vs. your tool account [15:30:58] Do I need to login with my tool account? [15:31:35] I am sorry. Git pull shows data is up to date [15:31:37] `become` does that...but it only owns files you create after you become [15:31:40] But git push doesn;t work [15:32:26] Ah push is on the github side. Are you pushing to your fork or the main repository on github? [15:33:24] `git remote -v` will tell you [15:34:14] origin https://github.com/rwjuk/fireflytools (fetch) origin https://github.com/rwjuk/fireflytools (push) [15:34:23] This was the thing displayed [15:35:24] Is that your fork of the fireflytools repo on github? [15:35:54] Or is that the orginal? [15:35:59] Original [15:37:08] Ok, for you to push to it you have to remove the folder on toolforge or locally, and the clone your fork instead [15:37:53] Do you need/want your code to end up in the original eventually? [15:38:05] Yes [15:38:19] I actually needs the tool to get updated [15:38:27] It is my initial goal [15:38:49] Through cron job I was able to update 3 pages but not remaining [15:42:03] Is the tool controlled by someone else? Like on Toolforge? Generally on github, you would edit your fork, make a pukl request and the maintainer would approve or reject your request to merge the changes [15:42:22] *pull request [15:42:45] I have given the pull request [15:43:14] The tool is controlled by rwjuk who is known as firefly in toolforge [15:43:54] He became inactive by July 23 and I adopted that tool (toolforge only) recently [15:44:11] ah ok [15:45:40] Then unless the maintainer accepts the pull, you will need to change the git remote to your fork. There is a page on github about that. Let me find it [15:45:51] Ok [15:46:24] https://help.github.com/articles/changing-a-remote-s-url/ [15:46:40] Let me check that [15:47:06] Basically, if you change to your fork instead of the original, you can push and pull as needed [15:47:31] Let me check that [15:51:23] Yes. I am able to see my name there [15:51:29] Let me try pull now [16:45:06] !log toolserver-legacy pointing traffic to the new eqiad1-r instance [16:45:08] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolserver-legacy/SAL [17:48:44] !help pickle.dump(redirects, open("/data/project/fireflytools/www/python/src/data/long_redirects_{}.dat".format(wiki), "wb")) According to this statement, should there be a file named long_redirects_dump to exist? [17:48:44] Adithyak1997: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [17:50:32] Adithyak1997: no, "wb" mode=write binary, so the file can be unexistant and will be created by the script [17:50:49] Ok [17:53:35] I accidentally created the file(recently) named long_redirects_dump.sh. Then I deleted the file. But the files long_redirects_dump.err and .out exist. How can that be removed? [18:28:20] How can I schedule to run a py file? [18:28:24] !help [18:28:24] Adithyak1997: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [18:29:27] For instance, I need to run https://github.com/adithyak04/fireflytools/blob/master/long_redirects.py this file [18:50:19] !log toolserver-legacy shutting down relic-stretch-eqiad in anticipation of deleting it [18:50:20] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolserver-legacy/SAL [18:54:05] !help Can I schedule a py file to run every 30 minutes? [18:54:05] Adithyak1997: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [18:54:46] Adithyak1997: cron is probably the right tool for that. [18:55:10] I think these are the docs: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Grid#Scheduling_jobs_at_regular_intervals_with_cron [18:56:36] I have done the same steps. But my main doubt is whether it supports a py file [19:05:24] It supports an executable [19:06:07] So `chmod +x ` and make sure it has a shebang line at the top [19:06:07] You mean an exe file? [19:06:59] In which page do I need to mention that? [19:07:01] The first line needs to be `#!/usr/bin/python` or python3 etc [19:07:13] Page? [19:07:26] I mean the file [19:07:54] #!/usr/bin/env python is the heading of my linter_counts.py file [19:08:01] that works [19:08:44] So then you run `chmod +x linter_counts.py` to make it executable [19:09:36] jsub just needs a file that can execute. [19:10:44] That doc also mentions details about python [19:11:14] https://wikitech.wikimedia.org/wiki/Help:Toolforge/Grid#Scheduling_jobs_at_regular_intervals_with_cron [19:11:27] Because there can be quirks. [19:14:04] Let me check that [19:14:53] I wrap mine in a shell script [19:16:12] 0,5 * * * * jsub -once /data/project/fireflytools/long_redirects.py This is the statement I included in cron job [19:16:17] Ok [19:16:58] If that has issues, I wrap mine in a shell script to set the type encoding like this: [19:17:23] https://www.irccloud.com/pastebin/BuqSEcnT/ [19:17:31] Also, I'm using a virtualenv [19:17:46] If you don't have a virtualenv to worry about, you don't need that "source" stuff [19:18:00] But if you do need that, and it fails for that reason, you can use basically what I did [19:18:09] Ok [19:18:21] I am not using a virtual environment [19:18:27] Just in case it gives you trouble running as a bare python file. [19:18:31] Ok [19:19:04] Obviously, if it Just Works as is, you should be good :) [19:19:17] The encoding stuff is sometimes necessary. [19:19:49] Ok [19:26:47] I have set the time to 5 minutes. I didn't get any mail yet [19:32:48] It should at least give some kind of err message [19:32:53] in your logs [19:33:42] You have a lot of errors [19:33:51] check `linter_counts.err` [19:33:57] [Sat Nov 17 13:30:09 2018] /mnt/nfs/labstore-secondary-tools-project/fireflytools/www/python/src/templates/linter_counts.html: not an executable file [19:34:12] Wait, this is not linter_counts [19:34:17] This is long_redirects? [19:34:37] you have a python syntax error. [19:35:24] I think :) That's just from the log anyway [19:35:32] I'm heading back to the weekend [19:35:37] Good luck! [21:37:21] is it possible via database to get something like page_counter, showing hits to a certain page? [21:38:10] SQL: something like https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/Pageview_hourly? [21:38:26] TheresNoTime: <3 [21:38:45] https://meta.wikimedia.org/wiki/Research:Page_view is the public view it seems? [21:39:27] TheresNoTime: I don't think I have access to hive however heh [21:40:26] SQL: API ok? https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews