[01:54:32] @zhuyifei1999_: sorry my latest upload says upload from 'tools.sibutest'; will fix it next time. [01:56:21] Here https://github.com/infobliss/sibutest2/blob/master/OOP/GenericGLAM.py#L28 [01:57:50] I am passing the username which is 'Infobliss' as printed by https://github.com/infobliss/sibutest2/blob/master/OOP/GenericGLAM.py#L21 [01:58:32] But pywikibot returned the error "User 'None' does not have upload rights on site commons:commons." [06:25:39] infobliss: where did you set oauth info? [06:31:00] a single username isn't enough [08:36:23] hi zhuyifei1999_ [08:36:41] hi [08:36:47] see logs [08:37:08] Earlier while using UploadRobot I didn't have to separately set Oauth info [08:37:33] o.O [08:37:44] how do you login? [08:37:57] without password or oauth [08:38:00] ok wait [08:38:50] https://github.com/infobliss/sibutest2/blob/master/app.py#L30 [08:39:20] yeah and? [08:39:36] pywikibot doesn't know about the stuffs in flask [08:39:50] unless you tell pywikibot explicitly [08:40:43] so what else is needed other than username for site.upload()? [08:41:05] login with oauth [08:41:46] I mean let pywikibot know about the tokens and login to commons [08:41:51] with oauth [08:43:18] I think UploadRobot did all these on it's own [08:43:34] extracting username and tokens from the session [08:43:42] it won't [08:44:18] pywikibot never assumes the existence of flask [08:44:54] you probably used password login a long time ago and pywikibot cached the creds [08:44:55] ok [08:45:10] no many others have tested [08:45:22] including the rep from NA glam [08:45:29] and Srishti [08:45:39] then you probably removed some relevant code [08:45:51] she has shared screenshots too [08:46:07] yeah may be [08:47:10] honestly, I still don't get the layout of your repo [08:47:45] and idk how to find which commit is responsible for the removal given this layout [08:48:15] all code are committed from ~/www/python/src [08:48:49] eg. https://github.com/infobliss/sibutest2/search?utf8=%E2%9C%93&q=upload_file&type= 5 files are doing the upload [08:50:05] lol [08:50:20] OOP/GenericGLAM.py is the current upload module [08:50:44] which one is the one used at the time? [08:51:02] NationaalArchief2.py [08:51:57] https://github.com/infobliss/sibutest2/blob/master/NationaalArchief2.py#L234 [08:52:16] also I don't find how editing the files on tool labs is any more comfortable than editing locally and commit [08:53:00] yeah it's difficult to edit in toollabs [08:53:13] but for the sake of testing I am doing it there [08:53:40] then commit and pull. It's also easier for us to see the progress [08:54:17] I am editing and committing [08:54:28] editing on tool labs = no auto linting [08:54:39] yeah [08:54:45] very cumbersome [08:55:13] anyways the login code of that file is https://github.com/infobliss/sibutest2/blob/master/NationaalArchief2.py#L234 [08:55:20] but can I do Oauth integration and testing from local machine? [08:55:35] yeah that's it. thanks for that :) [08:55:57] yes, apply for an new oauth consumer [08:56:29] only you can use it when it isn't approved [08:56:57] yeah great. ok let me do that. [10:51:03] hi zhuyifei1999_ [10:51:26] thanks a lot for suggesting to propose a new consumer [10:51:32] np [10:51:41] I am able to load the flask app from local machine now [10:51:55] but there's some import related things [10:51:58] for e.g [10:52:25] I have a file OOP/NationaalArichiefGLAM.py [10:52:34] which I import from app.py [10:53:02] similarly GenericGLAM.py [10:53:06] as from OOP.GenericGLAM import GenericGLAM [10:53:22] But here it says import error no module found [10:53:32] are you using python 3? [10:53:40] python 2 [10:53:43] 2.7 [10:53:46] and are you on master? [10:53:52] yes [10:53:58] hmm [10:54:31] what's the command you started app.py? python app.py? [10:54:39] no [10:54:41] inside the repo [10:54:46] ? [10:55:46] a sec [10:56:42] $ export FLASK_APP=app.py $ python -m flask run [10:57:13] * zhuyifei1999_ never heard about ^ [10:57:39] http://flask.pocoo.org/docs/0.12/quickstart/ [10:59:00] try from .OOP.GenericGLAM import GenericGLAM [10:59:13] switch to relative import [10:59:13] ok [10:59:28] I can't guarantee it working though [10:59:49] usually I test within vagrant [11:00:26] but it doesn't work for v2c due to labs being too complex and cv2c is too dependent on labs environment [11:00:41] let me see [11:00:50] a sec [11:01:33] no luck [11:01:35] ValueError: Attempted relative import in non-package [11:01:49] (so I have a video2commons-test tool that's a duplicate of the production tool) [11:01:52] uh [11:02:52] well I copied these files from OOP to the home folder where app.py resides [11:03:01] ? [11:03:15] and then importedas "from GenericGLAM import GenericGLAM" [11:03:25] this works fine [11:03:34] oh I think I know why [11:03:56] https://github.com/infobliss/sibutest2/blob/master/OOP/__init__.py exists right? [11:04:18] omg no [11:04:28] how come?? [11:04:28] then you are not on master [11:04:39] I pulled from the master :( [11:04:52] did you merge? [11:04:56] but looks like some files are missing [11:05:01] no not merged [11:05:37] actually, can you give me your folder? [11:06:01] ok [11:06:07] I think you screwed up somewhere [11:06:15] I did not merge [11:06:31] that's what is the issue [11:06:34] I think [11:06:44] you should not merge [11:06:54] Ow [11:07:07] wait I will share my folder [11:07:16] actually, give me `git status` and `git log` first [11:07:29] the first few lines of `git log` [11:08:34] ok wait [11:09:27] is it under .git/logs? [11:09:48] no [11:09:58] $ git log | head [11:10:11] ok [11:11:55] https://justpaste.it/19adv [11:12:54] this says 18 Jul :( [11:13:11] though I had issued git pull command [11:14:29] you are on https://github.com/infobliss/sibutest2/commit/334c227379a76a3cb6f9326b86667583146facc5 [11:14:38] a fast-forward merge is okay [11:15:23] but if you see any non-fast-forward merge, inspect the situation first [11:17:06] uh I have to read what these merges are [11:17:23] ? [11:18:11] fast and non-fast-forward merge etc [11:18:16] but anyways [11:18:24] non-fast-forward merge = a -> b -> d and a -> c -> d wants to happen together [11:18:25] My imports are working now [11:18:37] d must be a merge [11:18:50] alright [11:19:27] fast-forward merge = a -> b -> c and a -> b wants to happen together [11:19:43] the HEAD gets updated to c [11:19:59] rebase = a -> b -> d and a -> c -> d wants to happen together [11:20:14] result = a -> b -> c -> d [11:20:22] uh wait [11:20:33] result = a -> b -> e -> f [11:21:22] e and f are like cherry-picks, same diff and different parents and timestamps [11:21:27] *but different [11:21:58] ok [11:22:53] merge usually make least disruption (it won't cause pulling / pushing issues, since the commit sha1s are the same), but can cause a lot of mess [11:23:18] rebase will make the repo cleaner (less branching) [11:23:43] ok [11:24:31] but if the rebased commit has never been pushed, rebase won't do much evil, (it's like git commit --amend, you can do as many amends as you want until you push) [11:25:33] ok [11:25:54] I get the OAuth exception on running the flask app [11:26:01] ? [11:26:07] https://justpaste.it/19adv [11:26:30] don't do owner only [11:26:36] make a regular [11:26:58] 16:56 only you can use it when it isn't approved [11:27:11] oh [11:27:32] regulars have a expiry until nobody can use it [11:27:43] if it's not approved [11:28:00] but owner-onlys work forever [11:28:12] it's like a testing period [11:28:38] but why is owner-only giving exception? [11:29:12] because owner-only have a different workflow [11:29:29] a sec [11:31:13] the consumer tokens and access tokens are constant and there is no request tokens [11:31:34] yeah [11:31:38] but regulars have many access tokens and request tokens [11:32:16] also for owner-only consumer I got 4 tokens [11:32:21] whereas earlier for regular I got only 2 [11:32:33] each oauth/initiate generate a pair of request tokens [11:32:47] no not that [11:33:09] and oauth/complete generate a pair of access tokens [11:33:18] ^ for regulars [11:33:18] the "OAuth consumer registration" page gave the following [11:33:40] ok I got you :) [11:34:02] but for owner only oauth/initiate don't make sense as there are no request tokens, so the workflow make no sense [11:34:04] k [11:34:47] hmm that was good enlightenment :) [11:34:53] thanks [11:34:56] np [11:35:48] but then what is the use of owner only? [11:35:56] if testing is not possible [11:36:03] bots [11:36:41] some bots use oauth for login [11:37:01] this way you can finely control what rights should be given to the bot [11:37:12] a bot is just a program right? [11:37:45] https://en.wikipedia.org/wiki/Wikipedia:Bots [11:38:54] bots usually have a limited wiki accounts (usually 1), unlike tools which can have an infinite wiki accounts [11:39:10] ok [11:39:31] bots are essentially using their own logins, but tools are using the logins of their user [11:39:59] I run 6 bots https://en.wikipedia.org/wiki/Wikipedia:Bots [11:40:03] uh wrong link [11:40:11] https://commons.wikimedia.org/wiki/User:Zhuyifei1999 [11:40:23] click the "userboxes" [11:41:14] (yeah, I think I run the most bots of all commons bot operators ;) ) [11:41:40] * infobliss is amazed [11:42:07] 4 of them are based on pywikibot [11:43:09] * infobliss completed reading your userboxes [11:43:16] pretty long :) [11:43:27] ... [11:43:41] there are userbox collecters out there [11:44:02] theirs can be like a dozen times longer :) [11:44:09] lol [11:44:48] "This user may not respond swiftly to queries." <= Nobody responds quicker than you ;) [11:45:17] meh [11:49:51] https://commons.wikimedia.org/wiki/User:FlickreviewR_2 almost 4 million edits [11:50:56] 0mg [11:51:07] also, https://commons.wikimedia.org/wiki/Special:Log/delete/Embedded_Data_Bot <= adminbot here (/me is proud of this one) [11:53:09] anyways, this Embedded_Data_Bot uses oauth for login [11:53:17] great one [11:53:31] owner only? [11:53:41] yes [11:54:21] ok [11:54:42] so I will be afk for a while [11:55:00] the code will only run under this user (and other admin account), unlike tools that has a big "login" button so anyone can login and run the code under their account [11:55:13] hm [11:55:37] that's what owner-onlys are for [11:56:20] right got it [11:56:54] btw, https://commons.wikimedia.org/wiki/Commons:Bots/Secure_login [11:57:06] ^ why we use oauth/botpasswords [11:57:34] botpasswords is a randomly generated (by mediawiki) password that can only be used in the api [11:58:07] and you can specify the grants [11:58:31] like an owner-only oauth consumer [12:06:25] btw, '19:44 "This user may not respond swiftly to queries." <= Nobody responds quicker than you ;)' <= because I'm mostly on holiday :) [14:00:07] @ zhuyifei1999_ : sweet holidaying life :) [14:00:19] :) [14:02:11] * zhuyifei1999_ just turned a running-forever mysql query into a <1 second one [14:02:37] cheers !!! [14:02:52] in case you are wondering: [14:03:02] select page_namespace, page_title from templatelinks inner join page on page_id =tl_from where tl_namespace = 2 and tl_title = "Avicennasis/emptycat" and not page_title in (select cl_to from categorylinks); [14:03:10] original query ^ [14:03:20] select page_namespace, page_title from templatelinks inner join page on page_id =tl_from where tl_namespace = 2 and tl_title = "Avicennasis/emptycat" and not page_title in (select * from (select cl_to from categorylinks) all_cats); [14:03:24] attempt 1 ^ [14:03:35] select page_namespace, page_title from templatelinks inner join page on page_id =tl_from where tl_namespace = 2 and tl_title = "Avicennasis/emptycat" and not exists (select 1 from categorylinks where cl_to = page_title); [14:03:39] attempt 2 ^ [14:04:01] all these materializes, which is bad in this case [14:04:09] select page_namespace, page_title from templatelinks inner join page on page_id =tl_from left join categorylinks on cl_to = page_title where tl_namespace = 2 and tl_title = "Avicennasis/emptycat" and cl_from IS NULL; [14:04:17] attempt 3 saved the day [14:04:25] didn't materialize [14:05:25] ok [14:05:27] good [14:05:58] where was this code? [14:06:05] one of my bots [14:06:10] a very old one [14:06:28] one among the 6? [14:06:47] uh, an old task of the first bot [14:06:59] ok [17:57:57] hi [17:58:32] * zhuyifei1999_ spoke a lot of off-topics today [17:59:05] I've read the discussion, on local testing, nice if that is going to work [19:22:52] ok I'm off for today. Infobliss could you give a notice if you can't make it? 15:00 sundays, 20:00 weekdays were what we were aiming for (both Amsterdam time, so +3:30 for you)