[10:08:32] hi [11:46:54] hi basvb [11:47:05] were you looking for me? [11:47:41] not per se [11:47:51] ok [11:47:51] I'm reformatting amsterdamMuseum [11:48:01] I made some changes to genericglam and gen_lib [11:48:14] ok let me check [11:48:15] almost done [11:48:20] I'll push in 10 min [11:48:32] ok [11:48:35] well it's done, but have to test it now [11:48:44] ok [11:49:55] Anyways you can commit then I will be able to merge [11:50:18] and test too [11:54:23] it will imply some changes to your work likely as well [11:54:48] figured out some kind of flow for handling the uploads, but zhuyfei should check whether it is proper way to do it as well [12:02:24] sounds good [12:07:44] ok it's getting pushed [12:08:00] only have to do the searchstring to ids [12:11:31] zhuyifei1999_: could you give https://github.com/infobliss/sibutest2/blob/master/Glam_mappings/AmsterdamMuseum.py a quick review [12:11:52] I think I'm currently doing the forwarding of parameters still wrong [12:12:01] he may be afk [12:12:54] for example https://github.com/infobliss/sibutest2/blob/master/Glam_mappings/AmsterdamMuseum.py#L310 I should not have "data" as a parameter of that function but rather just call self.data everywhere in that function? [12:13:00] yep infobliss he can answer later [12:15:30] But is data defined as a class variable? [12:17:13] as a variable of self [12:17:23] self.data in __init__ [12:17:35] ok [12:26:35] so infobliss the idea now is that you first initialize this class [12:26:53] e.g. image= AmsterdamMuseum(214) [12:27:15] if that returns false something is wrong and upload not possible for that id [12:27:35] license_checker is called in the init [12:27:36] priref is int? [12:28:16] can be either int or string [12:28:19] it's a number [12:28:30] but I typecast it to a string in the function [12:29:11] after that you can call generate_image_information(list_of_categories) [12:29:22] and it returns a title, wikitext and image_url [12:29:50] for the multiple image case you first call: search_to_ids() [12:29:55] (have to write that still [12:30:05] search_to_ids('search') [12:30:09] it returns a list of ids [12:30:27] for all of these you initialize first the Amsterdammuseum(ID) [12:30:49] then you can do Amsterdammuseum(ID).get_thumbnail [12:30:55] it gives a thumbnail [12:31:03] you can specify the size of it [12:31:28] after you get the thumbnails you can show these to the user [12:31:37] the user selects some images [12:32:10] for those image you can loop over Amsterdammuseum(ID).generate_image_information(list_of_categories) [12:32:33] and the uploading of those should be outside of here [12:32:41] I saw you did uploading in GenericGlam? [12:32:48] I'm not sure if that is the correct location [12:33:43] do you prefer creating a separate uploader class? [12:34:49] I'm not sure what's best I think genericglam should be a template which has to be implemented for each glam [12:34:56] upload always works the same [12:35:03] given a title, url and wikitext [12:35:29] please also compare https://github.com/infobliss/sibutest2/blob/master/libraries/GenericGLAM.py#L62 to https://github.com/infobliss/sibutest2/blob/master/libraries/gen_lib.py#L47 [12:35:40] imo generic functions can be part of GenericGLAM [12:35:48] uploading is a generic function [12:36:04] I think I disagree, it's not a generic function related to glams [12:36:08] it's just a generic function [12:36:34] none of the glams will/should override the upload function [12:36:54] where-as the other functions there can be overwritten [12:37:15] right [12:37:25] but we can ask zhuyifei1999_ about that as well [12:37:30] Then inside GenericGLAM there's not much action [12:39:36] true [12:39:49] but it defines a structure which you can build on with other code [12:39:58] each glam has to adhere to that structure [12:40:41] that's correct [12:41:19] I was wondering will it affect us at all if we do away with GenericGLAM [12:41:29] Everything will still work [12:41:46] Isn't it? [12:42:09] http://www.jesshamrick.com/2011/05/18/an-introduction-to-classes-and-inheritance-in-python/ [12:42:18] please read that [12:42:47] that explains the purpose of classes and inheritance much better than I can do it [12:43:12] sure [12:44:00] I think https://github.com/infobliss/sibutest2/blob/master/libraries/gen_lib.py#L47 can be part of the genericClass [12:46:45] ok yes one more thing [12:47:10] In NA we assumed all images to be of photograph infobox type [12:47:20] In AM I see all are artwork [12:47:23] Am I right? [12:47:56] sorry [12:48:00] art_photo [12:49:52] yes [12:50:02] that's a decision on the glam-side [12:50:13] can also be multiple different infobox-types for one glam [12:50:49] but if you construct the infobox in the glam-specific-code you can use https://github.com/infobliss/sibutest2/blob/master/libraries/gen_lib.py#L47 to transform it to a proper wikipage [12:53:54] yeah this is done inside fill_template now [12:54:27] can be replaced by this separate function [12:54:52] yes, but that code is a bit of a mess, fill_template [12:55:31] it doesn't work for all kinds of infoboxes [12:55:42] https://github.com/infobliss/sibutest2/blob/master/libraries/GenericGLAM.py#L75 that for example is an issue [12:56:07] that parameter is not used for all infoboxes I believe [12:57:33] ok [12:58:45] I think now that AM GLAM mapping is finished I can start merging this to the tool [12:59:27] or may be zhuyifei1999_ can give his review and then I proceed [13:01:17] yes [13:36:04] ok I added the searchstring to identifiers function [13:36:21] not yet on the correct location I think (we were discussing it as static before) [13:36:36] also it does not support getting more than 100 results (it just gives the first 100) [17:45:28] sorry I went to sleep like 4pm [17:49:44] looks so much sensible than the older version :) [17:50:26] https://github.com/infobliss/sibutest2/blob/master/Glam_mappings/AmsterdamMuseum.py#L359 classmethod / staticmethod [17:50:41] I don't think importing the function directly is sane [18:24:31] hi zhuyifei1999_ thanks for the remarks [18:24:32] yep, will do that as a classmethod as discussed [18:25:11] the other thing is that a lot of functions (example https://github.com/infobliss/sibutest2/blob/master/Glam_mappings/AmsterdamMuseum.py#L214) the IDE mentions that they could be static methods [18:25:22] I guess I shouldn't be sending date but rather get that from self.data?/ [18:25:39] sending start and end in this case [18:25:46] yeah [18:26:16] ok those 2 were my main doubts [18:26:37] once I suggested then instead of passing a dict to the str.format(), we could pass the glam object [18:26:53] so make a @property date [18:27:02] then you can {obj.date} [18:27:19] sorry I don't follow [18:27:33] you wouldn't need a super large dict builder [18:27:41] it's okay [18:28:06] is that about self.parameters (parameters for infobox template) [18:28:41] and for these date @property (instead of a static parse_date), fetch data from cached api response [18:29:07] yeah [18:29:17] but why is that an instance property? [18:30:11] sorry, my terminology is not that good as such I don't really get what you try to bring accross still [18:30:25] def parse_date(self): [18:30:30] a sec [18:30:38] I'll make some dummy code [18:30:57] and then I replace start with self.data['production.date.start'][0] [18:31:09] that's what I have to do to make these not static [18:38:13] https://www.irccloud.com/pastebin/7zWT5Uh6/ [18:38:30] basvb: ^ [18:39:28] I don't require this though [18:39:43] I simply feel like it's more manageable [18:40:43] so date is set as a property of the glam object [18:40:52] yes [18:40:58] and by doing glam_object.date it's retrieved [18:41:00] ok get ti [18:41:01] it [18:41:40] for the next mapping ;) [18:41:57] a.k.a. forever [18:42:01] :P [18:44:14] basvb : Why cap at 100? https://github.com/infobliss/sibutest2/blob/master/Glam_mappings/AmsterdamMuseum.py#L366 [18:45:27] http://amdata.adlibsoft.com/wwwopac.ashx?database=AMcollect&q=amsterdam&limit=100&output=json [18:45:31] I set the limit to 100 [18:45:39] you can up it, loading time suffers from that [18:45:49] and I doubt often more than 100 images are to be uploaded [18:46:07] depending on what you want to implement the images can be requested per 100 [18:46:22] how many are you going to show on a page? [18:47:11] the function can be optimized, more than 100 has to be implemented also it's possible to only load the priref field (quicker loading) and continue the search, also search string with multiple terms has to be implemented [18:47:18] yeah alright. Once we have pagination this upper limit may be removed right? [18:47:20] but for now it has the basic functionality [18:47:30] no you don't want to remove the limit [18:47:36] you want to set it at the pagination amount [18:47:53] and then use continue=id (or something) to request the ids for the second page [18:48:01] theres 5000 hits of amsterdam [18:48:11] you don't want to load 5000 I believe [18:48:38] If I do http://amdata.adlibsoft.com/wwwopac.ashx?database=AMcollect&q=amsterdam&limit=1000&output=json it starts to crash, not sure if that's from the json loading in the browser or server-side [18:50:43] yeah true it's loading forever [18:52:14] oh it gave the json [18:52:39] took close to 2 minutes [18:54:55] so not feasible if it's server side [18:55:42] hm [18:55:45] I'm sorry but I have to finish some stuff the last two hours of the weekend, see you tomorrow (I'll be checking here once or twice but not active participation) [18:55:59] np [18:56:09] let me merge this code by today [18:56:25] yep If you worked on the same files that might be difficult