[09:12:59] <Amir1>	 Hey, a question regarding API.. First: result of wbgetentity and query is different 
[09:13:00] <Amir1>	 https://www.wikidata.org/w/api.php?action=wbgetentities&ids=Q42&props=claims
[09:13:07] <Amir1>	 https://www.wikidata.org/w/api.php?action=query&prop=revisions&titles=Q42&rvprop=content
[09:13:41] <Amir1>	 these are different (e.g. claims does not have 'datatype' in mainsnak
[09:14:36] <Amir1>	 so I want to have a consistent result but I need to query certain revisions (and not the current revision)
[09:15:32] <Amir1>	 so is it possible to make query return result as the same as wbgetentity? or make wbgetentity returns results for revisions too?
[09:21:06] <Amir1>	 DanielK_WMDE_: ^
[09:21:09] <Amir1>	 Hey :)
[09:32:34] <DanielK_WMDE_>	 Amir1: hi!
[09:33:00] <DanielK_WMDE_>	 Amir1: if the API output is missing teh datatype in value snaks, that#s a bug. please report it.
[09:33:18] <DanielK_WMDE_>	 Addshore is currently rewriting the API result generation, please poke him about it
[09:33:50] <addshore>	 *readsup*
[09:34:18] <DanielK_WMDE_>	 Amir1: getting specific revisions from getentities would be useful, and easy enough to do. Please file a feature request (and poke lydia about scheduling it)
[09:34:22] <addshore>	 Amir1: we dont really support the whole query thing at all, dont use it
[09:34:26] <addshore>	 :P
[09:34:35] <Amir1>	 Hey all
[09:34:38] <addshore>	 DanielK_WMDE_: I think there may already be a ticket somewhere!
[09:34:39] <Amir1>	 thanks :)
[09:34:55] <DanielK_WMDE_>	 oh, right, i was thinking of action=wbsearchentities. action=query is completely unsupported.
[09:35:40] <addshore>	 yeh, Amir1 query stuff will get you what is in the DB :/ whcih although it may look similar to the output of getentities sometimes youll get very old serializations etc
[09:35:48] <addshore>	 https://phabricator.wikimedia.org/T40971
[09:35:52] <DanielK_WMDE_>	 Amir1: is this for calculating diffs? an api module that would return a structural diff would still be the best solutioin I think.
[09:36:13] <Amir1>	 DanielK_WMDE_: it's not just that
[09:36:31] <addshore>	 there is aslo a ticket for that ;)
[09:36:33] <Amir1>	 the system needs to current number of claims, etc. as features
[09:36:34] <DanielK_WMDE_>	 Amir1: raw revision content, as returnsed by action=query, should be treated as an opaque blob. It is not guiaranteed to be consistent or to conform to any documented structure. 
[09:36:46] <Amir1>	 I see
[09:36:55] <addshore>	 https://phabricator.wikimedia.org/T106306
[09:37:04] <Amir1>	 (and I assume it is the same about xml dumps of Wikidata)
[09:37:11] <addshore>	 Amir1: yup
[09:37:16] <DanielK_WMDE_>	 Amir1: yes indeed. use the json dumps
[09:37:53] <DanielK_WMDE_>	 Amir1: allowing individual revisions to be queries from wbgetentities should not be hard. 
[09:38:15] <addshore>	 indeed!
[09:38:21] <Amir1>	 yup
[09:38:22] <Amir1>	 https://phabricator.wikimedia.org/T40971
[09:38:33] <DanielK_WMDE_>	 addshore: the result would then be keyed by revision id, not entity id. that
[09:38:41] <DanielK_WMDE_>	 ...that's a bit odd, but not terrible, i think
[09:38:47] <addshore>	 mhhhhm
[09:38:49] <Amir1>	 There was a patch by legoktm to implement such feature
[09:38:51] <DanielK_WMDE_>	 but the two modes should not be mixed
[09:39:00] <addshore>	 Amir1: yeh I thought I remembered something like that!
[09:39:04] <multichill>	 Amir1: Was looking for you
[09:39:12] <Amir1>	 multichill: hey :)
[09:39:15] <multichill>	 Have you ever done something with word distances?
[09:39:40] <Amir1>	 do mean word distances in NLP?
[09:39:54] <multichill>	 We have a list of painters that have an item and a list of painters that are not matched yet (https://tools.wmflabs.org/multichill/queries/wikidata/top_unmatched_painters.txt)
[09:40:11] <multichill>	 Finding matches between the two of them. All exact matches have already been covered
[09:40:16] <Amir1>	 I did several similar works before
[09:40:58] <Amir1>	 multichill: before I start
[09:41:23] <Amir1>	 Do you want to find matches between this list or this list and members of another list?
[09:41:29] <Amir1>	 (I just want to be sure)
[09:43:23] <Amir1>	 multichill: ^
[09:43:27] <multichill>	 List A : http://tools.wmflabs.org/autolist/autolist1.html?q=CLAIM%5B31%3A5%5D%20AND%20CLAIM%5B106%3A1028181%5D (all the painters we currently have on Wikidata)
[09:43:36] <multichill>	 List B : https://tools.wmflabs.org/multichill/queries/wikidata/top_unmatched_painters.txt
[09:43:46] <multichill>	 Possible matches between List A & B.
[09:44:33] <multichill>	 Amir1: For list A I could probably do a database query to get all labels and aliases in all languages so more chance of finding something
[09:44:40] <Amir1>	 Is there a way to download English labels?
[09:44:56] <Amir1>	 multichill: that would be really good
[09:45:16] <multichill>	 I just do a database query and I except I might have some weird stuff in there
[09:45:51] <Amir1>	 it's okay
[09:45:54] <Amir1>	 I will handle that
[09:46:59] <multichill>	 Amir1: The query is something like. Give me every pages that links to painter and give me all the labels and aliases for this page
[09:47:30] <multichill>	 It will contain some wrong items (subclass of painter, etc), but that's a small number and it needs human verification anyway
[10:08:55] <multichill>	 Amir1: Is this something you could work with?
[10:09:11] <multichill>	 addshore: Didn't you have a bot or something to find failed merges like https://www.wikidata.org/w/index.php?title=Q18565822&action=history ?
[10:09:24] <Amir1>	 Of course 
[10:09:38] <addshore>	 multichill: nope :/
[10:09:40] <Amir1>	 I was hoping you gave me the list and I start working on it
[10:09:56] <addshore>	 I have a script which restores deleted items that were merged and creates a redirect
[10:10:11] <multichill>	 Right, that's probably the one I'm mixing it up with
[10:10:51] <multichill>	 Will do Amir1. Is the form <wikidata Q id> - <label/alias> suitable for you? The same Q id will be in the list multiple times, but I'll sort it by Qid
[10:11:11] <Amir1>	 That would be good
[10:11:39] <addshore>	 multichill: such a bot shouldnt be that hard to create though
[10:12:07] <multichill>	 Or just a database query. Almost empty items with last revision containing "blaldd die bla merge"
[10:12:17] <addshore>	 find all summaries that say this item has been merged to somewhere else, load the history of that item and make sure there is a revision saying the merge worked
[10:12:26] <addshore>	 multichill: yeh that would also work!
[10:12:43] <addshore>	 or infact, any item that has one of those summaries that isnt a redirect and is not deleted
[10:12:52] <addshore>	 [=
[10:14:18] <multichill>	 addshore: And yet another item for the todo list :P
[10:14:25] <addshore>	 yup :/
[10:27:36] <sjoerddebruin>	 multichill: Most failed merges are listed here. https://www.wikidata.org/wiki/Special:ShortPages
[12:52:16] <multichill>	 Amir1: Running the query now. That might take a while :-)
[12:52:51] <Amir1>	 oh cool :)
[12:57:53] <Lydia_WMDE>	 Jonas_WMDE: https://phabricator.wikimedia.org/T109800
[13:02:56] <multichill>	 Amir1: https://tools.wmflabs.org/multichill/queries/wikidata/painters_on_wikidata.sql will make the database cry, it will pop up at https://tools.wmflabs.org/multichill/queries/wikidata/painters_on_wikidata.txt at some point
[13:03:39] <Amir1>	 :D
[13:03:41] <Amir1>	 Okay
[13:03:57] <Amir1>	 I check it and start to work once it's there
[13:08:00] <multichill>	 Other list is at https://tools.wmflabs.org/multichill/queries/wikidata/top_unmatched_painters.txt
[13:11:23] <Amir1>	 I downloaded it
[13:13:22] <addshore>	 hoo: :(
[13:13:38] <addshore>	 I think I'm going to increase the size of the json dump again :/
[13:15:39] <JeroenDeDauw>	 addshore: what are you raging about?
[13:15:40] <hoo>	 addshore: Uhm... yay?
[13:16:01] <addshore>	 hoo: firstly https://phabricator.wikimedia.org/T109830 *facepalm*
[13:16:30] <addshore>	 secondly https://gerrit.wikimedia.org/r/#/c/227686/18 Note: ALL output may now give empty elements, ie. labels when an entity has none ... and I just remembered the json dumper uses this result builder thing
[13:17:05] <addshore>	 I mean, compressed the size shouldnt be that much bigger, but I imagine extracted it will be a bit!
[13:17:20] <addshore>	 I was going to check the current dumps but, will take 40 mins for me to download, yay
[13:17:44] <addshore>	 JeroenDeDauw: https://github.com/wmde/WikibaseDataModelServices/pulls
[13:19:25] <JeroenDeDauw>	 addshore: that confuses me
[13:19:46] <JeroenDeDauw>	 this means code on master says it throws exceptions that do not exist?
[13:20:05] <addshore>	 JeroenDeDauw: yes, but it is only an interface, there is no implementation in there to be tested
[13:20:41] <JeroenDeDauw>	 addshore: hey, no discrimintating of interfaces! They are code too you know!
[13:20:48] <addshore>	 :P
[14:07:49] <multichill>	 Amir1: Done 56m29.653s / 159894 lines
[14:34:58] <JeroenDeDauw>	 Lydia_WMDE: you where looking for me?
[14:39:33] <Amir1>	 multichill: thanks :) I start working
[14:40:46] <multichill>	 Amir1: Great! Quite a big data set to crunch. I wonder what kind of fun stuff you'll find
[14:41:11] <Amir1>	 I found one of already Q207 George Bush :D
[14:41:31] <Amir1>	 He paints puppies
[14:45:04] <multichill>	 Amir1: He's not in https://tools.wmflabs.org/multichill/queries/wikidata/top_unmatched_painters.txt so how can you find it?
[14:45:31] <Amir1>	 I found it at top of current ones
[14:45:49] <multichill>	 Of course, Hitler is also in there somewhere
[14:51:26] <multichill>	 Amir1: Especially older European painters have a lot of variantions in the name.
[14:51:53] <Amir1>	 multichill: good comparison :D
[14:52:28] <Amir1>	 I'm looking at the libraries to find out best approach
[14:54:25] <multichill>	 Amir1: Something like https://pypi.python.org/pypi/python-Levenshtein/ was my first guess
[14:54:49] <multichill>	 But it sucks that you would have to do a full matrix calculation. That's over a billion operations :-(
[14:55:14] <Amir1>	 no, libraries like nltk can handle that
[14:56:02] <multichill>	 https://github.com/seatgeek/fuzzywuzzy looks nice too :-)
[14:56:14] <multichill>	 "https://github.com/seatgeek/fuzzywuzzy"
[14:56:19] <multichill>	 ahum "Fuzzy string matching like a boss."
[22:28:45] <ankry>	 Hi
[22:29:03] <ankry>	 I have problem logging with my bot account to Wikidata
[22:29:09] <ankry>	 any hints?
[22:29:28] <ankry>	 The bot is centrally logged, but not on Wikidata
[22:29:49] <hare>	 ankry: what's your bot's user name?
[22:29:50] <ankry>	 account on wikidata created 15 min ago
[22:29:54] <ankry>	 AkBot
[22:30:04] <ankry>	 hare: ^^^^
[22:30:28] <hare>	 so, the wikidata account exists as you said. when was the last time you tried having your bot log in?
[22:30:43] <ankry>	 2 min ago
[22:31:01] <ankry>	 I may try once again to relogin
[22:32:32] <ankry>	 hare: after providing username & pass I am still not logged in
[22:32:47] <ankry>	 next login attemts gives me:
[22:32:49] <ankry>	 Central login
[22:32:49] <ankry>	 You are centrally logged in. Reload the page to apply your user settings.
[22:32:56] <hare>	 That means you're logged in
[22:33:08] <ankry>	 and after reload I am still  not logged in
[22:33:24] <ankry>	 I edit as IP
[22:33:41] <ankry>	 while logged centrally
[22:33:59] <hare>	 Interesting. I hope you figure it out! (I am out of ideas, sadly.)
[22:34:19] <ankry>	 phabricator ?
[22:56:04] <ankry>	 hare: seems to me that I hit: https://phabricator.wikimedia.org/T109038
[22:56:15] <ankry>	 cleaning old cookies helped
[22:59:14] <Harmonia_Amanda>	 wow fr:wikipedia really don't like wikidata O_o (some users because of course, there are many French Wikipedians who are also Wikidatians)
[22:59:20] <Harmonia_Amanda>	 https://fr.wikipedia.org/wiki/Discussion_mod%C3%A8le:Wikidata_list/Suppression