[07:36:15] Wikidata is still at 8. :( [09:02:38] https://phabricator.wikimedia.org/T47839 [09:35:23] https://phabricator.wikimedia.org/T47839 [09:35:29] aude: ^ [10:51:26] aude: everything fine with the script? [10:51:30] can I run the bot then? [11:01:42] benestar: i think so [11:01:58] maybe do a trial run for one wiki, to see the badges appear in the special page [11:02:24] makes sense :) [11:05:50] meh, I should've really implemented an entity lookup based on the db :S [11:06:02] benestar: i don't think it's possible on labs [11:06:13] why do you need an entity? [11:06:20] isn't it? I think I can access the revision table? [11:06:30] aude: I need to get the badges for the sitelinks ;) [11:06:30] that doesn't have text blobs [11:06:58] afaik, all you need ot know is does the site link have a badge [11:07:06] true or false [11:08:03] aude: yes, but for that I need to load the entire entity [11:08:05] * aude thinks [11:08:32] probably using lookup by api is good enough though [11:09:28] ah, all you need are the badge items [11:10:53] * aude looks at the database [11:12:57] select page_title from pagelinks left join page on (page_id=pl_from) where pl_title = 'Q17437796' limit 10; [11:13:05] then you have a list of item ids [11:13:28] * aude would then query items_per_site [11:13:49] select page_title from pagelinks left join page on (page_id=pl_from) where pl_title = 'Q17437796' and page_namespace = 0 [11:15:59] aude: did you really look at my code? :P [11:16:04] that's exactly what I'm doing ;) [11:16:15] i glanced at it [11:17:15] but you are nto using items per site? [11:17:54] nope [11:18:02] that doesn't contain badges [11:18:07] doesn't matter [11:18:15] and I don't want to purge all pages which are linked to a page that has a badge [11:18:22] * aude would :) [11:18:28] i would need to purge about 10 times more pages ;) [11:18:33] oh really? [11:18:43] perhaps even more [11:19:00] * aude wonders how so? [11:19:35] i'd need to purge all pages listed at https://www.wikidata.org/wiki/Q64 [11:19:56] and only ~10 of them have badges and actually need to get purged [11:19:58] ah [11:20:15] makes sense [11:20:26] how many items can I load at one time using wbgetentities? [11:20:36] i think 50? [11:20:36] are 100 too many? [11:20:49] maybe more if you have bot rights or logged in [11:20:52] but think 50 is ok [11:21:09] * aude thinks your script looks good :) [11:21:13] :) [11:21:18] will load it in chunks of 100 [11:21:21] while logged in [11:21:26] ok [11:21:44] Maximum number of values is 50 (500 for bots). [11:23:21] aude: where did you get that from? [11:24:19] https://www.wikidata.org/w/api.php?action=help&modules=wbgetentities [11:24:38] ty [11:24:41] will do 500 then [11:24:49] * aude wonders if it will time out :O [11:24:58] you can try :) [11:25:05] let's see ;) [11:25:08] k [11:25:34] https://fr.wikipedia.org/wiki/Grand_Prix_de_Fourmies_2015 is such a nice infobox :) [11:31:09] Only needs references. :O [11:33:20] we don't put references in the infobox on fr: [11:33:34] aude: worked with 500 though the api request took veeery long [11:33:39] still faster than doing it one by one ^^ [11:34:04] * benestar wonders if dump processing would've been a better idea [11:34:06] * aude would get ~100 at a time [11:34:20] but whatever works :) [11:35:35] 500 of 70602 loaded \o/ [11:35:40] wow [11:35:47] will take some time now... [11:35:54] no hurry [11:36:06] also I first load all of them and then process them per wiki [11:36:22] another idea https://www.wikidata.org/wiki/Q4387444 [11:36:22] hopefully the script won't die inbetween [11:36:34] get all category members for all such categories [11:36:50] hopefully it is in sync with wikidata badges [11:36:55] oh yes, that would've been much simpler -.- [11:36:59] aude: it isn't [11:37:08] but I'm creating a bot which fixes that [11:37:10] yeah :/ [11:37:17] but need the page props first to do proper joins [11:37:40] hm [11:43:05] !nyan [11:43:06] ~=[,,_,,]:3 [11:44:36] it's a JeroenDeDauw ;) [11:44:48] !nyan [11:44:48] ~=[,,_,,]:3 [12:20:27] aude: almost all badges fetched [12:20:37] will start purging soon [12:20:48] \o/ [13:41:00] DanielK_WMDE: Jonas_WMDE https://en.wikipedia.org/wiki/Fragment_identifier :) [13:41:11] http://www.w3.org/Addressing/URL/4_2_Fragments.html [13:46:31] hi there I have a question [13:46:40] I created item:https://www.wikidata.org/wiki/Q20965808 [13:46:50] but I amunable to save my Wikipedia ink on that page [13:46:53] https://en.wikipedia.org/wiki/More_Than_(company) [13:46:57] why is that? [13:48:07] Alfalf007: that page is already linked on another item https://www.wikidata.org/wiki/Q20713193 [13:48:18] but think the item could be merged into the one you link [13:48:37] I see [13:48:40] can we please merge them two [13:48:48] I must have created the previous item and forgotten [13:49:04] looks like you have to ask an admin [13:49:42] where is the link for that? [13:49:59] maybe https://www.wikidata.org/wiki/Wikidata:Administrators%27_noticeboard? [13:51:15] I think that page I created today has been already deleted [13:51:21] so thanks for your help [13:51:22] bye [13:51:25] ok [14:15:10] where can i find the description of wikidata db tables? [14:40:35] I think these two entries should be merged but not sure? https://www.wikidata.org/wiki/Q116269 + https://www.wikidata.org/wiki/Q15724995 [15:26:31] addshore: still around? [15:26:36] yup [15:27:02] addshore: any reason MWTimeIsoParser did not get moved to datavalues-time yet? [15:27:13] * @author Adam Shorland [15:27:13] * [15:27:13] * @todo move me to DataValues-time [15:27:15] because its is heavily bound to the language stuff [15:27:25] I did make a PR like a year and a half ago but it never got merged [15:27:46] addshore: ah, so we first need a proper abstraction of the mediawiki language stuff, right? [15:27:56] or I might have had a pacth in Wikibase refactoring it and removing the binding to Language [15:29:08] addshore: but as far as I can see all the parsing stuff is only used in repo, right? [15:29:14] yup [15:29:17] afaik [15:29:53] addshore: so I'll remove it from lib \o/ [15:30:06] ohhhhh, well, make sure its not in client anywhere ;) [15:31:01] addshore: :PP [15:31:12] I trust your afaik [15:49:23] addshore: nah, tricky thing [15:49:33] the tests check for roundtrip compatibility of formatters and parsers [16:02:21] addshore: no usage of Lib here anymore \o/ https://gerrit.wikimedia.org/r/#/c/237685/1/repo/WikibaseRepo.datatypes.php [16:37:09] benestar: around? [18:13:02] multichill: already got a answer, they want to launch in november. (Het Nieuwe Instituut) [18:13:45] Same id's? Or different ones? If it's the same we can just update the url [18:15:19] What's the point of ID's then? ;) [19:01:56] sjoerddebruin: Id's change all the time I'm afraid. They expose some internal database record id and switch to a different backend -> different id. Doesn't hurt to double check [19:05:34] Will ask when I got time, she works Tuesday again. [20:26:09] aude: meh, script crashed inbetween :S [20:44:10] oh noes :( [20:58:42] aude: restarted it and currently purging enwiki [20:58:49] dewiki is already done [21:04:01] benestar: sounds good [21:04:31] I definitely need to implement some functionality that a job can automatically be restarted from the state it has failed [21:08:20] benestar: Purging stuff? Can you also purge the items which don't have the sitelinks page_prop set? [21:08:27] Were quite a few of them last time I checked [21:08:47] multichill: you mean pages on wikidata.org? [21:08:54] Items [21:09:07] could do ... how does it come they don't have sitelinks set? [21:10:23] Because nobody purged them after the pageprop was implemented? [21:11:19] benestar: I'll rerun the query to see how many items it is right now [21:11:55] if you give me the sql query I can just run the script [21:26:54] benestar: Sorry, missed that. Query is at https://tools.wmflabs.org/multichill/queries/wikidata/no_pageprops.sql and result will be at https://tools.wmflabs.org/multichill/queries/wikidata/no_pageprops.txt [21:27:24] benestar: Just finished 2476796 no_pageprops.txt [21:28:03] Took 16m29.279s to run that query.... [21:31:38] multichill: where does thata check for the sitelinks page prop? [21:32:29] benestar: The toollabs replica of the Wikidata database? [21:33:54] as far as I can see it checks for the wb-claims prop [21:35:32] Yeah, that's the missing page_prop [21:37:07] benestar: MariaDB [wikidatawiki_p]> SELECT * FROM page JOIN page_props ON page_id=pp_page WHERE page_is_redirect=0 AND page_namespace=0 AND page_title='Q10001030'; -> emtpy set [21:37:25] https://www.wikidata.org/w/api.php?action=purge&titles=Q10001030&forcelinkupdate -> 4 rows in set (0.00 sec) [21:37:32] ok, I'm confused now :P [21:37:41] benestar: Purging stuff? Can you also purge the items which don't have the sitelinks page_prop set? [21:38:03] Sorry, mixed up wb-claims and wb-sitelinks [21:38:27] Both are incomplete as you can see on this item. They query is for wb-claims [21:39:45] benestar: Anyway, plenty of purging to do. Can you help out here? [21:40:04] multichill: will do, perhaps not today but tomorrow might work :) [21:41:30] How many items can you purge in a day without causing problems benestar? [21:42:26] I don't know, didn't try it out yet ... [21:42:42] I'm purging at quite a high level atm but didn't cause any problems yet [21:42:51] though that is wikipedia so no change dispatching etc [21:43:00] Now you have 2 million items to figure it out ;-) [21:43:32] benestar: Do you include &forcelinkupdate ? [21:43:45] yes [21:43:57] otherwise page_props won't get updated afaik [21:44:03] Exactly [21:45:14] multichill: and see, it works \o/ https://en.wikipedia.org/w/index.php?title=Special:PagesWithBadges&limit=50000&offset=0&badge=Q17437796 [21:45:28] Looks like every item should have at least 4 page_props. displaytitle, wb-claims, wb-sitelinks and wb-status [21:46:19] while the displaytitle thing is rather a hack :S [21:48:46] benestar: I updated the query for the three wd- fields and I'll run it [21:48:54] thanks! [21:49:00] and please use a nicer format [21:49:36] benestar: This is a very nice format because pywikibot will eat it :-) [21:49:40] https://tools.wmflabs.org/multichill/queries/wikidata/no_pageprops2.txt limit 1M