[00:17:07] * Romaine waves to addshore [00:17:22] :) [04:51:57] I'm not the only one that gets this weird page freeze glitch if I press Enter too quickly while adding sources, right? [10:10:33] New review: Zfilipin; "(1 comment)" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/62598 [11:47:41] how can i find out if a "influenced by" property was already suggested? [12:03:21] Denny_WMDE: https://www.google.com/search?q=influenced+by+property+site%3Awikidata.org&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:nl:official&client=firefox-a [12:03:49] Oh, and why doesn't http://en.wikidata.org/wiki/Property_talk:P139 redirect to www? [12:22:41] Change on 12mediawiki a page Extension:Wikibase was modified, changed by Jeroen De Dauw link https://www.mediawiki.org/w/index.php?diff=687430 edit summary: [+0] Rejected the last text change (by [[Special:Contributions/90.216.85.28|90.216.85.28]]) and restored revision 674766 by Peachey88 [12:22:42] [1] 10https://www.wikidata.org/wiki/Special:Contributions/90%2E216%2E85%2E28 [12:23:18] Change on 12mediawiki a page Extension:Wikibase was modified, changed by Jeroen De Dauw link https://www.mediawiki.org/w/index.php?diff=687431 edit summary: [+0] /* Requirements */ [12:33:42] multichill: thx! [13:06:59] I think some help needed on splitting/merging these: [13:06:59] http://www.wikidata.org/wiki/Q3072495 [13:06:59] http://www.wikidata.org/wiki/Q1008659 [13:06:59] http://www.wikidata.org/wiki/Q2386890 [13:06:59] http://www.wikidata.org/wiki/Q647525 [13:16:15] is it possible to add sources at this stage? [13:16:30] and if not.. why are we already using wikidata to store information that's supposed to be sourced to RS :/ [13:47:28] Also on [13:47:28] http://www.wikidata.org/wiki/Q501898 [13:47:28] http://www.wikidata.org/wiki/Q49985 [13:47:28] http://www.wikidata.org/wiki/Q8236831 [13:47:28] http://www.wikidata.org/wiki/Q13048280 [14:31:54] addshore: you there? [14:32:02] yep [14:32:26] would you have me a status update on "removing language links from the wikipedias"? [14:32:33] just curious [14:33:14] 2.32 million pages left (ish) [14:33:55] sweet [14:34:00] spread over all languages i assume? [14:34:12] my bot is currently moving from wmflabsbots to wmflabstools which means it isnt running right this second (it stopped yesterday afternoon) [14:34:58] but 90% of the job is done :) sweet [14:35:09] it appears that aa|bi|bm|cho|ho|hz|ii|kj|kr|mh|mo|mus|ne|ng|sq|st|udm|xh langs are 'done' which means they shouldnt have any although they may have new articles created with them yet not in my db [14:35:53] the majority of other langs have either very few <1000, other have a few thousand and the biggest ate ..: [14:36:50] de 772k (the bot still needs to check lots of these) en 159k (same as de), ja (94k) as ja requested i restrict the speed of the bot [14:37:13] pt 303k, i have no idea why, i may have to look into this, [14:37:38] as yuo can see those big ones make up at least 3% of the remaining 10% ;p [14:37:45] :) [14:39:07] Hello all [14:39:26] hi pragunbhutani ;) [14:39:40] my guess is by time the hackathon rolls around there will only be a few left ;p [14:39:44] aude: towards the end of our conversation yesterday, you'd given me a few pointers about how to approach that bug [14:39:50] could you repeat that please? [14:39:53] hoo: hello! :D [14:40:19] hoo: hopefully, I'm going to have another patch in tonight :) [14:40:52] pragunbhutani: Sounds great... today is a public holiday in Germany, so you might not reach aude [14:40:55] Denny_WMDE: just thought of another page you might link with a sortable list of everything >> http://www.wikidata.org/wiki/Wikidata:Wikidata_migration/Progress [14:41:26] oh that is great [14:41:31] thanks addshore [14:41:38] hoo: oh! In any case, I'll try tinkering around myself [14:41:38] np :) [14:42:05] hoo: by any chance, could you give me any leads on https://bugzilla.wikimedia.org/show_bug.cgi?id=47726 ? [14:42:20] what's the trick used on wikidata to convert P373 values into links? [14:42:24] I've got wikiabase repo and client set up on my system [14:42:34] P373 is the Commons category one [14:42:48] pragunbhutani: Sure :) [14:43:41] pragunbhutani: You need the 'InfoAction' hook on the client [14:45:15] is there someplace where I can read about that? [14:45:59] pragunbhutani: Sure: mediawiki got a documentation for all hooks in docs/hooks.txt and there's https://www.mediawiki.org/wiki/Manual:Hooks/InfoAction [14:46:23] hoo: thanks, I'll read and see if that gives me some insight :) [14:46:29] For the wikibase logic maybe ClientHooks::onSkinTemplateOutputPageBeforeExec can give you a hint [14:50:36] addshore: Where can I find these stats? [14:51:05] multichill: which stats specifically? [14:51:13] Interwiki completeness [14:51:19] multichill: This? http://www.wikidata.org/wiki/Wikidata:Wikidata_migration/Progress [14:52:15] Yes, and where can I find the list of articles that are not done yet? Is that public? [14:52:41] http://tools.wmflabs.org/addshore/addbot/iwlinks/raw.html [14:52:46] for bots [14:52:51] hi Denny_WMDE [14:52:55] or a human readable http://tools.wmflabs.org/addshore/addbot/iwlinks [14:53:20] multichill: planning on your bot going through them? :) [14:53:51] No, just wondering. Don't feel like doing double work [14:54:02] Did you already think about a strategy for new pages? [14:54:24] multichill: I scan the DB dumps every time they are released currently adding them to my db to be checked [14:55:15] addshore: does your bot remove iwlinks from /doc pages if the template itself is already wikidata-linked? [14:55:48] not yet, its on my to do list :) [14:56:35] addshore: ok and remember to remove any empty :) [14:56:53] will do ;p [14:58:01] hopfully at the hackathon everything will be coded and I will have set up some sort of automation to add things to the db :) [14:58:32] hi liangent [15:00:39] Denny_WMDE: I imagine in my design there'll be two kinds of languages (from users' view) [15:00:42] addshore: What I was thinking about. I think we have three cases. 1. Properly interlinked articles 2. Articles with interwiki conflicts 3. New articles [15:00:59] one for the "literal one" - returned from data as is [15:01:18] another for derivated ones (with languageconverter) [15:01:20] 1. No work 2. Publish on a page like https://nl.wikipedia.org/wiki/Wikipedia:Interwikiconflicten_oplossen [15:01:40] liangent: i hope they are mutually exclusive [15:01:57] 3. Publish daily a page in each language with new articles with suggested items to link to. After n days (say 7) just create new items [15:02:13] multichill: that is pretty much my plan :) [15:02:17] i.e. something like sr-latin is always an automatically converted value [15:02:46] once the bot has finally run through everything in the current db once (and therefor we have a much smaller db) it will be more realistic :) [15:02:50] Denny_WMDE: but we still need to manually override some values in case automatic conversion fails [15:03:26] that's exactly what i tried to avoid… :P [15:03:31] ie. it does not always output a correct value [15:03:56] this may rarely happen in those latn-cyrl conversions [15:03:57] i am glad the original 25million rows is now down to 2.5, but there is still some way to go ;p Once everything is a bit smaller I'm going to review my code and slowly increase the number of checks e.t.c that occour and potentially add more details to the db [15:04:09] but it does often happen in Chinese conversion [15:04:19] addshore: Just adding the pages in https://en.wikipedia.org/wiki/Special:NewPages shouldn't be too hard [15:05:00] liangent: do you have an idea of how frequently the conversion fails? [15:05:05] Denny_WMDE: including converter directives in data may avoid this, but I think it pollutes data in database [15:06:57] multichill: I havn't really got any solid ammounts of time until the afternoon of the 21st. I should be able to squeue bits in on the 13th 17th 18th and tomorow :P [15:07:40] also multichill I was thinking of potentially adding some sort of web accessible api layer to my db [15:07:48] meaning other bots could run from it and update it :) [15:08:15] adding articles, updating when the articles was last checked, removing articles with no links :0 [15:08:35] Denny_WMDE: I don't have any stats number but we have a bunch of "dictionaries" on zh.wikipedia [15:08:48] Would be nice, btw, you seem to have an escaping problem at http://tools.wmflabs.org/addshore/addbot/iwlinks/index.html?lang=nl [15:09:03] Sjabloon:Interwiki's beheerp [15:09:16] You should urlencode that [15:10:02] its on github if you want to fix it for me ;p [15:10:19] that file is very messy [15:10:52] i havn't had any time for any of this and its all just hacky fix upon hacky fix upon hacky enhancment and then another hacky fix [15:11:24] https://github.com/addshore/addwiki/blob/master/public_html/addbot/iwlinks/index.html [15:11:38] Denny_WMDE: those dictionaries are transcluded on various articles to provide extra conversion rules [15:12:06] liangent: i wondering how to tell whether "this is a good conversion and needs no override" from "this is a bad conversion and needs an override" [15:13:03] addshore: Would it be possible to do a final run on nlwp so we know for sure it's only pages in conflicted state left? [15:13:34] multichill: not really at the moment :/ im in the middle of rewriting the main script for toolslabs :/ [15:14:23] according to the db only 20 in that list havnt been checked by the bot, so the rest should be conflicting or not yet imported [15:14:43] Denny_WMDE: on wikipedia people just fix it manually (inline with converter directives, or submit it to sysops to have it fixed site wide, or submit it to mw devs to do it in php) when they see a bad conversion [15:19:41] so the idea would be: [15:20:10] if there is something in zh-hans, display it [15:20:32] if there is nothing in zh-hans, take zh, convert it, and display it as zh-hans [15:20:56] if someone clicks on edit, prefill the field with the converted from zh, and let the user edit [15:21:33] when displayed, the autoconverted and the actual entry look the same, you cannot distinguish them [15:21:58] but in the database, and for bots etc., there is nothing saved for zh-hans [15:23:29] Denny_WMDE: the problem is, it's impossible to distinguish whether the user is trying to fix a conversion error, or change the data itself, except for asking them explicitly [15:24:05] hmm, i understand [15:24:08] but does it matter? [15:24:30] if you change de, you are not expecting to also change en [15:24:32] if it's the former case, data given by user should be stored in zh-hans, and in the latter case, user input should be stored in zh, so zh-hant users can see new data [15:25:19] well, if we ignore the latter case, then we will have a few cases where the zh is not being corrected [15:25:35] we would still assume that a zh-user may correct it later [15:25:58] and we have less decisions and less complicated ui [15:26:29] in real world, I imagine few users are using zh [15:26:46] most of people would choose either zh-hans or zh-hant [15:26:57] or even a region-variant (like zh-cn, zh-tw) [15:28:10] in this case they could add zh explicitly to their languages [15:28:12] and get both displayed [15:28:16] so finally the whole thing may become: data get corrected in every individual variant, stored as overrides, with incorrect data in zh [15:28:20] and then they can change both, zh-hans and zh if they want to [15:28:39] liangent: yes, that might happen [15:28:57] but it is unlikely [15:29:22] there will likely analysis tool arise to check for this kind of inconsistencies [15:29:32] and then the labels will get cleaned up again [15:32:31] liangent: do we have a place - a wikipage or sth - where we keep the current state of the design and discussion? [15:32:41] besides the proposal itself [15:33:20] Denny_WMDE: I want to have one [15:33:30] haven't created it yet [15:35:01] in the past I put some ideas in this section: https://www.mediawiki.org/wiki/User:Liangent/wb-lang#Technical_notes [15:36:48] Denny_WMDE: I'm moving it to https://www.mediawiki.org/wiki/User:Liangent/wb-lang/dev now, any more stuff can be put there, or do we want a meta page or something? [15:45:32] in the end there should be a meta page. for now anything will do :) [15:45:54] (in the end == if the proposal gets accepted, and then) [17:28:45] Wheeeee [17:28:50] Loads of wikidata exceptions [17:29:49] :) [17:30:21] Lydia_WMDE: aude ^ [17:30:26] https://bugzilla.wikimedia.org/show_bug.cgi?id=48298 [17:30:39] Seems wikidata (though, I blame the bot editors) are causing 2-3 exceptions a second [17:30:52] thanks Reedy [17:31:02] * Reedy glares at legoktm [17:31:19] o.O? [17:34:06] Reedy: fwiw, my bot has been dead until I finish cleaning up the mess it made last week :/ [17:34:18] Oh, sorry then ;) [17:34:19] if ( !$this->status->isOK() ) { [17:34:19] $this->errorType |= self::PERMISSION_ERROR; [17:34:19] wfProfileOut( __METHOD__ ); [17:34:19] throw new \PermissionsError( $action, $permissionStatus->getErrorsArray() ); [17:34:19] * legoktm redirects Reedy's glare at addshore [17:34:21] } [17:34:45] * addshore reads up [17:35:01] heh [17:35:08] * addshore re redirects reedys glare to legoktm ;p [17:35:24] * Reedy glares generally at #wikimedia-wikidata [17:38:33] is wikidata going to switch to always using https at some point? I seem to remember someone saying it would [17:40:22] nikki: yeah, but that will be WMF-wide [17:40:37] ah... any idea when that'll happen? [17:40:46] Reedy would! [17:42:47] also, legoktm, musicbrainz now allows links to wikidata :) [17:42:52] awesome :D [17:44:40] I'm hoping we can eventually stop storing links to wikipedia and instead just fetch them from wikidata [17:46:58] legoktm: Made a harvest mode for templates [17:47:06] And published it in Pywikipedia [17:47:11] :D [17:47:17] ill take a look this evening, was about to go afk [17:47:27] Rebuild claimit.py too [17:48:22] So with claimit.py you can do all the general stuff and with harvest_template.py you can harvest the templates. [17:48:41] nice [17:48:46] will bbl now [18:13:03] FLOW office hours chat going on now in #wikimedia-office [18:18:21] legoktm: > now [18:18:31] ? [18:21:28] always using HTTPS will be at some point greater than now [18:24:06] I never said it would be now... [18:24:11] oh [18:24:14] nikki: ^ [18:24:29] Precisely [18:24:40] I'm not sure we'll ever force everyone to use HTTPS [18:24:54] There is that option for logged in users to be redirected to https etc [18:26:30] thanks :) [18:38:29] nikki: Install HTTPS Everywhere if you want to make sure you use ssl [18:39:31] it's not that :) we (musicbrainz) want to make sure our links are consistent [18:40:37] normally we use http for them all, but if you were planning to switch to only https soon, it wouldn't make sense to do that [18:42:02] why not use protocol-relative links? [18:44:47] legoktm: Published at https://www.wikidata.org/wiki/Wikidata:Project_chat#Pywikipedia_scripts_to_mass_add_claims [18:44:48] at the moment I don't think we actually support storing urls that way :/ [18:45:04] multichill: :D [18:46:20] legoktm: Had enough of playing around with custom bots so I put these online [18:46:23] and I'm not sure if it makes sense to store urls without the protocol, my experience so far has been that they only work inside a html page, not in my irc client, not in my email client and not in my browser's address bar :/ [18:46:34] I wonder if this will help to get more people to help out with the bots [19:34:03] https://www.wikidata.org/wiki/Wikidata:Pok%C3%A9mon_task_force <-- looks pretty legit [21:30:31] President Obama just signed an executive order that "Newly generated government data will be required to be made available in open, machine-readable format by default " [21:30:49] according to an email I just got from the White House [21:31:15] csv for everyone! [21:31:59] http://www.whitehouse.gov/blog/2013/05/09/landmark-steps-liberate-open-data [21:32:02] csv? [21:33:19] http://www.whitehouse.gov/blog/2013/05/09/landmark-steps-liberate-open-data [21:33:25] oh, oops [21:33:31] Reedy: what's CSV? [21:33:41] Comma seperated values [21:35:25] :DDD [21:38:11] oh [22:05:51] New review: Zfilipin; "Setting "Code-Review" to "-1 There's a problem with this change, please improve" so I can tell I hav..." [mediawiki/extensions/Wikibase] (master) C: -1; - https://gerrit.wikimedia.org/r/62598