[01:46:13] Does anyone know if there is any bot that removes non-existent links from items? [01:55:39] Hazard-SJ: my bot can do, but i disallowed this [01:55:56] i also have a complete list of such sitelinks [01:56:45] OK [01:56:59] Any reason why you have it disallowed? [01:58:08] i think it is not inlcuded in my existing blot flag request and most of the community are not aware of that problem which results in long dicussions [02:01:22] Hazard-SJ: on other wiki many user are asking why my bot removes langlink although the page was moved. but bots are not able to read move logs [02:02:07] (but if the bot is starting at the moved page it will readd the sitelink) [02:03:11] Hazard-SJ: also all sitelink must be pdated after changed namespacenames on a local wiki [02:17:28] Merlissimo: How about making a public list so humans can go through and fix? [02:18:52] 33000 sitelinks? [02:19:11] i am currently working on reports for wikis (conflict, missing , ...) [02:19:59] ok [08:56:03] Wikidata entry Q666 is Number of the Beast [08:56:04] nice.. [09:51:35] * Lydia_WMDE waves [09:51:40] i'm back peeps! :) [10:22:16] New patchset: Henning Snater; "Improving entity selector widget style" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36555 [10:28:00] Change merged: Henning Snater; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36590 [10:30:31] Change merged: Henning Snater; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36788 [10:33:51] New patchset: Jens Ohlig; "(no bug) Correct singular form in output of pollForChanges.php" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36783 [10:40:41] !nyan [10:40:41] ~=[,,_,,]:3 [10:40:50] New review: Jens Ohlig; "You guys! :3" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36783 [10:41:30] Hey all! Who wanted me to turn on db profiling? Some more details on #42542 would be great. [10:41:49] What do you want to log? [10:42:42] DanielK_WMDE: ^ [10:42:52] * aude uses the default settings [10:44:26] Silke_WMDE: will add details [10:44:31] thx [10:45:41] https://github.com/filbertkm/mediawiki-config/blob/master/DebugSettings.php is what i use, but i'm sure it's not the perfect config [10:48:35] aude, Silke_WMDE: i added instructions to https://bugzilla.wikimedia.org/show_bug.cgi?id=42542 [10:49:27] :) ok [10:51:03] oh, presents? [10:58:11] New patchset: Tobias Gritschacher; "Improving entity selector widget style" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36555 [10:58:25] Change merged: Tobias Gritschacher; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36555 [10:58:49] johl: https://gerrit.wikimedia.org/r/#/c/36565/ [10:59:31] johl: and https://gerrit.wikimedia.org/r/#/c/36562/ (already approved by anjeve) [11:00:11] Change merged: Jens Ohlig; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36565 [11:03:35] Change merged: Jens Ohlig; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36562 [11:03:38] New patchset: Daniel Kinzler; "(bug 42683) new option withoutTermSearchKey" [mediawiki/extensions/Wikibase] (mw1.21-wmf5) - https://gerrit.wikimedia.org/r/36962 [11:04:25] Change merged: Jens Ohlig; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36756 [11:04:45] New review: Daniel Kinzler; "self-approving backport" [mediawiki/extensions/Wikibase] (mw1.21-wmf5); V: 1 C: 2; - https://gerrit.wikimedia.org/r/36962 [11:04:45] Change merged: Daniel Kinzler; [mediawiki/extensions/Wikibase] (mw1.21-wmf5) - https://gerrit.wikimedia.org/r/36962 [11:06:10] Change merged: Jens Ohlig; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/33558 [11:17:25] Change merged: Aude; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36783 [11:26:27] The Cyclons are coming! O_O [11:26:40] New patchset: Daniel Kinzler; "(minor) Improving documentation of profiling settings." [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36963 [11:27:07] Silke_WMDE: improved documentation for the profiling settings: https://gerrit.wikimedia.org/r/#/c/36963/ [11:27:11] Improving doc? Do we have any doc? o_O [11:27:25] Jeblad_WMDE: inline in the file [11:28:45] New review: John Erling Blad; "Yeah yeah... It seems to work.." [mediawiki/extensions/Wikibase] (mw1.21-wmf5) - https://gerrit.wikimedia.org/r/36962 [11:29:07] Change merged: Aude; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36963 [11:31:14] back? [11:34:55] Change merged: Aude; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36770 [12:14:49] New patchset: Daniel Kinzler; "(bug 42534) make repo link work with protocol relative" [mediawiki/extensions/Wikibase] (mw1.21-wmf5) - https://gerrit.wikimedia.org/r/36978 [12:14:49] New patchset: Daniel Kinzler; "use write connection for recent changes save hook" [mediawiki/extensions/Wikibase] (mw1.21-wmf5) - https://gerrit.wikimedia.org/r/36979 [12:14:50] New patchset: Daniel Kinzler; "(bug 42325) Avoid race condition in SiteLinkTable." [mediawiki/extensions/Wikibase] (mw1.21-wmf5) - https://gerrit.wikimedia.org/r/36980 [12:20:31] New review: Daniel Kinzler; "self-approving backport" [mediawiki/extensions/Wikibase] (mw1.21-wmf5); V: 1 C: 2; - https://gerrit.wikimedia.org/r/36978 [12:20:32] Change merged: Daniel Kinzler; [mediawiki/extensions/Wikibase] (mw1.21-wmf5) - https://gerrit.wikimedia.org/r/36978 [12:20:44] New review: Daniel Kinzler; "self-approving backport" [mediawiki/extensions/Wikibase] (mw1.21-wmf5); V: 1 C: 2; - https://gerrit.wikimedia.org/r/36979 [12:20:44] Change merged: Daniel Kinzler; [mediawiki/extensions/Wikibase] (mw1.21-wmf5) - https://gerrit.wikimedia.org/r/36979 [12:20:52] New review: Daniel Kinzler; "self-approving backport" [mediawiki/extensions/Wikibase] (mw1.21-wmf5); V: 1 C: 2; - https://gerrit.wikimedia.org/r/36980 [12:20:52] Change merged: Daniel Kinzler; [mediawiki/extensions/Wikibase] (mw1.21-wmf5) - https://gerrit.wikimedia.org/r/36980 [12:38:18] DanielK_WMDE_: https://gerrit.wikimedia.org/r/#/c/36205/ [12:38:27] * aude loves "jenkins-bot Dec 3 [12:38:28] Patch Set 1: Looks good to me, but someone else must approve" [12:38:39] awww, jenkins bot can't approve? ;) [12:44:41] TobiasG_WMDE: uls is at commit 442561663cdb8973add7be87d3e8f315c55cbf5a [13:04:07] New patchset: Henning Snater; "Removing suggester's "selected" event" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36985 [13:04:07] New patchset: Henning Snater; "Optimizing entityselector widget's input validation" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36986 [13:16:17] DanielK_WMDE_: https://gerrit.wikimedia.org/r/#/c/36990/ [13:18:39] let's backport https://gerrit.wikimedia.org/r/#/c/36174/ [13:18:54] and https://gerrit.wikimedia.org/r/#/c/36175/ [13:19:45] Please fill out the list of feedback/review wishes from WMF until demo time again: https://docs.google.com/a/wikimedia.de/document/d/1gwRAq_am3v-Chuwf2qFGMSrQ-NTp3RY9MYN1JJJfgYU/edit [13:19:47] thanks! [13:22:27] code freeze ist in 10 min [13:25:48] New patchset: Daniel Werner; "$.valueview.widget() can register valueviews per DataType now" [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/36991 [13:25:48] New patchset: Daniel Werner; "Introduces a LinkedSingleInputWidget" [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/36992 [13:32:24] Change merged: Tobias Gritschacher; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36985 [13:32:36] Change merged: Tobias Gritschacher; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36986 [13:34:01] New patchset: Daniel Werner; "Changed used data value type of wikibase-item data type to 'string'" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36994 [13:34:01] New patchset: Daniel Werner; "(bug 42571) view for editing values of type item" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36995 [13:34:29] Change merged: Henning Snater; [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/36991 [13:34:46] Change merged: Henning Snater; [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/36992 [13:40:41] New patchset: Daniel Werner; "(bug 42571) view for editing values of type item" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36995 [13:40:56] Change merged: Henning Snater; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36994 [13:41:35] Change merged: Henning Snater; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36995 [14:00:54] New patchset: Henning Snater; "Always display edit group's cancel button on the right" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36999 [14:01:13] Wikibase, Diff, DataValues are tagged 2012-12-05 [14:02:22] Nikerabbit: hi, anything i can do about https://gerrit.wikimedia.org/r/#/c/33505/ ? [14:03:08] New patchset: Daniel Werner; "Added a commonsmedia valueview which adds links to displayed value" [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/37000 [14:05:02] AnjaJ_WMDE: WIkibase should be a branch, not a tag [14:05:04] ah [14:05:06] sorry [14:05:09] misunderstood, ignore me [14:09:31] New patchset: Daniel Kinzler; "(hot fix) make SiteLinkRemoteTable take a DB object as a param." [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/37002 [14:10:00] Denny_WMDE1: you addressed everything we talked few days ago? [14:10:09] Change abandoned: Daniel Kinzler; "darn, wrong branch" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/37002 [14:10:25] Change merged: Tobias Gritschacher; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/36999 [14:11:48] Nikerabbit: preg_quote, line length i verified. i am told yes, but I didn't do it myself [14:11:55] New patchset: Daniel Kinzler; "(hot fix) make SiteLinkRemoteTable take a DB object as a param." [mediawiki/extensions/Wikibase] (mw1.21-wmf5) - https://gerrit.wikimedia.org/r/37003 [14:12:11] Nikerabbit: if it is still not good I will make the changes myself and then ping you again [14:15:51] I don't everything I said then [14:17:45] the verb is missing [14:18:00] see? remember? miss? :) [14:18:09] I don't even. [14:20:33] $wgCacheDirectory = false to switch it off, Silke_WMDE [14:20:38] Reedy: this is not. [14:21:43] DanielK_WMDE_: http://192.168.2.251:8080/job/wikibase-phpunit/83/console [14:22:03] there are some phpunit failures when running all tests [14:22:24] no issues with groups wikibase wikibaseapi, datavalueextensions, diff [14:22:50] Nikerabbit: can you repeat your last sentence? it wasn't complete [14:27:19] is there a page where all suggestions are collected to be added, like Link FA, link GA, etc? [14:30:46] i try to collect everything in bugzilla [14:31:10] Romaine: do you mean feature suggestions to the software, or other kind of suggestions? [14:31:31] sort of [14:32:15] a page where users can see that suggested items are registered as desired or so [14:32:15] there's http://www.wikidata.org/wiki/Wikidata:Contact_the_development_team or bugzilla [14:33:07] mmm, feels not suitable [14:33:21] Romaine: sorry, what do you mean with item? an item like Q64? or an item like "we need more ponies on the interface"? :) [14:33:44] I think we should have a list of subjects we say about to the public: these subjects will be possible in future [14:33:44] remember [14:33:56] Denny_WMDE1: [14:34:20] already to much hypothesis and stories go around, without grounds for it [14:34:21] Change merged: Aude; [mediawiki/extensions/Wikibase] (mw1.21-wmf5) - https://gerrit.wikimedia.org/r/37003 [14:34:51] Nikerabbit: I have it copypasted for reference, actually :) [14:35:06] Denny_WMDE1: my comment on preg_quote stand [14:35:06] Denny_WMDE1: and I would prefer if the lines would be shorter than 100 chars [14:35:06] // Quote eveerything to avoid unexpected matches due to parenthesis form [14:35:06] Nikerabbit typo 17:51 [14:35:07] Nikerabbit // It is not necessary to quote any additional chars except the special chars for the regex 17:51 [14:35:07] Nikerabbit it is also necessary tuo quote the separator char, / 17:52 [14:35:08] Jeblad_WMDE the regexp chars _is_ included 17:52 [14:35:08] Jeblad_WMDE Its what the preg_quote do 17:52 [14:35:09] Nikerabbit but / is not regexp char 17:52 [14:35:09] Nikerabbit it has no special meaning 17:52 [14:35:10] Nikerabbit it is just convention, preg_match( '~foo~', ... ) works as well 17:53 [14:35:10] Nikerabbit and for bonus points add @since date/version 17:54 [14:35:11] Nikerabbit Denny_WMDE1: anything else? [14:35:39] Denny_WMDE1: like mentioning, Link_GA, Link_FA, VIAF, IMDb, etc are possible in future [14:35:40] oo [14:35:49] ah right [14:36:03] Romaine: hey [14:36:17] Romaine: for most of this we have bugs registered in bugzilla [14:36:32] Romaine: or they're part of the technical proposal on meta [14:36:39] (like imdb) [14:36:51] let me see if we have something for FA/GA [14:37:16] Denny_WMDE1: The thing was that the char / is a delimiter and I think PECRE fails to include it in preg_quote [14:37:23] Romaine: https://bugzilla.wikimedia.org/show_bug.cgi?id=40810 [14:37:41] Ie., it quotes regex-chars but not the delimiter [14:38:05] [14:30:46] i try to collect everything in bugzilla [14:38:09] Bugzilla Hoarder!! [14:38:29] Lydia_WMDE: nice, but I think more transparency would be nice too [14:38:45] The delimiter should not be regarded as delimiter inside the string, but due to the modifier chars it is dificult to get this right [14:38:52] somethingish.. [14:39:13] Romaine: it's in bugzilla where everyone can search for it and i am happy to point people to different bugs - but it is not good to duplicate this stuff in several places [14:39:19] So nikerabit have sort of right.. /grumpf [14:40:26] Romaine: the LinFa/GA feature was removed some month ago [14:41:03] Lydia_WMDE: perhaps bugzilla is for technical users close by, but for others far away [14:41:28] Romaine: i understand - that's why i said i am happy to help people find stuff there [14:41:50] and i've done so over the last weeks [14:41:50] just a page with the future plans (linked to the bugzilla items) would be more helpful [14:42:10] We should have a page where we list all bugs, just to make it easy for the wikiholics.. [14:42:20] http://meta.wikimedia.org/wiki/Wikidata/Development/Current_sprint [14:42:27] and sorry no [14:42:36] we can't keep this information up-to-date in several places [14:42:53] AnjaJ_WMDE: suggest we use friday's tag for datavalues and diff, as that's what we tested with [14:43:10] +1 to aude [14:43:13] Lydia_WMDE: that meta-page would be fine [14:43:16] I think there is an extension to make nice lists of bugs [14:43:17] you mean for deployment, right, aude [14:44:09] Merlissimo: why removed? [14:44:10] And a Lydia_WMDE re-appeared [14:44:38] JeroenDeDauw: i did! [14:45:08] Romaine: Jeblad_WMDE merged some code and later badge featre was removed from api. i don't know why. Maybe another data model? [14:45:24] The code was dead [14:45:39] We never had working code for badges [14:47:12] Lydia_WMDE: for the core developer team it would be all make sense with meta and bugzilla, but it still feels to me a totally different world than the active situation on Wikidata [14:47:17] sorry to say [14:47:19] a gap [14:47:43] wikidata needs a Raymond [14:48:18] my job on wiki's often is to explain to users what the tech guys are doing, and to the tech quys what the communities think [14:49:14] Romaine: on dewiki Raymond creates http://de.wikipedia.org/wiki/Wikipedia:NEU [14:49:48] that is what the weekly status updates are for [14:49:52] that is what I do too for other projects yes [14:50:23] and yes I read the weekly status carefully [14:50:47] (thanks for that btw) [14:51:13] i could create and list for important api bug [14:52:06] aude: was just referring to our weekly wednesday tag for demo time [14:57:28] Romaine: how does your bot login into mediawiki? api login ir does it read set cookie headers? [14:57:44] I use AWB [14:57:51] that is api so far I know [14:58:13] but it also uses set header cookie [14:58:36] https://bugzilla.wikimedia.org/show_bug.cgi?id=41586 isn't still working for me [14:58:56] only when i add centralauth cookie read by set cookie header [14:59:45] DanielK_WMDE_: FYI, I did the schema changes on the empty tables already [14:59:52] Worth verifying though at least [15:05:43] Reedy, its demo time here so it will take some time before DanielK_WMDE_ is available [15:06:31] Lydia_WMDE keeps track of everything.. ;) [15:08:37] New patchset: Aude; "change "Entity Data" to "Entity data", more consistent" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/37011 [15:09:54] Reedy: thanks [15:11:41] Merlissimo: i think we had the badges some mockups but not in the code [15:12:02] * aude doesn't remember them in the api [15:13:29] aude: i only know the api funktions. i don't know the called function were empty or not. but it was only an info for Romaine that you already have though of that feature [15:23:35] New patchset: Aude; "use PROTO_RELATIVE in client links" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/37013 [15:27:11] are there some minor problems with the wikidata servers? my bot had two timeout and one internerl server error [15:30:52] Sk1d: on wikidata.org? [15:30:57] or test system? [15:49:26] ahoy [15:50:11] Sk1d: usually refers to wikidata.org [15:50:23] there are no issues i am aware of [15:51:53] hey peeps :) we'll be doing another attempt at upgrading wikidata.org in a few hours [15:51:59] the database will be locked for that [15:52:02] nevali: Lydia_WMDE will answer all your questions :D [15:52:09] nevali: hey :) [15:52:12] nice delegation ;) [15:52:15] hehe [15:52:18] how can i help you? [15:52:36] well, wikidata looks splendid... as a human [15:53:07] * nevali was wondering what the plans were for structured non-HTML serialisations (Turtle, for example), HTTP content negotiation, and the like [15:53:38] nevali: sorry i am not sure i understand what you are talking about [15:54:13] * nevali rephrases... [15:54:34] yes wikidata.org [15:54:43] how does one get structured, machine-processable, data out of wikidata, using standard protocols which don't require specific knowledge of how wikidata.org works? [15:55:05] ('you can't, currently' being fine as an answer, of course, more interested in future plans) [15:55:23] api? [15:55:39] HTTP is an API [15:55:49] the api returns a json structured item [15:55:53] a wikidata-specific API violates the 'prior knowledge' constraint [15:56:01] right [15:56:12] nevali: the api [15:56:14] so, starting with that, when can I use HTTP Content Negotiation to get that? [15:56:19] you need to know something about wikidata to use it [15:56:24] i don't think you can get around that [15:56:40] well, if you stick to JSON, you certainly need to know the wikidata schema, sure [15:56:56] hence Turtle being given as an example in the original query [15:57:50] what is turtle? [15:58:10] an RDF serialisation [15:58:27] (probably the most popular; though RDF/XML is the most widely-supported by tools just now) [16:01:11] there will be a rdf export [16:01:11] http://meta.wikimedia.org/wiki/Wikidata/Development/RDF [16:01:49] nevali: if you can read python, playing around with https://github.com/jcreus/pywikidata is a good start [16:02:11] Lydia_WMDE: ah, that's promising [16:02:35] * nevali notes "Linked data principles" in the list, which hopefully means 'dereferencing a wikidata  [16:02:42] ...URI will get you the data' [16:07:59] http://wikidata-dev-repo.wikimedia.de/wiki/Special:EntityData/Q3 [16:08:39] you can get the JSON serialization of an entity (Q3, for example) through Special:EntityData [16:08:41] promising [16:09:26] however, the api offers far more powerful methods and serializations in xml and other formats. [16:09:27] feature-request, then... making a request for /wiki/Q3 with an Accept header asking for 'application/json' has identical results (or, indeed, asking for /wiki/Q3.json) [16:11:59] nevali: http://wikidata-dev-repo.wikimedia.de/w/api.php?action=wbgetentities&ids=q3 [16:12:19] nevali: http://wikidata-dev-repo.wikimedia.de/w/api.php?action=wbgetentities&ids=q3&format=jsonfm [16:12:58] nevali: we will be working on conneg and proper linked data later [16:13:06] we dont have that yet [16:13:11] i am not sure if we will support turtle [16:13:16] it is is not a standard yet [16:13:20] we might just use rdf/xml [16:13:34] you can easily transform that into turtle, anyway [16:16:17] basically we will have uris for the entities [16:16:31] probably they will be http://wikidata.org/id/Q64 etc. [16:16:38] and they will conneg [16:16:47] but we don't have them yet [16:17:09] it is described here: http://meta.wikimedia.org/wiki/Wikidata/Notes/URI_scheme [16:17:58] thanks Denny [16:18:25] for the record, I wouldn't hold up dbpedia as a shining example [16:19:04] (its use of 303s all over the place make life needlessly complex all over the place) [16:19:48] * nevali suspects turtle will be standard by the time you get to it, given it's at Last Call status ;) [16:20:13] nevali: I'd rather not hold my breath for a W3C standard to be finalized [16:20:41] I'd not wait for the status flag to be toggled on the doc, in that case :) [16:20:48] DBpedia's use of 303s is a widely suggested pattern, check for example Tom's and Chris' LOD book [16:21:07] yes, I'm aware of that [16:21:25] I don't have the vocabulary to recommend strongly enough that you don't do that. [16:21:55] it makes debugging a nightmare, increases round-trips, and prevents a certain degree of caching from occurring [16:22:19] only redeeming feature in practice is 'people don't have to know what a fragment is' [16:22:40] never mind the ontology of fragments :) [16:22:59] heh [16:23:03] .oO (semantic web geekery ) [16:23:08] yep :) [16:23:25] before long, all of freenode will be embroiled in an httpRange-14 discussion [16:23:30] (urgh) [16:23:34] i would be surprised [16:23:46] my assumption is that what will be widely used is our existing API [16:24:04] are you aware of njh's dbpedialite, btw? [16:24:09] very much so [16:24:17] ah, splendid [16:24:43] i am curious to see the logs of how much of the linked data wizardry will actually be used compared to the normal api [16:25:06] * nevali is bemused at the linked data part being considered wizardry [16:25:57] well, unless you're planning on read/write RDF support, I imagine *that* would be deeply magical [16:26:14] we will see re wizardry. actual numbers speak louder than assumptions [16:26:32] no plans for write support for rdf from our side [16:26:35] only export [16:26:49] reasonable [16:27:12] if someone from the outside plans to implement such a magical artifact, they will be welcomed [16:27:42] i guess something like a sparql update endpoint that transforms into the appropriate api-calls would be a rather interesting students project [16:28:04] honestly, I wouldn't bother [16:28:44] though yes, it would be *interesting* [16:30:19] rdfio is an extension to smw that does something similar [16:35:13] * Silke_WMDE has to restart mysql on the demo system [16:35:39] ok, i am heading off and be online later again for the deployment [16:44:28] The demo system will now be updated with the latest code [16:59:03] !nyan [16:59:03] ~=[,,_,,]:3 [17:04:37] New patchset: Daniel Werner; "Dropdown for commonsmedia view for choosing Commons Media file (DO NOT MERGE)" [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/37028 [17:15:07] New review: Jeroen De Dauw; "I'm merging this now and will add in a TODO, so you're not blocked by my stuff." [mediawiki/extensions/DataValues] (master); V: 1 C: 2; - https://gerrit.wikimedia.org/r/35034 [17:15:08] Change merged: Jeroen De Dauw; [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/35034 [17:16:39] !nyan [17:16:39] ~=[,,_,,]:3 [17:16:43] :) [17:16:54] New patchset: Jeroen De Dauw; "Fixes for a113142b7214d15047a771d692fa8d3079def370" [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/37029 [17:17:03] Change merged: Jeroen De Dauw; [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/37029 [17:46:29] New patchset: Henning Snater; "Updated statements Selenium tests due to adding linked claim values" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/37034 [17:52:30] Change merged: Henning Snater; [mediawiki/extensions/DataValues] (master) - https://gerrit.wikimedia.org/r/37000 [17:52:31] Change merged: Daniel Werner; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/37034 [17:55:48] Lydia_WMDE: do you know on which date the wmf5 tree was branched from master? So that e.g. bugs which are closed before this date should be reopened [17:56:22] Merlissimo: you mean the branch being deployed today? [17:56:28] yes [17:56:32] let me check [17:59:51] Merlissimo: sorry i can't find it - Denny_WMDE do you know? [18:00:16] Merlissimo: of core or of wikidata? [18:00:28] wikidata repo [18:00:32] last wednesday [18:00:39] we were backporting some fixes since then [18:00:53] ok, so nov28th [18:01:02] i think it was actually nov 30th [18:01:06] but something around that [18:01:39] ok, i only want to test some bug i reported in live system, if it ok now [18:02:12] we havent changed it yet [18:02:19] especially today my error log was very long [18:08:10] * aude waves [18:20:56] * Silke_WMDE keeps her thumbs crossed [18:21:48] Our demo repo is not editable right now. Sorry for that. [18:36:02] my internetz crashed [18:36:11] now everything restarted [18:36:44] Is everyone getting excited? [18:37:04] If not, why not? [18:37:37] :D [18:38:07] It's like, the biggest thing, since the last big thing [18:39:00] yay [18:39:18] it is, virtually, the next big thing! [18:39:24] until the next big thing that is [18:39:36] this update will fix how wikidata looks with modern skin, chick, etc. :D [18:39:50] Which may be big, but whether it's bigger than the last big thing, is unclear [18:40:14] bigness is a problem of perspective. the closer you are the bigger it seems [18:40:20] and we are getting closer by the minute [18:40:24] so it seems pretty big [18:41:15] * aude is using cologne blue as my preference [18:41:24] http://www.youtube.com/watch?v=vh5kZ4uIUC0 [18:42:01] :D [18:42:04] didn't know that one [18:44:20] now fatching more father ted clips [18:45:13] heh [18:45:48] *watching [18:48:17] * Jeblad_WMDE locks for popcorn for the upcoming roll-out [18:48:58] noisy apache logs are noisy [18:49:16] our fault? [18:49:28] Nope [18:49:33] Mostly file/swift related [18:49:53] * Merlissimo loads his wappens with urls showing bugs in the past [18:50:04] The noise in operations isn't the most encouraging [18:56:28] * aharoni is very eagerly awaiting the deployment of his fixes to https://bugzilla.wikimedia.org/show_bug.cgi?id=41005 and similar bugs. [18:59:40] let's get this out of the way ;) [19:03:04] read only yet? [19:03:35] Need to just do wmf4 -> wmf5 stuff [19:04:44] ok [19:05:04] Done, feel free to help browse wikipedias that aren't en to make sure they're not broken ;) [19:05:40] re [19:06:45] aw, just created an item :D [19:08:01] hm... do we have an item about wikidata on wikidata.org? we should have reserved a cool id for that :P [19:08:21] 2013 [19:08:28] IIRC [19:08:34] we do [19:09:06] indeed :) [19:09:09] https://www.wikidata.org/wiki/Q2013 [19:09:48] * [new tag] deploy-2012-12-03 -> deploy-2012-12-03 [19:09:48] * [new tag] foobar -> foobar [19:09:48] * [new tag] ohithere -> ohithere [19:09:51] Did someone get bored? :p [19:11:04] ok, readonly time [19:11:22] You should check some of the tests.. o_O [19:12:40] heh [19:13:00] Jeblad_WMDE: ? [19:13:04] Other than code updates, and pushing the updated config for wikidatawiki, is there anything else to do? Or is the rest client wiki side? [19:13:12] hr.wikipedia seems to work fine [19:13:23] turn the changes table back on [19:13:34] but that's for the client to work [19:13:55] oh, the grow fast table? :p [19:14:09] do not turn the changes table back on before the cron job for the prune script is up [19:14:14] we'll fill the disk [19:14:26] it's not part of the rollout [19:14:29] haha [19:14:39] i'm currently writing a mail about it [19:15:04] Reedy: the pruning script will take care of it [19:15:32] Why are WikiData in readmode only? [19:15:43] Simeondahl: code update [19:15:49] Ahh :) [19:16:02] Should only be a few minutes [19:16:09] Great :) [19:16:15] Thanks [19:19:23] Right. That should be code and config out and updated for wikidata.org [19:19:47] 2 PHP Fatal error: require() [function.require]: Failed opening required '/usr/local/apache/common-local/php-1.21wmf5/extensions/W [19:19:47] ikibase/lib/includes/item/ItemObject.php' (include_path='/usr/local/apache/common-local/php-1.21wmf5/extensions/TimedMediaHandler/handlers/OggHandler/PEAR/File_Ogg:/usr [19:19:47] /local/apache/common-local/php-1.21wmf5:/usr/local/apache/common-local/php-1.21wmf5/lib:/usr/local/lib/php:/usr/share/php') in /usr/local/apache/common-local/php-1.21wm [19:19:47] f5/includes/AutoLoader.php on line 1157 [19:19:51] * Merlissimo deletes 51G of log files for the last three weeks [19:19:57] Ahh [19:20:11] PHP Fatal error: require() [function.require]: Failed opening required '/usr/local/apache/common-local/php-1.21wmf5/extensions/Wikibase/lib/includes/item/ItemObject.php' (include_path='/usr/local/apache/common-local/php-1.21wmf5/extensions/TimedMediaHandler/handlers/OggHandler/PEAR/File_Ogg:/usr/local/apache/common-local/php-1.21wmf5:/usr/local/apache/common-local/php-1. [19:20:11] 21wmf5/lib:/usr/local/lib/php:/usr/share/php') in /usr/local/apache/common-local/php-1.21wmf5/includes/AutoLoader.php on line 1157 [19:20:18] wtf? [19:20:25] PHP error [19:20:27] ?action=purge [19:20:33] uups [19:20:40] o_O [19:20:42] PHP Fatal error: require() [function.require]: Failed opening required '/usr/local/apache/common-local/php-1.21wmf5/extensions/Wikibase/repo/includes/api/ApiGetItems.php' (include_path='/usr/local/apache/common-local/php-1.21wmf5/extensions/TimedMediaHandler/handlers/OggHandler/PEAR/File_Ogg:/usr/local/apache/common-local/php-1.21wmf5:/usr/local/apache/common-local/php-1. [19:20:42] 21wmf5/lib:/usr/local/lib/php:/usr/share/php') in /usr/local/apache/common-local/php-1.21wmf5/includes/AutoLoader.php on line 1157 [19:21:30] Whats wrong on line 1157? [19:21:35] This file no longer exists [19:21:49] Looks like someone backported something but only partially [19:21:56] Autoloader stuff looks correct on master [19:21:57] :o [19:22:23] ugh [19:22:23] It might've just been transient [19:22:32] The errors stopped at 19:16 [19:22:34] php cache [19:22:41] website looks good [19:23:23] localisation cache? [19:23:28] we shall update it? [19:23:32] <special-entitieswithoutlabel> [19:23:44] We will do [19:23:44] ld list' (10.0.6.44) INSERT INTO `wb_terms` (term_language,term_type,term_text,term_search_key,term_entity_id,term_entity_type) VALUES ('en','description','Urban-type settlement in Leningrad Oblast, Russia','urban-type settlement in leningrad oblast, russia','432620','item') [19:23:44] Wed Dec 5 19:16:08 UTC 2012 mw51 wikidatawiki Wikibase\TermSqlCache::saveTermsOfEntity 10.0.6.44 1054 Unknown column 'term_search_key' in 'field list' (10.0.6.44) INSERT INTO `wb_terms` (term_language,term_type,term_text,term_search_key,term_entity_id,term_entity_type) VALUES ('en','description','Urban-type settlement in Leningrad Oblast, Russia','urban-type settlement in leningrad oblast, russia', [19:23:44] '432621','item') [19:24:20] Do you'r guys have access to DB? [19:24:20] Reedy: you updated the config? [19:24:21] Jeblad_WMDE: still not urls: http://www.wikidata.org/w/api.php?action=wbgetentities&props=sitelinks/urls&sites=brwiki|viwiki|kowiki&titles=Carol|Carol|%EC%BA%90%EB%9F%B4 [19:24:24] I already did... [19:24:57] there's a config to turn it off [19:25:36] Again, I wonder if that was transient [19:28:43] Could be [19:28:44] Occurance was around 19:16 too [19:28:44] $wgWBSettings['withoutTermSearchKey'] = true; [19:28:44] I think we can continue [19:28:44] ok :) [19:28:44] Do we want to wait for scap to run before removing readonly? [19:28:44] Seems like some messages are wrong in the recent changes.. [19:28:44] deployed, right? Labels look better in RTL now. [19:28:44] aharoni: we are in the middle of it [19:28:44] Nothing serious, other than it look a bit ugly [19:28:44] and, more importantly, aliases. [19:28:44] But the messages should be defined. [19:28:44] Reedy: ItemObject is registered for autoloading - correctly as far as I can see [19:28:44] great. [19:28:45] Jeblad_WMDE: localisation cache not update yet [19:28:45] Denny_WMDE: Guess so.. [19:28:45] Is https://gerrit.wikimedia.org/r/#/c/36363/ planned for today, too, by any chance? [19:28:45] oh, we need to update the entities per page [19:28:45] Jeblad_WMDE, Reedy: has the message cache been rebuild? [19:28:46] that needs doing [19:28:46] the entities without labels special page doesn't work without that [19:28:46] No [19:28:46] [19:25:34] Do we want to wait for scap to run before removing readonly? [19:28:46] aude: ok. later. [19:28:46] DanielK_WMDE: ok [19:28:46] aude: i'd like to test that script some more [19:28:46] ok [19:28:46] tomorrow, perhaps [19:28:46] * aude has never tried the script [19:29:04] i ran it once, and it seems to work, but i want to look at performance & recovery options [19:29:16] ok [19:29:48] There is an issue with new i18n messages: https://www.wikidata.org/wiki/Special:EntitiesWithoutLabel [19:29:50] Is it because the i18n cache is not purged? [19:30:00] Tpt: working on it [19:30:23] I'm still waiting for someone to answer my question.... [19:30:48] Reedy: probably wait [19:30:57] I suspect with the userbase we don't need "wait" for scap [19:30:57] no idea if it matters [19:31:09] how long does it take? [19:31:14] scap? ;) [19:31:41] how long is a peice of string? :? [19:31:47] heh [19:32:16] Reedy: i'm blurry on what that means. teh scap is what pushes the new code to the apaches, right? for that we should be read only, i guess. [19:32:25] DanielK_WMDE: believe so [19:32:27] All code is already pushed [19:32:41] scap rebuilds the localisation cache, then pushes EVERYTHING to the apaches [19:32:41] so, what is scap doing what we'd need to wait for? [19:32:43] ok folks - list of things people will care about in this update: lots of bug fixes and special:entitieswithoutlables - what else? [19:33:01] localisation cache is probably nice [19:33:03] Making stuff not appear like [19:33:08] * aude is impatient though [19:33:10] Reedy: then wait i think [19:33:17] * aude in the middle of creating an item :D [19:33:26] Lydia_WMDE: first test client. but that won't be ready untill tomorrow or even monday. [19:33:35] slurpinterwiki script gave an error.... api is read only [19:34:00] Lydia_WMDE: also, EntitiesWithoutLabels doesn't work at the moment, we need to run a script to rebuild a table. tomorrow, probably. [19:34:16] mpfh [19:34:16] Let me update extensions-list too so DataValues will get fixed [19:34:29] ah, extensions-list [19:34:49] Reedy: let's stay read only until the scap it through [19:34:59] or does that take, like, hours? [19:35:26] either I got access to an old process or something is wrong.. A fix that is pretty old doesn't show up in the api [19:36:39] Jeblad_WMDE: i check my bot and the version that is compatible need url. must i change my bot or do you think it is possible that url will work for api soon? [19:37:15] maby a config problem because it always worked on testwiki? [19:37:20] http://www.wikidata.org/w/api.php and down to action=wbgetitems and then down to examples [19:37:44] First link reads api.php?action=wbgetitems&ids=42 and the id should be q42 [19:37:56] wbgetitem gives a module error [19:38:09] Reedy: Wikidata, WikidataLib, Diff, DataValues. And WikidataCLient, if appropriate [19:38:18] The rest are already there [19:38:40] Merlissimo: what error? [19:38:50] Merlissimo: can you give the request URL? [19:38:59] DanielK_WMDE: empty url attribute http://www.wikidata.org/w/api.php?action=wbgetentities&props=sitelinks/urls&sites=brwiki|viwiki|kowiki&titles=Carol|Carol|%EC%BA%90%EB%9F%B4 [19:39:24] some stuff might be still cached somewhere [19:39:34] Merlissimo: works for me. [19:39:41] ah, no [19:39:43] empty URLs [19:39:47] that's odd [19:39:51] i thought we fixed that?! [19:39:53] i am getting [19:40:01] Jeblad_WMDE: --^ [19:40:17] There is also other errorsd in the api, seems like something isn't updated [19:40:19] me too [19:40:27] Merlissimo: btw - why is there a disambiguation page in that list? [19:41:07] DanielK_WMDE: because these pages also have langlinks in wp? [19:41:35] Merlissimo: but they should then only link to other disambig pages. not your fault, i know :) [19:42:07] DanielK_WMDE: all sitelinks in this item are disambig pages [19:42:23] The module wbsetitem is there, so something is pretty old [19:42:23] there are many disambiguation pages in wikidata [19:42:27] that link to each other [19:42:46] Starting to run scap [19:42:53] I'll not let it run fully so it can be done quicker ;) [19:43:04] New patchset: Daniel Werner; "Some cleanup of comments and added two TODOs" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/37049 [19:43:22] indeed, wbsetitem is in the list of modules [19:43:25] something is very odd [19:44:09] Api help page is cached for some time [19:44:13] oh, now it reports that it uses wbgetitems.. =D [19:44:19] 10 minutes maybe? [19:44:24] ah ok [19:44:52] yes [19:44:59] in http://www.wikidata.org/wiki/Special:ApiSandbox it is not there [19:45:00] 10 min is ok [19:45:03] thanks for clarifying [19:45:32] http://www.wikidata.org/wiki/Special:ApiSandbox [19:45:34] works for me [19:46:10] i see getentities [19:46:17] which is good [19:46:26] $wgAPICacheHelpTimeout = 60*60; [19:46:30] oh, i see claism there [19:46:38] probably shouldn't be there yet [19:46:47] http://www.wikidata.org/w/api.php?action=wbgetentities&sites=enwiki&titles=Berlin&languages=en&props=sitelinks/urls [19:46:57] urls are empty, strange [19:47:04] damn, my local test database is utterly broken [19:47:12] aww, Too many values supplied for parameter 'languages': the limit is 50" [19:47:22] can't pick them all :o [19:47:32] That's going to be 60 minutes.. [19:47:53] Reedy: think we can live with that [19:48:02] not ideal [19:48:06] Jeblad_WMDE: it is the same thing here http://wikidata-dev-repo.wikimedia.de/w/api.php?action=wbgetentities&sites=enwiki&titles=Helium&languages=en&props=sitelinks/urls [19:48:10] on the current dev [19:48:18] you sure this was ever fixed? :P [19:48:26] it's easily ish purged [19:48:28] "url": false [19:48:42] Denny_WMDE: we really need a way to mark items that just connect disambiguation pages. they are really non-items.. [19:49:23] DanielK_WMDE: philosophical differentiation. same thing with category pages. [19:50:00] urls did work, I'm pretty sure of that [19:50:05] DanielK_WMDE: it'd not want them as targets of item references... [19:50:08] oh, whatever [19:50:29] we even demoed it [19:50:30] DanielK_WMDE: why would anyone choose "Springfield | Disambiguation" as a target of an item reference? [19:50:55] Jeblad_WMDE: I also have the memory of that demo [19:51:04] that is why i am surprised to see it not working on dev [19:51:06] Denny_WMDE: why do we want to clutter the UI with stuff that is not useful in that context? [19:51:40] Jeblad_WMDE: last week testwiki was still ok. i am sure [19:52:06] some backport I guess.. Git's ways are mysterious.. [19:52:34] DanielK_WMDE: it is not that we *want* to do that. but they should be ranked lower than real items just by magic, i.e. by the fact they are used more than the disambigs [19:53:08] I don't think we need to do anything in order to get the proper behaviour automatically, when using a search like Solr here [19:53:44] Jeblad_WMDE: Merlissimo: I am indeed confused by the empty urls, but it is not a deployment issue. there is an issue somewhere else [19:53:50] aude, what did you mean with url = false? [19:54:19] Denny_WMDE: sitelinks [19:54:28] Agree with Denny, if urls doesn't work its not a reason for a rollback [19:54:32] aude: where is it set to false? [19:54:34] api result [19:54:39] "itwiki": { [19:54:39] "site": "itwiki", [19:54:39] "title": "Cuba", [19:54:39] "url": false [19:54:40] }, [19:54:44] oops [19:54:47] ah... [19:54:50] so somethign fails. [19:54:59] i'm rebuilding my local test db [19:55:05] wow, import is slow. [19:55:11] aaaaannnnndddd: failed! [19:55:13] boom. [19:57:03] DanielK_WMDE: this kind of messages might lead to the impression that the deployment failed :) maybe you would like to rephrase them in moments like these [19:57:50] aude: that is weird. don't we have a test to catch for empty urls? [19:58:10] context is everything :) [19:58:20] Denny_WMDE: Jeblad_WMDE: when I run action=wbgetentities&sites=enwiki&titles=Berlin&languages=en&props=sitelinks/urls on my local wiki (current master) I get back the URL as it should [19:58:29] Denny_WMDE: i'd think so [19:58:29] so, for the record: that was a local failure caused by me tempering with the database. [19:58:36] on *my* box [19:59:31] TobiasG_WMDE: hmm. neither test nor dev nor production behave like that though [19:59:45] that is weird [19:59:49] Change merged: Jeroen De Dauw; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/37049 [19:59:49] difference in settings? [19:59:55] Jeblad_WMDE: do settings matter in this case? [19:59:58] could be something fails if it's trying to do select on a DB_MASTER connection [20:00:03] no idea if that's the case there [20:00:15] * aude hopes so [20:00:17] locally, i'm getting urls for *some* sitelinks, and not for others. [20:00:21] will debug now. [20:00:38] oh, hm... [20:00:39] * Lydia_WMDE goes to watch some coursera lectures - poke me when you need me and/or when it's back from read-only :) [20:00:55] Lydia_WMDE: but tell us about what :) [20:00:58] Reedy: what does the sites table look like? it didn't get truncated or something? [20:00:59] Denny_WMDE: I don't think so.. [20:01:18] Denny_WMDE: think again https://class.coursera.org/thinkagain-2012-001 ;-) [20:01:44] just started - basic so far [20:01:48] URLs also fails in master on my machine [20:01:52] read schopenhauer. should be more interesting ;) [20:01:55] ah, some test overrode my sites table. [20:02:09] hehe [20:02:16] sites table is there on toolserver [20:02:16] aude: can you check the sites table on dev-repo? [20:02:22] hm... [20:02:29] damn, would have been a nice explanation [20:02:37] and owould be consistent with what i see locally. [20:02:55] New patchset: Jeroen De Dauw; "up rel notes" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/37053 [20:03:05] Change merged: Jeroen De Dauw; [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/37053 [20:03:39] DanielK_WMDE: yes it's on dev repo [20:04:30] * aude thinks there would be no option to add *any* sitelinks if no stuff in sites table [20:04:39] that's what i had earlier when re-setting up my repo [20:05:36] !nyan | Abraham_WMDE1 [20:05:36] Abraham_WMDE1: ~=[,,_,,]:3 [20:05:43] no sites table stuff === wikidata thinks all the links have been added for an item [20:05:53] aude: yes, people couldn't add anything. i was thinking that the table got somehow lost during the upgrade [20:05:55] heh [20:05:56] but that's not it [20:06:18] Denny_WMDE, Jeblad_WMDE: i see URLs locally, on the wmf5 branch as well as master [20:06:34] * aude thinks it's a read only thing [20:06:47] we did set some sitelink stuff to use DB_MASTER [20:06:51] hope so [20:07:08] no idea exactly how read only works [20:07:16] aude: that wouldnt explain why it doesnt work on test and dev [20:07:26] oh, really? [20:07:40] Something is different, but what.. [20:07:44] http://wikidata-dev-repo.wikimedia.de/w/api.php?action=wbgetentities&sites=enwiki&titles=Helium&languages=en&props=sitelinks/urls [20:07:54] http://wikidata-test-repo.wikimedia.de/w/api.php?action=wbgetentities&sites=enwiki&titles=Helium&languages=en&props=sitelinks/urls [20:08:46] no urls on my test wiki [20:08:50] there's 775 rows in the site table [20:08:51] * aude tries experimental mode [20:09:09] Reedy: in what site table where? [20:09:23] wikidatawiki on the db master [20:09:28] ah, ok, thanks [20:09:49] no difference [20:10:55] so DanielK_WMDE gets URLs and everyone does not [20:11:02] not fair not fair [20:11:38] New patchset: Jeroen De Dauw; "Bump version to 0.4 alpha" [mediawiki/extensions/Wikibase] (master) - https://gerrit.wikimedia.org/r/37055 [20:11:47] Denny_WMDE: i do also get [20:12:08] on master & on wmf5 branches [20:12:10] make a club [20:12:16] yay [20:12:17] that is weird [20:12:20] hrm [20:13:11] TobiasG_WMDE: you on windows. all public ones are on linux. katie is on … mac? daniel on … linux? [20:13:13] doesnt help [20:13:17] * aude tried old versions of wikibase [20:13:19] no difference [20:13:44] items in the main namespace [20:13:49] errr. item namespace [20:14:11] aude: hmm... mine are in Item: [20:14:18] will switch to Main [20:16:33] that would be sooo bah if that was the issue [20:17:05] aude: no, still works for me with items in main ns [20:17:18] no, on test it is in main, on dev in item, and both have no urls [20:17:22] mine are in the main ns. and working. [20:17:57] Denny_WMDE: i can try echo-debugging on the dev box... [20:18:07] if i remember how to log into that :) [20:18:22] yeah, i guess [20:18:41] i am happy it is nothing more important than that [20:19:07] Lydia_WMDE: next time we should probably put a sitenotice up (write that down in the deployment procedure document, Abraham_WMDE1 ) [20:19:23] good idea [20:19:26] Denny_WMDE: good point [20:19:31] bleh [20:19:34] we have 6 instances [20:19:37] which one is it? [20:19:42] Denny_WMDE: done [20:19:48] oh, the one with a public IP, i guess :P [20:20:28] http://www.wikidata.org/wiki/Q432139 [20:20:36] does my item look strange to anyone else? [20:20:38] the label? [20:21:01] nah, it's better with debug=true [20:21:10] label looks fine to me in english. ff/aurora 19 [20:21:16] ok [20:21:25] but be something strange in my cache [20:22:18] aude: looks fine to me too. chrome [20:22:39] ok [20:22:51] the label had no stying for me but now is ok [20:22:58] styling [20:22:58] Here too, firefox [20:23:21] aude: can't find the doc root on the labs box. do you know where it is? [20:23:27] aude: looks fine to me in firefox, chrome and safari [20:24:15] probably playing with my skins.... [20:24:34] still have chick on enwiki and switched back to vector on wikidata just before the read only [20:24:40] locally I now got the session-failure thingy [20:24:43] DanielK_WMDE: /srv/ [20:24:44] why are some people able to edit in wikidata.org? [20:24:51] aude: ah! thanks [20:24:54] Merlissimo: was before read only [20:25:17] aude: 10 minutes ago? [20:25:26] slurpinterwiki does not work in cologne blue :( [20:25:34] no big deal [20:26:23] what, Jeblad_WMDE is editing? [20:26:24] aude: http://www.wikidata.org/w/index.php?title=Q432623&curid=408071&action=history [20:26:27] I can create an item but not add link to it [20:27:43] * aude loves the new create item form :D [20:27:51] Seems like CreateItem bypasses permissions [20:28:03] But how is this possible.. [20:28:31] * Jeblad_WMDE opens a window to his local bank [20:28:40] Yay [20:28:41] seems a bug [20:28:45] Scap is nearly done [20:29:19] Jeblad_WMDE: that would be bad [20:29:26] will you file a bug and set it to high? [20:29:30] the DB is set readonly! [20:29:36] editing usering wbeditentity is reject because of read only [20:29:37] this shouldnt be bypassable :? [20:29:49] I can not add links [20:29:57] yup [20:30:23] hm, i have a suspicion... [20:30:26] Denny_WMDE: you have two dbs i think. [20:30:37] It is done [20:30:38] PARTY [20:30:49] Yesterday or the day before that i could also do that [20:31:11] I merged it so its partly my fault [20:31:17] * Jeblad_WMDE thinks... [20:31:25] Reedy: i might know how to fix the url issue. I'd need you to run a script. but i want to verify on labs first [20:31:29] Merlissimo: i do not think so [20:31:40] Jeblad_WMDE: what did you merge? [20:31:58] do we still go for test2 as a client? [20:31:59] gah! [20:32:01] bash: fork: Cannot allocate memory [20:32:07] we can has bigger VM? [20:32:15] i guess memcached is filling it [20:32:17] Denny_WMDE: Jeblad_WMDE: its also possible to change label using Special:SetLabel [20:32:18] DanielK_WMDE: let us concentrate on test2 as client before fixing the URL issue [20:32:21] should be on a separate box [20:32:23] the URL issue is lower prio [20:32:36] DanielK_WMDE: we've already a big machine for dev [20:32:40] TobiasG_WMDE: that is bad [20:33:07] Abraham_WMDE1, Denny_WMDE: memcached should run on its own instance. that's pretty important, it shouldn't compete with apache and mysql for memory. [20:33:07] think we need to check our special pages [20:33:15] (we may want separate db instances, too) [20:33:25] line 197 in ApiModifyEntity [20:33:34] evaluates to false [20:33:53] And then turns off permissions checks in EditEntity [20:33:57] DanielK_WMDE: we talked about it with silke before i went home [20:34:11] we will try separating stuff [20:34:18] Jeblad_WMDE: how does that eval to false? [20:34:42] Jeblad_WMDE: '' should fail the token checki, right? [20:34:45] Reedy: this is awesome, thanks! [20:35:14] Jeblad_WMDE: this check is wrong (may be my fault), but to me it looks like it would always fail the token check if needsToke() returns false [20:35:16] actually, not sure how the client will work with the wb_changes truncated [20:35:20] Denny_WMDE: ^ [20:35:40] we might need some script to sort of rebuild stuff and do poll for changes on the old items [20:35:49] DanielK_WMDE: what do you think? [20:35:52] aude, Denny_WMDE: the client can show item info, but doesn't get updated [20:36:04] hm... well, it also needs entity_per_page [20:36:09] huh? [20:36:10] without that, nothing will be found [20:36:15] Shall I take wikidata.org out of readonly mode? [20:36:15] yeah [20:36:23] Reedy: sure [20:36:27] yes, please [20:36:35] damn. i'm very sure i'm right about rebuilding the sites table [20:36:48] but i can't do anything on the labs box, because it's out of ram. [20:37:00] DanielK_WMDE: turn off memcached? [20:37:04] yea [20:37:07] gah, why is it so bad? [20:37:14] the old test box was fine [20:37:20] the labs instance [20:37:22] root@wikidata-dev-9:/srv/devrepo/w/extensions/Wikibase/lib# /etc/init.d/memcached stop [20:37:24] bash: fork: Cannot allocate memory [20:37:28] lol [20:37:33] just hard reboot it [20:37:40] thought we got a bigger, better one this time [20:38:09] DanielK_WMDE: I even cannot connect to the instance [20:38:23] root@wikidata-dev-9:/srv/devrepo/w/extensions/Wikibase/lib# free [20:38:24] bash: fork: Cannot allocate memory [20:38:26] yea. [20:38:29] it's completely out of ram [20:38:31] time to reboot [20:38:48] DanielK_WMDE: I will reboot via the admin interface on labs [20:38:55] reboot of wikidata-dev-9 in 5 [20:38:57] 4 [20:38:59] 3 [20:39:01] 2 [20:39:02] 1 [20:39:05] it works fine daniel's way [20:39:06] Lol [20:39:12] Some of your code doesn't prevent writing... [20:39:17] * aude has done that with the maps project [20:39:27] Reedy: noticed :o [20:39:38] what? in readonly mode? [20:39:38] * aude made a new item [20:39:43] ugh [20:39:46] DanielK_WMDE: yes, we can create items [20:39:56] when readonly? that sucks [20:40:01] +1 [20:40:02] There was 9 edits during hte time... [20:40:12] why only 9? [20:40:16] that's what yopu get in software without decent auth layer [20:40:21] o_O [20:40:21] and special:setlabels seems also to break through readonly [20:40:31] * aude and Jeblad_WMDE and a few other people [20:40:35] Reedy: the bodx didn't reboot [20:40:44] DanielK_WMDE: ugh [20:40:47] Reedy: i guess it didn't have enough memory to process the command >_< [20:40:49] Can't you power cycle it through labsconsole? [20:41:13] root@wikidata-dev-9:/srv/devrepo/w/extensions/Wikibase/lib# reboot [20:41:15] bash: fork: Cannot allocate memory [20:41:23] Reedy: how? [20:41:38] Reedy: i did click "reboot" in the instance list [20:41:40] no dice [20:41:59] lol [20:42:56] So, are we doing test2wiki today? [20:43:39] Setlabel does not work for me [20:43:42] The bots aren't editing [20:43:56] Reedy: it probably won't work so nice without the entities per page table [20:44:17] * aude tries truncating my table [20:44:49] Reedy: is the sites table added to test2wiki? [20:44:52] that's in core [20:45:01] I can check [20:45:16] Reedy: ok, screw this. We'll just try this on the live site. [20:45:18] ok [20:45:38] | DB Name | DB Size in GB | [20:45:38] +--------------+----------------+ [20:45:38] | wikidatawiki | 5.133468627930 | [20:45:42] oooh [20:45:48] Reedy: please make a copy or dump of the sites table. then truncate it, and then run lib/maintenance/populateSitesTable.php [20:46:17] Reedy: that will hopefully fix the problem in the api. if not, no harm should be done [20:46:25] if all else fails, we restore from the copy [20:46:26] ok? [20:46:33] wait? [20:46:34] why? [20:46:40] the api problem is not that big [20:46:45] Denny_WMDE: to fix the empty urls [20:46:46] why trying this without test before? [20:46:59] well, the cure could be worse than the disease [20:47:02] Denny_WMDE: we at some point changed where and how the base url is stored in the sites table [20:47:08] so now it's not found [20:47:21] ok, without entity_per_page, client just shows no links [20:47:24] but no errors [20:47:26] Denny_WMDE: that's why i asked reedy to make a backup first [20:47:31] is the only issue that the URLs are not reported in that call? [20:47:44] that's not so terrible that it cannot wait for the next deployment cycle [20:47:47] Denny_WMDE: no, we will have the same problem when adding sitelinks [20:47:56] ...or will we? [20:48:07] no sure - depends on whether the base url for the APIs also got lost [20:48:17] if yes, no sitelinks can be added [20:48:25] anyway, rebuilding shouldn't be a problem. [20:48:25] let us try that first [20:48:29] aude: testwiki doesn't have a sites table [20:48:32] easily rectified [20:48:38] because dev allows for editing sitelinks [20:48:42] and has no url result [20:48:44] Reedy: ok and it too needs to be populated with the script [20:49:00] it's on our todo to move the script to core, but currently it's in the extension [20:49:09] err. that won't work [20:49:10] ugh [20:49:18] or will it? [20:49:21] sitelinks are being added [20:49:32] http://www.wikidata.org/w/index.php?title=Q432625&diff=953097&oldid=953096 [20:49:39] Denny_WMDE: i am editing [20:49:48] All good in the hood then? [20:49:55] seems so [20:49:57] seems slurpinterwiki is missing [20:50:24] there is an issue with the resultset of a module, but we don't have to try any hotfixes on the production server without tests for that [20:50:42] this can wait for the next deployment if that is the only problem [20:50:45] adding sitelinks works [20:50:47] ooh, it accepts stuff like "nl:Brookeville" in the page title field [20:51:04] ? [20:51:23] did it by accident but it gets accepted and seems correct [20:51:54] what is the page title field? :) [20:52:23] http://www.wikidata.org/w/api.php?action=wbgetentities&props=sitelinks/urls&sites=enwiki&ids=q432625 [20:52:26] it's correct [20:52:30] Denny_WMDE: adding site links [20:52:37] it's a non-issue if it works [20:53:04] i'd rather see client deployed to test2wiki than trying to fix the URL element [20:53:21] I need to go home now, perhaps I'm on a bit later [20:53:32] thanks for hanging in Jeblad_WMDE [20:53:49] Abraham_WMDE1: can we send an email out to drop tomorrows daily? [20:54:02] the first meeting is the allhands in the afternoon [20:54:06] no one has to be in for 11am [20:54:13] * aude will be there [20:54:36] but happy to just do coding [20:55:04] Abraham_WMDE1: seems to be still contemplating :) [20:55:20] i am surprised jeroendedauw_ and Danwe are not agreeing with me :) [20:55:24] Denny_WMDE: sure [20:55:34] for the client, i think it would not fail if it's enabled on test2wiki [20:55:42] Denny_WMDE: one moment [20:55:52] but you won't see any difference, except the recent changes filter would be there [20:56:05] aude: ok, so we can deploy the code, but it wont do anything until we have the tables rebuilt and the script running, correct? [20:56:06] ok, shit. [20:56:08] we need entity_per_page [20:56:16] DanielK_WMDE: ? [20:56:21] sites table on the client [20:56:26] Denny_WMDE, Reedy: didn't work. rebuilding the sites table is still a good idea, but apparently, that wasn't the (only) issue [20:56:48] will crwal into the install on labs now [20:56:52] and need a rebuild script to process all the existing entities that are not in the changes table [20:57:18] just turning on pollforchanges will start with what's in the changes table now... e.g. new stuff [20:57:32] understood [20:57:38] so shall we still deploy client? [20:57:39] sorry, it's an oversight but we can fix that tomororw i think [20:57:48] up to reedy and you [20:57:53] i would say yes [20:58:02] i think Reedy is another channel [20:58:08] I'm everywhere [20:58:12] :O [20:58:19] sites rebuilt on wikidatawiki [20:58:21] also, it worries me slightly the populate sites table script [20:58:25] and we thought we could talk badly about you here [20:58:25] sites tables created on test2wiki [20:58:28] it requires wikibase lib [20:58:43] https://www.wikidata.org/wiki/Q432139 <- heh, wtf? [20:58:54] Reedy: can you just dump the contents of the sites table on wikidatawiki and import it to test2wiki? [20:58:54] aude: err. why do we need to have changes for all existing items? [20:58:55] Wiki13: ? [20:59:02] I could [20:59:03] I see only english [20:59:07] and site identifiers [20:59:11] @Denny_WMDE [20:59:13] DanielK_WMDE: for pollforchanges [20:59:14] aude: that would be bad if we did. and i don't think we do. [20:59:28] aude: why do we need to push existing items to the client? [20:59:29] changes get added to the table during editing items [20:59:32] what would that achive? [20:59:38] ? Wiki13: looks fine to me, sorry can you elaborate [20:59:39] oh, true [20:59:48] they can magically be there? [20:59:56] I believe there is an issue with this item: https://www.wikidata.org/wiki/Q100237 two 'en' and no 'de' [20:59:59] aude: yes, no entries in the client's recentchanges until we start to push changes. but other than that? [21:00:05] ok [21:00:16] should be ok [21:00:18] I only see 1 en [21:00:26] !! [21:00:27] Tpt: i have seen that once locally, but couldn't reporduce it... [21:00:34] that is a problem Tpt [21:00:48] DanielK: that´s something I had too [21:00:51] like 2 mins ago [21:00:57] aude: well, the client accesses item data directly from the repo db. so they are there, yes [21:00:58] on https://www.wikidata.org/wiki/Q432139 [21:01:05] DanielK_WMDE: that's fine [21:01:05] it might be because sitestable was rebuilt [21:01:07] * aude is tired [21:01:11] in that moment [21:01:17] indeed [21:01:29] ah [21:01:31] its everywhere [21:01:35] fixed I see [21:01:38] Q100237 looks fien to me [21:01:44] after reloading [21:01:57] everything is broken for me right now [21:01:57] i think that was indeed the problem [21:02:09] https://www.wikidata.org/wiki/Q432139 looks good [21:02:22] i am getting a [21:02:22] Links to pages are already set for all known sites. [21:02:35] on all items [21:02:52] i cannot add another sitelink anywhere [21:02:57] insert into test2wiki.sites select * from wikidatawiki.sites; [21:03:03] oh no, the "create a new item" link seems missing from item by title [21:03:12] insert into test2wiki.site_identifiers select * from wikidatawiki.site_identifiers; [21:03:13] doned [21:03:14] where's the new create item form? [21:03:39] is everything broken right now? [21:03:41] new create item form was there during read only [21:03:46] can anyone add a sitelink? [21:03:48] I've update slurp Interwiki to work with the new API. [21:03:49] what happened? [21:03:54] Denny_WMDE: i could a few minutes ago [21:04:05] i can creat items [21:04:10] can't add site links [21:04:16] is sites table empty? [21:04:18] "add" link is disabled for me [21:04:29] ok, what happened [21:04:42] how many entries are there in the sites table now, Reedy ? [21:04:49] looks consistent with "sites table empty" [21:04:54] yes, exactly [21:05:07] that's for updating the sites table on the live system :P [21:05:07] slurp interwiki says "from" and gives me no choices [21:05:13] 782 [21:05:13] let's get that fixed again [21:05:29] hmmm [21:05:30] Tpt: thanks! [21:05:42] so sites is not empty? [21:05:46] why is this broken then? [21:06:11] Tpt? can you tell what´s wrong with https://en.wikipedia.org/wiki/User:Wiki13/common.js atm? [21:06:23] it doesn´t wotk for me [21:06:43] can anyone check the content of the sites table if it looks good? [21:06:52] even after changing wbgetitems to wbgetentities [21:06:53] api.php reports the full choice of sites in the help [21:06:59] that comes from the site table [21:07:01] oh... [21:07:08] | 784 | zuwikibooks | mediawiki | wikibooks | local | zu | http | gro.skoobikiw.uz. | a:1:{s:5:"paths";a:2:{s:9:"file_path";s:28:"http://zu.wikibooks.org/w/$1";s:9:"page_path";s:31:"http://zu.wikibooks.org/wiki/$1";}} | 0 | a:0:{} | [21:07:09] +---------+----------------------+-----------+-------------+-------------+---------------+---------------+-----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+-------------+ [21:07:09] 782 rows in set (0.00 sec) [21:07:09] mysql:wikiadmin@db34 [wikidatawiki]> [21:07:09] that list is served as an api module and cached on the client [21:07:12] The table is fine [21:07:22] if you got an empty list, it will stay empty [21:07:53] ..and if the squids got an empty list, that will stay there [21:08:04] grrrrr [21:08:09] 10 minutes? [21:08:13] Reedy: first guess: empty site list stuck in squids, affecting js [21:08:15] what's the url that needs purging? [21:08:27] i'll try to dig it up [21:08:32] maybe TobiasG_WMDE can help [21:08:45] Needs to be the same parameters etc.. [21:09:06] Reedy: with the js minifer stuff, it's going to be tricky to figure out [21:09:50] is danwe still around? he might have an idea. or henning [21:10:12] DanielK_WMDE: what's up? [21:10:15] Denny_WMDE: no they are not around [21:10:50] TobiasG_WMDE: https://www.wikidata.org/wiki/Q100237 [21:11:04] the add link is disabled, wikidata thinks there are no more languages [21:11:19] Denny_WMDE: http://wikidata.org/w/api.php?action=paraminfo&modules=wbeditentity does not return valid valus for site parameter [21:11:20] hm.. i'll have a look [21:11:22] Tpt? [21:11:41] Wiki13: Very strange, I believe it should work. The API return {"servedby":"srv250","error":{"code":"id-xor-wikititle","info":"Either provide the item \"id\" or pairs of \"site\" and \"title\" for a corresponding page."}} [21:11:42] Merlissimo: good hint [21:11:47] https://bugzilla.wikimedia.org/show_bug.cgi?id=38263 [21:12:04] Verified in Wikidata demo time for sprint 15 [21:12:15] Merlissimo: wbsetitem is now wbeditentity ? [21:12:29] benestar:Yes [21:12:30] I don´t see it Tpt [21:12:35] i implemented the missing url workaround, but not i have problem with this [21:12:38] even after clearing cache [21:12:39] any other changes? [21:12:48] benestar: yes, a few [21:12:57] is there an overview? [21:13:08] benestar: yes, but itemid is different and the old json format isn't accepted any more [21:13:18] 9_9 [21:13:29] so what to change? [21:13:31] TobiasG_WMDE: i don't get all that resource stuff :P [21:13:39] the new json format was also accepted by wbsetitem [21:13:55] Denny_WMDE: the sitelist on the clientside is empty [21:14:06] Merlissimo: which are the concret differences`? [21:14:12] TobiasG_WMDE: on the clientside? [21:14:15] try wb._sitelist in the console [21:14:28] ah on the browser client [21:14:29] in javascript [21:14:46] yes, the list seems empty in the api too: http://wikidata.org/w/api.php?action=paraminfo&modules=wbeditentity [21:14:49] as merlissimo posted [21:14:53] benestar: depends on you old code. if you used the last possible json format, only the idemid starts with a q now [21:14:55] TobiasG_WMDE: it's empty. [21:15:10] TobiasG_WMDE: do you know how this gets loaded? what the module is called? [21:15:16] Merlissimo: i hope it'll work with that change [21:15:25] DanielK_WMDE: that api.php reported the full list doesnt mean anything [21:15:28] it gets it from mw.config.get( 'wbSiteDetails' ) [21:15:41] api.php is cached as we learned earlier, for a ten minutes or so [21:17:13] wikibase.getSites(); is empty on wikidata.org but works on my test repo [21:17:34] denny i think there's a php thing that creates the sites json [21:18:02] remember trying it to see where wikidata got the http://en.wikipedia ..... etc links from [21:18:48] Wikibase\SitesModule [21:19:06] http://www.wikidata.org/wiki/Wikidata:Project_chat#Bugs [21:19:28] yes, somithing is broken with the sites-table I guess.. [21:19:33] mediaWiki.config.set( "wbSiteDetails", ' . \FormatJson::encode( $sites ) . ' ); [21:20:09] here the list is empty: [21:20:10] http://wikidata.org/w/api.php?action=paraminfo&modules=wbeditentity [21:20:12] foreach ( Sites::singleton()->getSites() as $site ) { [21:20:18] here it is not [21:20:18] http://wikidata-test-repo.wikimedia.de/w/api.php?action=paraminfo&modules=wbeditentity [21:20:29] then creates $sites array which is used in the format json thing [21:20:38] Sites is definitely the sites table [21:21:13] is jeroendedauw_ looking into this too? [21:21:32] Denny_WMDE: one sec [21:21:41] Lydia_WMDE: yes, we know, we are working on it [21:21:49] k [21:22:38] http://wikidata.org/w/api.php is also out of cache now, the list is gone here too [21:22:41] DanielK_WMDE: [21:22:57] api.php now also doesnt report the sites anymore [21:23:15] wikidata thinks that Sites is empty, definitively [21:23:30] but Reedy says it is not, but something is broken here [21:23:32] ok, according to the toolserver, it has my latest edits [21:23:35] and the sites table is full [21:23:53] Tpt: I don´t see any link appearing still... :S [21:23:58] * aude confused [21:24:23] whats with the site_identifiers? was this table truncated and updated too? [21:24:32] toolserver does not have wb_entity_per_page table [21:24:36] Wiki13: Strange. I don't understand this issue. I'll have a look to it tomorrow. [21:24:44] TobiasG_WMDE: oh, good catch! [21:24:51] not sure how schema changes get updated to toolserver [21:25:25] Tpt: ok. If you got the script working again, could you paste the code on my talkpage on Wikidata? [21:25:26] Reedy: sitelink uses a secondary table site_identifiers. that also needs to be updated. [21:25:35] at least for me the populateSites thing only works when site_identifiers is also truncated.. [21:25:37] the populate script should do that, but maybe somethign went wrong there. [21:25:45] just an idea: i saw an closed feature request to make sitelink protocol relativ. and site table contains http [21:25:49] TobiasG_WMDE: yes, indeed, had that problem earlier [21:25:59] Wiki13: Yes. [21:26:08] thx [21:26:37] Merlissimo: Oh [21:26:41] Needs a parameter for that [21:27:11] strip-protocols [21:27:12] Merlissimo: i think we fixed that [21:27:18] some time ago [21:27:20] please [21:27:25] Reedy: SQL daniel@sql-s3:wikidatawiki_p> select count(*) from sites join site_identifiers on si_site = site_id; [21:27:28] let us concentrate on the missing Site now [21:27:31] i remember spending an evening debugging that [21:27:38] Reedy: are you sure the populate script actually terminated successfully? [21:27:46] reedy@fenari:/home/wikipedia/common/php-1.21wmf5$ mwscript extensions/Wikibase/lib/maintenance/populateSitesTable.php wikidatawiki --strip-protocols [21:27:46] done. [21:28:01] After truncating and redoing it [21:28:04] there's still 782 rows [21:28:06] Denny_WMDE: the tables sites and site_identifiers are out of sync. both contain data, but when i join them, there are no matches [21:28:18] and 1564 in site_identifiers [21:28:31] how can we fix that? [21:28:36] mysql:wikiadmin@db34 [wikidatawiki]> select count(*) from sites join site_identifiers on si_site = site_id; [21:28:36] +----------+ [21:28:36] | count(*) | [21:28:36] +----------+ [21:28:36] | 1564 | [21:28:37] +----------+ [21:28:39] 1 row in set (0.02 sec) [21:28:44] truncate site_identifiers and rerun the script? [21:29:01] Reedy: looks a lot better: SQL daniel@sql-s3:wikidatawiki_p> select count(*) from sites join site_identifiers on si_site = site_id; [21:29:08] Denny_WMDE: he just did, i think [21:29:08] 1564 = 782 * 2 [21:29:12] yes. [21:29:15] two per site. [21:29:17] that ios correct [21:29:23] the gloabl and the local identifier [21:29:27] "wiki" appendix is missing for wikikipedias i think [21:29:39] Merlissimo: hm? where? [21:29:43] i see wikivoyage in the sites table and site_identifiers [21:29:56] it's newish since wikidata was launched so seems updated [21:30:00] still doenst seem to work [21:30:17] Reedy: please copy both tables to test2 again, once we know it's working [21:30:43] Denny_WMDE: caching? can we somehow access this stuff bypassing caches?.. [21:30:45] i guess not [21:31:08] this shouldnt be cached: http://wikidata.org/w/api.php?action=paraminfo&modules=wbeditentity [21:31:28] at least it was empty much before api.php [21:31:49] Denny_WMDE: i get a token error. as should be [21:32:01] where? [21:32:24] or... no. sorry. didn't look at what i was really doing :P [21:32:51] errr, [21:33:02] if the api siteslinks info is cached..... [21:33:19] Merlissimo: you get the info for your bot from the toolserver right and not the api? [21:33:53] only api.php itself is cached [21:33:55] just curious if it's an issue or the api wbgetentities is not cached [21:33:56] the paraminfo is not [21:33:57] my bot uses the api [21:33:58] ah, ok [21:34:14] i try to read all config using api [21:34:22] ok [21:34:31] so we have still not resolved the issue as far as i can tell [21:34:42] i'm not quite sure how all the caching works but should be ok for bots [21:35:03] is there a functioning equivalent of wbgetitems atm? wbgetentities seems to break even in the example given in the documentation [21:35:15] YairRand: right now everything is broken, sorry [21:35:19] only the missing uz-latn think is set manually for repo [21:35:40] Denny_WMDE: k, thanks [21:35:57] DanielK_WMDE: aude: anyone? ideas [21:36:18] let us reconstruct what happened [21:36:23] Reedy: can you help us? [21:36:29] Denny_WMDE: don't know [21:36:30] we have updated the site [21:36:33] http://www.wikidata.org/wiki/Special:CreateItem is showing me the old form [21:36:40] we have updated wikidata [21:36:42] it showed me the new one during read only mode [21:36:42] Denny_WMDE: wbgetentities is ok for me except the url bug [21:36:50] and then we were able to edit [21:36:53] and also add sitelinks [21:37:02] *then* what did reedy do? [21:37:12] DanielK_WMDE: suggested to him to truncate sites [21:37:21] and use the populate script [21:37:23] right? [21:37:35] since then [21:37:50] wikidata believes there are no sites anymore [21:38:13] Denny_WMDE: right, that are all steps I wrote down here as far I can see [21:38:14] is it cached somewhere else in your code? [21:38:20] Denny_WMDE: so... I broke it? and we can't restore now, because we didn't make a backup of the site_identifiers table? [21:38:27] Reedy: no. [21:38:35] well... I don't think so [21:38:45] i dont care whose fault is it [21:38:47] we were considering putting it into memcached, but i don't think we did [21:38:48] but [21:38:54] it's there in the database [21:39:02] so there's something wrong in the software, else the caching [21:39:05] i just try to figure out why it is not working right now [21:39:21] so are we all... [21:39:34] the sites and site identifiers look okay on the toolserver [21:39:43] when we setup a new site [21:39:55] we dont do anything else but use the populate script, right? [21:40:04] Reedy: remember you had to kick something to get the sites data in the api / js to update to be https [21:40:06] this sets up both sites and site-identifiers? [21:40:06] oh [21:40:07] like a month ago [21:40:12] Reedy: it IS in memcached! [21:40:14] what did you do to kick it [21:40:14] Yeah [21:40:20] Some repopulate method [21:40:26] $cache = wfGetMainCache(); [21:40:27] $sites = $cache->get( 'sites-cache' ); [21:40:33] oh [21:40:33] Reedy: any idea how to purge that? [21:40:50] also, shouldn't that haver a key prefix? [21:40:58] \Sites::singleton()->getSites( false ); [21:41:15] Try now? [21:41:18] sure [21:41:18] Reedy: please [21:41:25] Yup, fixed [21:41:28] oooh [21:41:32] \o/ [21:41:34] yay! [21:41:40] yaae [21:41:43] works again [21:41:48] perfect [21:41:52] * aude cheers! [21:41:54] Can someone PLEASE add this line of code to MAGICSTUFF.txt or something [21:41:57] can someone explain me though, what was the issue? [21:42:05] was it memcache or something else? [21:42:05] * aude slurping interwikis again :D [21:42:22] Denny_WMDE: we put the list of sites into memcached, so we don't have to load it from the database for every request [21:42:27] Yeah [21:42:28] so, the old version was still sitting there [21:42:42] It's possible the old/empty list ended up being stored [21:42:49] purging all the things fixes it [21:42:49] and Sites::singleton()->getsites(false) forced it to kill the cache? [21:43:00] Denny_WMDE: we should compile a list of issues/resolutions from the irc log. [21:43:00] it just forces it to reload it [21:43:06] and turn that into documentation/howto [21:43:09] which forces it to attempt to cache it [21:43:10] yes [21:43:14] ok thanks [21:43:26] * Reedy finds and internet to claim [21:43:39] Look, Jeroen knew it was his fault. He ran away ;) [21:43:41] that fixed adding sitelinks, i suppose? [21:43:41] SO [21:43:46] test2wiki? ;) [21:43:47] did it also fix empty urls? [21:43:54] Reedy: yes [21:44:03] DanielK_WMDE: i will check [21:44:13] you and aude guide Reedy for test2wiki ? [21:44:29] Denny_WMDE, Reedy: IF you want to attempt test2 now, the first thing is to run repo/maintenance/rebuildEntityPerPage.php [21:44:39] i did not thoroughly test that yet [21:44:39] https://gerrit.wikimedia.org/r/#/c/37117/ [21:44:43] I made the easy one ;) [21:44:50] yep [21:45:20] client won't fail without entity per page (not sure about sites) [21:45:26] but won't do anything [21:45:29] sites and site_identifiers are populated for test2wiki [21:45:34] ok, yay [21:45:41] DanielK_WMDE: it did not fix empty urls [21:45:45] DanielK_WMDE: a really old version of wikidata is available on toolserver s7, because itt has a replag of 20 days [21:45:57] Reedy: Any change you can have a look at https://gerrit.wikimedia.org/r/34843 any time soon? (Or cc someone who can) [21:46:11] (but not accesable by normal ts users) [21:46:16] Denny_WMDE: digging into that on dev now. will temporarily break dev-repo.wikimedia.de, [21:46:23] Not now [21:46:31] Any questions for me? [21:46:33] Abraham_WMDE1: ^ [21:46:36] Merlissimo: i don't see new schema changes on toolserver but see new data [21:46:39] yep, feel free to break dev-repo :) [21:46:42] JeroenDeDauw: YOU BROKE ALL THE THINGS [21:46:54] JeroenDeDauw: not now ^^ [21:46:55] don't see test2wiki.sites but believe it's there [21:47:10] Reedy: because I WROTE ALL THE THINGS? :p [21:47:13] test2 shouldn't be on the toolserver, i think [21:47:18] aude: only view are not changed on ts [21:47:22] DanielK_WMDE: it is [21:47:31] it's like all other wikipedias [21:47:37] Merlissimo: yep [21:47:49] not automatic i suppose [21:47:56] JeroenDeDauw: we2 took a while to figure out sitelinks are in memcached. made us wonder. and on dev, SiteObject seems to be used instead of MediaWikiSite. Strange. Looking into that now [21:48:24] Do any other SQL tables need adding to test2wiki too? [21:48:29] Reedy: no [21:48:29] JeroenDeDauw: putting them into memcached is the right thing, of course. just took a while to find out what to purge :P [21:48:45] So I can just enable it on test2? [21:48:50] sure [21:51:23] for JeroenDeDauw that would be more like ... [21:51:25] http://www.glassgiant.com/ascii/ascii.php [21:51:34] i mean [21:51:41] yay, wikidata client is on test2! [21:51:43] ah never mind [21:51:45] Done! [21:51:46] yay!! [21:51:57] but it doesnt do anything yet, right? [21:51:58] https://test2.wikipedia.org/w/index.php?title=Special:RecentChanges&hidewikidata=0 [21:52:01] just sits there [21:52:04] you can click the filter [21:52:05] carefully waiting [21:52:07] in the dark [21:52:09] for data [21:52:10] to come in [21:52:15] muhahah [21:52:25] but it doesn't show anything different [21:52:54] aude: is the new create item back on wikidata? [21:52:57] it's adding somethign to the query [21:53:03] now we can check the other things that came up :) [21:53:08] Denny_WMDE: shall try [21:53:23] Denny_WMDE: no [21:53:24] :( [21:53:34] YairRand: now, try again [21:53:44] YairRand: if it doesnt work now, we would be interested [21:53:47] works, thanks :) [21:53:52] aude: that is weird [21:53:55] JeroenDeDauw: on dev-repo at least, SitesTable::newRow is getting an empty array for $data [21:54:05] JeroenDeDauw: any iudea why? can you log in to mess with it? [21:54:17] JeroenDeDauw: the same bug seems to be present on wikidata.org. [21:54:21] can't reproduce locally [21:54:33] creating an item with ' ' does not work now :) [21:54:54] Reedy: just in case it is no big hassle [21:54:55] i see You can also create an item. again [21:55:04] you mentioned earlier that wikidatawiki is on 5.8 GB right now [21:55:07] oh, i see the new form now for create item [21:55:08] yeah [21:55:10] can you give us a breakdown per table? [21:55:13] yup [21:55:19] so we can see what is going on [21:55:21] ah, it only works with http://www.wikidata.org/w/index.php?title=Special:CreateItem&site=enwiki&page=Patuxent+River [21:55:25] with the page param [21:55:34] aude: yes, isnt that how it should be? [21:55:50] sure [21:55:53] i was confused [21:55:56] ok, good *phew* [21:56:02] no bug :) [21:56:04] http://p.defau.lt/?JuMvFQEh50IxNbcQPu_XXw [21:56:34] those aint no 5.8 GB? [21:56:42] 5.13 now, apparently.. [21:57:07] ah [21:57:10] me cannot read tables [21:57:13] DanielK_WMDE: no I can't [21:57:26] | DB Name | DB Size in GB | [21:57:26] DanielK_WMDE: unless it has no password [21:57:26] +--------------+----------------+ [21:57:26] | wikidatawiki | 5.135391235352 | [21:57:32] whats the difference between DATA and total_size? [21:57:38] ahh [21:57:39] hm, I don't know if this was the case before, but wbgetentities gives an odd error message if one leaves out parameters: "Either provide the item "id" or pairs of "site" and "title" for a corresponding page." "site" and "title" aren't valid, it's supposed to be "sites" and "titles" [21:57:41] DanielK_WMDE: or is it on labs? [21:57:43] total_size = idx + DATA [21:57:45] got it [21:57:47] thx [21:57:54] yup [21:58:04] big indexes are big [21:58:16] yes, need to rethink these indexes [21:58:37] cause this table is going to grow potentially by one or two orders of magintude [21:58:46] (my working assumption) [21:59:38] we have to think about that index [21:59:48] I wonder if we can get the size of each index somehow [22:00:28] Though, I suspect wb_terms_text is at fault [22:01:16] i am surprised about a second number [22:01:25] we only have 6 M items per site [22:01:31] but we have 12 M terms [22:01:54] that means we have much more aliases and descriptions then i expected [22:01:56] * aude wants admin rights on test2 [22:02:04] or at least to self-approve flagged revs [22:02:15] Merlissimo: do you add a label for every sitelink to an item? [22:02:26] * aude is [[User:Aude3]] there [22:02:26] 10[1] 10https://www.wikidata.org/wiki/User:Aude3 [22:02:29] like do you add an spanish label based on the spanish sitelink? [22:02:40] since i forgot my password for aude [22:03:00] Denny_WMDE: yes, but some exception if the label could be problematic [22:03:24] Denny_WMDE, JeroenDeDauw: ok, I tracked down the empty url issue a bit. Don't have a full explanation yet, but do know ehere to start looking tomorrow [22:03:43] but the bot can also add labels without sitelink ;-) [22:03:45] Merlissimo: urls will stay empty for tonight [22:03:48] will look at that tomorrow [22:03:54] Merlissimo: oh, how is that? [22:04:02] how do you add labels without sitelink? [22:04:21] by the way, since a few people are dropping off for the night [22:04:45] Denny_WMDE: for item having disambig pages all having same page title, i also set the de label [22:05:07] thanks DanielK_WMDE aude Reedy TobiasG_WMDE Merlissimo Abraham_WMDE1 and all, great job! [22:05:16] Anything else to do for test2wiki atm? [22:05:28] Reedy: don't think so [22:05:28] Lydia_WMDE: the bugs that you mentioned before should be resolved [22:05:37] k [22:05:38] thx [22:05:40] Lydia_WMDE: we can send a mail out that the new system is up [22:05:56] done already - [22:05:59] ;-) [22:06:01] Have you posted a list of changes anywhere? [22:06:03] thx :) [22:06:05] Or a condensed list? [22:06:05] todo items are to populate entity per page table, but DanielK_WMDE wants to test it first tomorrow [22:06:10] then enable pollforchanges to run [22:06:11] thx too, Denny_WMDE [22:06:13] and pruning [22:06:39] i think that's all [22:06:54] Merlissimo: do you also set aliases? [22:07:06] for some languages, yes [22:07:09] aude: yes, test first is good [22:07:22] Merlissimo: do you have a feeling how often that happens? [22:07:25] differently put: [22:07:33] we have 6 Million site links in wikidata now [22:07:42] and 12 Million terms (i.e. labels, descriptions, aliases) [22:07:46] does this sound right? [22:07:57] nearly every item has a alias in one language [22:08:11] ok, that sounds right then [22:10:37] Denny_WMDE: http://pastebin.com/b6Mp3urD [22:11:01] doesnt add up [22:11:17] that's 8.6 M terms [22:11:24] the termtable has 12 M entries [22:11:50] what am i doing wrong? [22:12:03] Denny_WMDE: ask mysql :P [22:12:17] :D [22:12:30] does toolserver have a copy of wikidatawiki.wb_terms? [22:12:37] it does [22:12:57] Denny_WMDE: that's where i ran that [22:13:29] how big is the table there? [22:13:36] number of entries [22:14:42] * aude not running count(*) on the table but maybe daniel knows [22:14:51] DanielK_WMDE: can you a ticket, so that ts-admin can update views for the new schema? [22:14:55] * DanielK_WMDE is running count(*) fon that tabler [22:15:00] 8759113 [22:15:02] took ~30 sec [22:15:09] ah, ok [22:15:10] that is consistent [22:15:28] Denny_WMDE: either the 12 M is wrong, or replication dropped entries for some reason i don't even want to know [22:15:40] yes [22:15:50] mysql count(*) isn't counting rows in a table. it is read from table properties [22:16:00] my rc timestamp: 20121205221541 [22:16:00] ok [22:16:03] that's seconds ago [22:16:13] but select term_type, count(*) as n from wb_terms group by term_type; would count [22:16:25] and the results from that query are consistent with the count(*) [22:16:30] Merlissimo: that's only true for MyISAM. For InnoDB, it does count rows. [22:16:34] which is why it takes forever# [22:16:43] :o [22:16:44] toolserver uses only myisam is think [22:16:55] Denny_WMDE: will rollup ;-) [22:16:55] Merlissimo: no [22:17:26] Merlissimo: for a database with frequent concurrent reads and writes, like on the toolserver, MyISAM would be horrible [22:17:29] it wouldn't work at all [22:17:59] much more so on the production boxes [22:18:20] Denny_WMDE: counting sitelinks now [22:18:30] Denny_WMDE: 5217442 [22:18:44] That's >9000! [22:18:47] how reliable are the numbers posted by Reedy [22:18:56] JeroenDeDauw: a little bit :) [22:19:22] Denny_WMDE: he probably uses the table stats that are produced by the indexer. they are not very reliable [22:19:25] just guestimates [22:19:31] but fast [22:19:42] ok [22:20:05] off by 50% isn't that bad :) [22:20:12] the order of magnitude is still correct [22:22:08] so 5.2 M sitelinks? [22:22:16] that means a quarter is done :) [22:23:44] Denny_WMDE: https://toolserver.org/~daniel/misc/sitelink-stats.txt [22:24:21] hu.wp is already covered 30% [22:24:22] almost [22:24:26] i mean [22:24:27] awesome [22:24:30] i am getting sleepy [22:24:35] :) [22:24:43] i'm off for tonight [22:24:45] have fun [22:24:56] cya tomorrow [22:25:03] * DanielK_WMDE is now going to shine his boots, so he gets sweets from santa [22:25:18] :D i think that only works for the kids ;) [22:25:24] thanks everyone [22:25:27] DanielK_WMDE: good point ^^ [22:25:35] good night! :) [22:25:54] Denny_WMDE: we'll see about that ;) [22:25:57] DanielK_WMDE: chrome things your stats page is in dutch :D [22:27:07] go home chrome, you are dunk! [22:27:11] or drunk. [22:27:14] :D [22:27:30] oh Sk1d is already up and running again [22:27:35] that was fast [22:27:43] and Merlissimo too [22:27:45] i mean, your bots [22:27:47] ^^ [22:28:27] as said, the sitelinks work is 25% done [22:28:33] Denny_WMDE: without the url and sitelist bug downtime would have benn 0 seconds [22:28:35] it whas about 5 code changes handling the respinding Q infront of an item id; and changing the action command [22:28:42] Merlissimo: sorry! [22:29:09] Denny_WMDE: any particular urgent backend work that needs to be done (exuding cleanup, bugfixes, everything that's not a big chunk of work)? [22:29:11] awesome work! [22:29:52] can't think of any right now. maybe figure out why the urls are not shown? dunno how complicated that is [22:30:08] JeroenDeDauw: ^ [22:30:17] when is the huwiki deploy going to be? and is it going to include siteside sitelinks editing from the start? [22:30:38] YairRand: it is probably not going to include siteside editing [22:30:39] Denny_WMDE: DanielK_WMDE is already looking at that [22:30:48] Denny_WMDE: and either way, that's bug hunt [22:31:05] Denny_WMDE: i had to backport an old version of my bot code so that it was able to work on wikidata.org after the site started, because it was already running on a newer version installed on testwiki in september [22:31:07] JeroenDeDauw: making sure, order is retained? [22:31:16] Merlissimo: ouch [22:31:33] Denny_WMDE: hmm ok [22:31:44] so siteside editing will come later? or not at all? [22:31:54] YairRand: start of Hungarian depends on if we have issues with the test setup [22:31:58] YairRand: later [22:32:07] YairRand: if everything goes supersmooth, we might even have it this year [22:32:08] first priority is the change propagation stuff [22:32:17] for multi wikis and using the job queue [22:32:23] YairRand: if not, first deployment cycle in january [22:32:40] Denny_WMDE: cool [22:32:56] as long as that goes okay, then we can enhance stuff on the client [22:33:08] (of course, a volunteer might come up with stuff too) [22:33:35] well, there's a reason why wikidata is Q2013 not Q2012 ;) [22:33:59] if huwiki client is live and https://bugzilla.wikimedia.org/show_bug.cgi?id=41345 not implemented, no interwikibot can run on huwiki anymore [22:34:05] the tricky thing with the site link editing is cross-wiki authentication [22:34:33] if a user has central auth / global account, then it's easy but what to do when that's not the case..... [22:35:03] Merlissimo: that is a priority [22:35:32] aude: not allow it :) [22:35:48] Denny_WMDE: probably what we'll do [22:35:59] whats the interwiki prefix for wikidata? [22:36:16] TBloemink: d: [22:36:26] got it, thanks [22:36:31] or wikidata [22:37:28] why is de.wikidata.org not using de als default language for anons. not implemented or not wanted? [22:37:45] not implemented/configured [22:37:50] Merlissimo: "de"? why would it?= [22:38:11] "en" is far more likely to be a decent choice for the visitor [22:38:14] refering from dewiki to wikidata and then no label is shown, because it exists only in de is annoying [22:38:34] Merlissimo: ah, that. we can not use per-user interface language for anons [22:38:48] the whole caching issue that caused *random* languages to be used [22:38:59] needs quite a bit of work to solve that [22:39:14] that depends on changes to the whole caching infrastructure [22:39:28] on my worklist on dewiki i used external links with uselang=de [22:39:48] http://de.wikipedia.org/wiki/Vorlage:WikiData-de [22:40:02] either code changes or config changes to the squid or wait until we can use varnish for caching [22:40:16] and even then, might not be perfect [22:41:13] aude: for site link editing, couldn't the local wiki just send the edit to wikidata, instead of going directly? and then attribute the edit to the user on the local wiki if they don't have a global account or something? or is that not possible? [22:41:15] and the code that makes all the subdomain stuff work on wikipedia is ill-suited for what wikidata is doing [22:41:44] YairRand: not very easy with how the core mediawiki works now [22:41:59] (I have no idea what I'm talking about, btw) [22:42:05] we have the similar issue with showing recent changes in the client [22:42:12] we're linking to the user on the repo, which works [22:42:36] and that couldn't just work the same the other way around? [22:42:53] but some places mediawiki makes assumptions about it being a local user [22:43:25] YairRand: maybe but there would be issues [22:43:36] ah [22:43:55] for the recent changes, we're treating the wikidata edits as "anon", as far as the anon only filter works there [22:44:25] not really great though and might require core changes to support "external" editors and cross-project editing [22:44:53] in a better way, but not sure how much time we can put into that right now [22:45:27] aude: do you remeber that domain names instead if ips where used at the beginning of wikipedia? [22:45:52] Merlissimo: not sure what you mean? [22:46:01] maybe anon with domain name to local project [22:46:13] ah [22:46:33] totay on anon edits ipv4/ipv6 addresses are schon [22:46:49] there's a lot of places where a user id is needed [22:46:54] for anons it's 0 [22:47:10] before mediawiki was used ip where domains audo.wikimedia.de [22:47:27] ah [22:48:34] http://de.wikipedia.org/w/index.php?title=Alan_Smithee&offset=20020923070917&action=history [22:48:59] interesting [22:49:00] the frist thre edits are anon (you can notice it by the missing contrib link) [22:49:53] i see