[00:25:00] *waves at aude* [02:59:35] hoo: You're an OAuth admin? [03:00:19] yeah [03:01:01] hoo: Can you approve my request? So that others can help debug the code :p [03:01:40] Open Access Reader [1.0] [03:01:40] ? [03:01:52] hoo: Yeah. [03:02:03] For the IEG. [03:02:55] Ok, done ;) [03:03:14] hoo: Thanks :) [03:03:42] You're welcome [03:03:51] hoo: "Trusted user" :o [03:04:18] hoo: Can I change the callback url? [03:07:16] Oh, I don't think so [03:07:56] Guess I just try and contort the config even more :p [03:09:07] (03Draft1) 10Hoo man: Clone cached tables before returning them to users in mw.wikibase [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177991 [03:11:39] (03PS13) 10Hoo man: Convert API help to use i18n pt.1 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177990 (owner: 10Sn1per) [03:16:31] (03CR) 10jenkins-bot: [V: 04-1] Convert API help to use i18n pt.1 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177990 (owner: 10Sn1per) [03:26:09] GEOFBOT: I'm taking care of those test failures right now, sorry for the disruption [03:27:08] ok, thanks :D [03:34:18] (03PS1) 10Hoo man: Adopt ApiConventionsTest to MediaWiki 1.25 API module changes [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178140 [03:37:30] (03PS14) 10Hoo man: Convert API help to use i18n pt.1 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177990 (owner: 10Sn1per) [03:38:08] doh [03:38:12] * hoo is an idiot [03:41:12] (03PS2) 10Hoo man: Adopt ApiConventionsTest to MediaWiki 1.25 API module changes [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178140 [03:42:14] (03CR) 10jenkins-bot: [V: 04-1] Convert API help to use i18n pt.1 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177990 (owner: 10Sn1per) [03:42:53] (03PS15) 10Hoo man: Convert API help to use i18n pt.1 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177990 (owner: 10Sn1per) [03:44:33] (03CR) 10Hoo man: "Rebased upon my changeset that fixes the ApiConventionsTest. Also fixed missing colons in @see" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177990 (owner: 10Sn1per) [03:44:41] (03CR) 10Hoo man: [C: 032] Convert API help to use i18n pt.1 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177990 (owner: 10Sn1per) [03:45:48] (03CR) 10Hoo man: "Minor documentation comment" (035 comments) [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178081 (owner: 10Sn1per) [03:46:05] GEOFBOT: That change will automatically be merged once someone reviews (and merges) my test fix [03:46:16] I guess that will in a few hours [03:46:41] hooray [03:46:49] thanks :) [03:55:35] (03PS6) 10Sn1per: Convert API help to use i18n pt.2 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178081 [03:55:46] (03PS7) 10Sn1per: Convert API help to use i18n pt.2 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178081 [03:56:20] (03PS7) 10Sn1per: Convert API help to use i18n pt.3 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178080 [03:57:44] (03PS8) 10Sn1per: Convert API help to use i18n pt.3 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178080 [03:57:54] (03PS6) 10Sn1per: Convert API help to use i18n pt.4 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178084 [04:17:53] Is an !admin available? [04:17:58] hi [04:21:12] (03CR) 10Legoktm: [C: 032] Adopt ApiConventionsTest to MediaWiki 1.25 API module changes [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178140 (owner: 10Hoo man) [04:22:20] (03PS7) 10Sn1per: Convert API help to use i18n pt.4 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178084 [04:24:14] (03PS7) 10Sn1per: Convert API help to use i18n pt.5 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178085 [04:25:10] (03PS8) 10Sn1per: Convert API help to use i18n pt.5 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178085 [04:25:54] (03Merged) 10jenkins-bot: Adopt ApiConventionsTest to MediaWiki 1.25 API module changes [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178140 (owner: 10Hoo man) [04:26:10] (03Merged) 10jenkins-bot: Convert API help to use i18n pt.1 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177990 (owner: 10Sn1per) [09:57:28] (03PS1) 10Tobias Gritschacher: New Wikidata Build - 2014/12/08 [extensions/Wikidata] - 10https://gerrit.wikimedia.org/r/178171 [10:01:22] (03CR) 10Thiemo Mättig (WMDE): "I know a single regex can be much faster than searching a list. But maintaining a regex is painful and error prone. This regex clearly nee" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177814 (owner: 10Daniel Kinzler) [10:07:47] (03CR) 10Daniel Kinzler: "what's the benchmark for actually building the full regex, by concatenating the escaped unit names? and how efficient would that automatic" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177814 (owner: 10Daniel Kinzler) [10:16:57] Thiemo_WMDE: i'm not that keen on using a regex for the units. it just seems the easiest solution that will scale to our needs. a list on the config doesn't. there's several scenarios we may want to cater to, but i'm not sure how to best do that. Have several settings, or one setting that may be a list or a regex, or use a hook to override, or... [10:19:06] the scenario of using items for units is another special case. i'd like it for wikidata, but it's not really a good default for 3rd parties [10:19:26] we could even have a mix of items and "real" units... [10:38:37] (03CR) 10Thiemo Mättig (WMDE): "All this should - obviously - only be done if the validation is actually needed. Caching is fine, sure." [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177814 (owner: 10Daniel Kinzler) [10:42:16] (03CR) 10Daniel Kinzler: "but a 1000 element array in the config would still be loaded for every page view. or shall we move this to an extra file? per default? per" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177814 (owner: 10Daniel Kinzler) [11:01:24] (03CR) 10Aude: "could this work like the patterns in UrlSchemeValidators? or such?" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177814 (owner: 10Daniel Kinzler) [11:03:29] (03CR) 10Aude: "or maybe like we handle globes?" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177814 (owner: 10Daniel Kinzler) [11:16:11] (03CR) 10Thiemo Mättig (WMDE): "Loading a static array of strings should be a few nanoseconds only." [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177814 (owner: 10Daniel Kinzler) [11:24:24] (03CR) 10Aude: [C: 032] Convert API help to use i18n pt.2 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178081 (owner: 10Sn1per) [11:27:54] (03Merged) 10jenkins-bot: Convert API help to use i18n pt.2 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178081 (owner: 10Sn1per) [11:30:32] (03CR) 10Aude: "documentation nitpick, per hoo" (033 comments) [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178080 (owner: 10Sn1per) [11:37:12] (03CR) 10JanZerebecki: "If we want units to be an open set of items that are instance of something (or subclass?), would we want to have a regexp check that dupli" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177814 (owner: 10Daniel Kinzler) [11:40:54] (03CR) 10Tobias Gritschacher: [C: 032 V: 032] New Wikidata Build - 2014/12/08 [extensions/Wikidata] - 10https://gerrit.wikimedia.org/r/178171 (owner: 10Tobias Gritschacher) [11:42:18] (03CR) 10Aude: [C: 04-1] "mostly looks good but have some questions and comment" (033 comments) [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178084 (owner: 10Sn1per) [11:43:58] (03CR) 10Aude: [C: 031] Convert API help to use i18n pt.5 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178085 (owner: 10Sn1per) [11:53:19] [13WikidataBuildResources] 15thiemowmde pushed 1 new commit to 06master: 02http://git.io/kwI6fQ [11:53:19] 13WikidataBuildResources/06master 14c68a5a5 15thiemowmde: Merge pull request #13 from wmde/cachekey... [12:05:33] (03CR) 10Aude: [C: 032] "manually tested, looks okay" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177991 (owner: 10Hoo man) [12:11:40] (03Merged) 10jenkins-bot: Clone cached tables before returning them to users in mw.wikibase [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177991 (owner: 10Hoo man) [12:16:39] (03CR) 10Daniel Kinzler: "@aude: i'm worried about having huge arrays in the config, that's mostly it. globes, calendars, url schemes all work fine that ways becaus" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177814 (owner: 10Daniel Kinzler) [12:51:02] Pharos: Hi, you ran out last time I anwered you ;-) [12:51:40] i might have missed it; can you share again? :) [12:59:33] multichill: ? [13:06:08] Pharos: what was the wikidata workshop for or what was it? [13:06:50] Pharos: We did a wikidata workshop for GLAMs, what is your audience [13:07:04] it's more an informal joint slearning esson by ppl who mostly know very little about Wikidata :P [13:07:13] ah [13:07:16] target audience: Wikimedians / general public [13:09:20] * aude can probably do something about wikidata when the walters has their hackathon [13:12:56] aude: i guess the wikidata game would be a good way into it? [13:13:04] yes [13:14:04] any other suggestions? [13:15:07] fixing issues reported in https://www.wikidata.org/wiki/Wikidata:Database_reports/Constraint_violations/Summary [13:15:23] https://www.wikidata.org/wiki/Wikidata:Tours, although it is incomplete [13:15:47] adding descriptions to items [13:16:31] http://tools.wmflabs.org/wikidata-terminator/ is nice [13:16:48] and https://www.wikidata.org/wiki/Wikidata:Tools/External_tools [13:18:03] beautifull [13:18:31] http://tools.wmflabs.org/wikidata-exports/miga/?classes# if you want to see what properties there area [13:18:52] are* [13:31:31] Pharos: We even got Liam to edit ;-) [13:31:47] Pharos: Oh, something else, will be in New York tomorrow :P [13:32:06] :) [13:32:10] :) :) [13:32:25] :) :) :) [13:32:26] [; [13:32:29] ;] [13:32:31] what brings you to our fair city? [13:32:36] be sure to visit ding dong :) [13:33:35] I might. I still had some free days left and tickets were on sale. Going with my father, we were supposed to do something together couple of months ago, but that fell through [13:33:48] come to DC, only 328 km away! [13:33:49] He's never been there so we decided to go to NYC :-) [13:33:56] nice [13:34:47] Going for a week [13:35:34] you are extremely invited to our wikidata event of course [13:35:39] as well as anything else [13:35:49] we have a red carpet in the chapter offices [13:36:01] "Wikidata event" [13:37:34] thank you, WMF trademark officer harej [13:38:02] I'm more confused about the fact that one apparently exists now [13:39:13] that's a win :) [13:39:49] Merging the wrong way and reusing the item... :/ https://www.wikidata.org/w/index.php?title=Q18130804&action=history [13:40:25] They said amateurs could never write a semantic data repository [13:44:01] When is the Wikidata event, Pharos? [13:44:09] Sunday [13:44:14] 1-5ish [13:44:16] That was soon! [13:44:39] I haven't gotten an email about it or anything! [13:44:43] Pharos: Oh? Link? I might drop by [13:44:44] we have the room till end of December [13:44:54] I am writing said email now :P [13:45:21] You also should consider writing an annual plan [14:23:04] aude: Thiemo_WMDE Tobi_WMDE_SW DanielK_WMDE JeroenDeDauw: pls to add to weekly summary document so i can send the weekly summary out in the next hour [14:23:06] thx [14:24:07] panic [14:24:29] indeed indeed [14:32:57] Lydia_WMDE: Currently in the edit interface Q123 gets converted to the actual item. https://www.wikidata.org/wiki/Q123 should to the same. I remeber talking about that, but can't find the bug [14:33:08] Does that ring a bell for you or should I just file a new one? [14:33:32] multichill: i am not sure what you mean. where does what get converted? [14:33:41] ohhhh [14:33:53] you mean when entering a new value you can enter Q123 and it works [14:33:59] and that should also work for the full url [14:34:00] ? [14:35:00] Lydia_WMDE: Take for example https://www.wikidata.org/wiki/Q18225989 [14:35:44] I click add, enter "instance of". I can enter "Q5", but not "https://www.wikidata.org/wiki/Q5" [14:35:56] ok [14:35:57] yeah [14:36:00] there is a bug for that [14:36:02] let me look [14:36:12] When mass editing it's really nice to be able to just dump in the full url [14:37:07] multichill: https://phabricator.wikimedia.org/T49983 for sitelinks. can't find anything for values [14:37:15] Should be fairly easy to implement in the .js magic, if you know where to look [14:37:25] Yes, for sitelinks it would be nice too, but different usecase [14:40:49] Lydia_WMDE: I'll just file a new bug [14:41:07] thx [14:44:18] [13WikibaseDataModelJavaScript] 15thiemowmde comment on pull request #20 14271b1e6: "of". 02http://git.io/NbU4gw [14:44:54] [15:44:44] MediaWiki-extensions-WikibaseRepository: Be able to use full urls to add claims - https://phabricator.wikimedia.org/T77021 (Multichill) NEW p:Normal a:Wikidata-bugs [14:44:57] Lydia_WMDE: ^^ [14:45:08] thx [14:47:02] [13WikibaseDataModelJavaScript] 15thiemowmde comment on pull request #20 14271b1e6: `{string|null}` 02http://git.io/bX1DEQ [14:56:38] [13WikibaseDataModelJavaScript] 15thiemowmde pushed 1 new commit to 06master: 02http://git.io/KRh2LQ [14:56:38] 13WikibaseDataModelJavaScript/06master 149049dbf 15thiemowmde: Merge pull request #20 from wmde/documentation... [14:58:31] [13WikibaseDataModelJavaScript] 15thiemowmde created 06docBits (+1 new commit): 02http://git.io/96GeHQ [14:58:31] 13WikibaseDataModelJavaScript/06docBits 14c6cbae8 15Thiemo Mättig: Fix bits in JSDocs [14:58:41] [13WikibaseDataModelJavaScript] 15thiemowmde opened pull request #22: Fix bits in JSDocs (06master...06docBits) 02http://git.io/nYN4uw [15:00:26] [13ValueView] 15thiemowmde created 06docBits2 (+1 new commit): 02http://git.io/ag6kzA [15:00:26] 13ValueView/06docBits2 140997c3b 15Thiemo Mättig: Add missing null to JSDoc [15:00:41] [13ValueView] 15thiemowmde opened pull request #136: Add missing null to JSDoc (06master...06docBits2) 02http://git.io/TTnXTw [15:38:00] Thiemo_WMDE: can you update your +2 ?: https://gerrit.wikimedia.org/r/#/c/170961/ [15:38:56] (03CR) 10Thiemo Mättig (WMDE): [C: 032] Determine update actions based on usage aspects. [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/170961 (https://bugzilla.wikimedia.org/71352) (owner: 10Daniel Kinzler) [15:42:12] Thiemo_WMDE: yay! [15:42:47] (03Merged) 10jenkins-bot: Determine update actions based on usage aspects. [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/170961 (https://bugzilla.wikimedia.org/71352) (owner: 10Daniel Kinzler) [15:43:00] oh, jenkis is quick today [15:43:30] Lydia_WMDE: i just added a few lines to the weekly. too late? [16:11:42] DanielK_WMDE: thanks [17:26:46] DanielK_WMDE: https://phabricator.wikimedia.org/T73352 can be closed now I think. Am I right? [17:36:44] ohnoez, its a JohnFLewis [17:47:27] indeed, ohnoez [17:48:33] +1 [17:51:39] :) [18:01:17] Tobi_WMDE_SW_NA: yep, closing [18:06:02] Lydia_WMDE: Funny to see https://en.wikipedia.org/wiki/Template:RedQ again. You know it was deleted about a year ago? :P [18:06:25] multichill: hehe yeah [18:06:52] It doesn't scale [18:09:09] I've deleted the "Please copy it to other-language Wikipedias"-part because you need consensus for such things imo, but i've got reverted. [18:09:32] Lydia_WMDE: Got a very happy and excited email from the Rijksmuseum based on my process on https://www.wikidata.org/wiki/User:Multichill/Rijksmuseum_creators_RKD [18:10:05] Pretty sure we can do a Wikidata case study with them next year to convince other GLAMs [18:10:52] \o/ [18:10:55] awesome [18:11:04] let me know if you need help then [18:11:34] sjoerddebruin: meh sorry. didn't see that edit and revert [18:11:37] sorry [18:11:47] Probably asking Sebastiaan to help out with that part [18:11:48] have to leave now - meeting with musicbrainz guys :) [18:11:52] cool [18:11:56] Another great database. :) [18:12:02] indeed! [18:12:09] and great people [18:12:10] Still need to talk with him about exporting our Wikidata for GLAMs training [19:07:31] aude: ping [19:14:33] pong [19:14:47] we are doing swat tonight [19:15:03] * aude confused about when swat was earlier :( [19:28:54] [13WikibaseDataModelSerialization] 15filbertkm 04deleted 06optimize2 at 14ce631ae: 02http://git.io/r_lREA [19:36:59] aude: ah ok, that was one of my questions [19:37:17] other thing: Will you branch tomorrow or shall I? [19:37:28] Not going to be around before 6pm or so [19:38:41] i can [19:38:58] Nice :) [19:40:12] for the caching patch, i would prefer to make a "wmf11" branch and put it on test2 [19:40:35] or otherwise ask ori how urgent (i think we already see enough improvements that it's less urgent) [19:40:49] Yeah, maybe [19:40:55] other patch can go tonight [19:40:57] I have no idea how big the actual impact will be [19:41:06] i think it will help a bit [19:41:07] especially regarding cpu time I have no idea [19:41:21] saving 2ms is a lot, but the question is how often we actually save that [19:41:31] well, loading all the sites [19:41:52] Loading and unserializing the SiteList is not included in those 2ms [19:42:12] it's expensive and a lot of objects [19:42:17] was a very synthetic benchmark on an idle server [19:42:51] always shows up on my profiling stuff as significant [19:43:09] Ok... also the actual compiling of the setting? [19:43:35] probably not as much [19:44:04] Well, the 2ms I measured would be about 1% of the average backend response time [19:44:37] which, btw all this client stuff... it's significant also on wikidata, since it is a client also [19:45:08] so i have snapshots of my large item [19:46:10] SiteList::getGroup and SiteList::preSetElement take a bit of time [19:46:26] Yep, and we call that several times [19:46:34] I tested it with dewiki's production settings [19:47:06] also, obviously, called for the SiteLinkView [19:47:33] idk if i have the beta feature enabled on my repo [19:48:41] but my items are not connected as site links [19:49:59] aude: Btw, did anyone from us look into the dumps failure? [19:50:30] * of us [19:52:22] i am not aware of that [19:52:44] Ok [19:52:49] I'll have a look then [19:53:00] maybe it's something simple and we can backport also an create a new dump overnight [19:53:09] * hoo crosses fingers [19:55:19] ori: How important do you consider https://gerrit.wikimedia.org/r/177735 to be? I wasn't able to pull useful enough metrics from graphite to tell :S [19:56:12] wow, we spend a lot fo time validating language codes :( [19:56:17] pretty important, but not sure [19:56:35] aude: Validating? [19:56:39] You mean the stuff from babel? [19:56:49] * hoo thought that was pretty ok [19:57:10] language fallback [19:57:20] Oh, that's a mess [19:57:41] no log about json dump stuff on fluorine [19:57:48] maybe the snapshots don't log there :S [19:57:54] encodeLuaVar is most expensive [19:58:06] in client? [19:58:08] yes [19:58:15] That's why I cache in the lua module :P [19:58:19] (The stuff that just broke) [19:58:33] Also I have bugs open to significantly reduce that [19:58:38] yeah [19:59:07] called ~20,000 times for my page [19:59:17] which uses lua in several places [19:59:58] wow [20:00:10] is that only Wikibase stuff? [20:00:31] Or do you use scribunto functions which interact wtih PHP [20:01:02] mostly wikibase [20:01:13] * aude has coordinates modules also [20:01:38] {{#invoke:Coordinates|coord}} [20:03:33] doh [20:03:40] dumpJson works for testwikidata [20:03:42] mh [20:06:04] how do i add a statement ? [20:06:20] .... really !! [20:08:00] (03CR) 10JanZerebecki: [C: 04-1] "This commit mixes API changes (change protected functions to private) that are not mentioned in the commit message with multiple other thi" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/172997 (owner: 10Thiemo Mättig (WMDE)) [20:08:25] GerardM-: https://www.wikidata.org/wiki/Help:Statements#How_to_add_statements [20:08:28] * Tobi_WMDE_SW_NA ;) [20:08:41] (03CR) 10Daniel Kinzler: [C: 04-1] "needs a rebase" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/171569 (owner: 10Thiemo Mättig (WMDE)) [20:09:31] * hoo facepalms hard [20:09:39] I found a bug [20:09:43] but not the one I searched [20:09:46] for [20:11:11] Tobi_WMDE_SW_NA: nice one the edit buttons are gone [20:11:26] so again, how do I add statements [20:11:57] as far as I am concerned Wikidata is broken [20:12:25] (03CR) 10JanZerebecki: [C: 04-1] "I don't think hard coding constants over the code is worth saving use clauses. (This is not a case of saving a package dependency which wo" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/177237 (owner: 10Thiemo Mättig (WMDE)) [20:12:50] GerardM-: JS was broken Wikimedia wide a moment ago [20:12:57] might be that this is still fall out from that [20:13:07] (03PS1) 10Hoo man: Fix "Dumping shard i/n" output from dumpJson [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178258 [20:13:31] I'm seeing edit buttons [20:13:34] easy peasy CR [20:13:44] aude: ^ [20:14:00] I do not [20:15:06] I did a ?action=purge [20:15:08] no effect [20:16:32] hoo: ok, that's not *the* bug but ok... [20:16:36] Yeah [20:16:42] it's a minor annoyance :P [20:17:00] No idea about the bug yet: No error on fluorine and all shards start fine [20:17:04] (03CR) 10Aude: [C: 032] Fix "Dumping shard i/n" output from dumpJson [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178258 (owner: 10Hoo man) [20:17:13] I guess I have to kick of a dummy run and see where stuff explodes [20:17:36] Maybe I should take the time and log the stderr output on the snapshot hosts [20:17:53] but then I probably couldn't look at that because ops don't like me sudoing into that user -.- [20:18:56] rage [20:20:38] (03Merged) 10jenkins-bot: Fix "Dumping shard i/n" output from dumpJson [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178258 (owner: 10Hoo man) [20:21:53] T77904 you are welcome [20:24:43] aude: Started the whole dump thing on terbium now and am watching [20:26:13] * hoo feels like https://www.youtube.com/watch?v=9VDvgL58h_Y :P [20:30:01] heh [20:41:27] snapshot hosts still have zend [20:41:33] maybe they could benefit from hhvm [20:41:45] I know hhvm is bad for cli, but those process are long running [20:44:02] still no failure... I have the bad feeling that they will just run through [20:52:10] (03PS9) 10Sn1per: Convert API help to use i18n pt.3 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178080 [20:57:13] (03PS8) 10Sn1per: Convert API help to use i18n pt.4 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178084 [21:03:20] hey all, question: the IDs on the claims in JSON output depend on claim content? i.e. if the claim changes, does the ID change? [21:03:47] it does [21:04:07] it's as if one was deleted and a new one created (internally) [21:04:26] (03PS9) 10Sn1per: Convert API help to use i18n pt.4 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178084 [21:04:56] (03PS9) 10Sn1per: Convert API help to use i18n pt.5 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178085 [21:04:59] aude: thanls. Can it change for any other reason or it stays the same if the claim is not edited? Also, does it change if claim qualifier values, etc. change? [21:06:08] qualifier change, yet afaik [21:06:20] reference, i think not since it's actually part of the statement (not the claim) [21:06:53] aude: i think i fixed everything :P [21:07:52] i don't think it changes for other reasons but there might be some weird exception [21:08:03] GEOFBOT: looking [21:09:13] Oh, Lydia_WMDE is still alive. ;) [21:09:27] sjoerddebruin: hehe yeah [21:09:30] just got back [21:09:44] sitting in my hotel room in barcelona now and pondering going for a swim [21:10:01] Sssssh, stop making me jealous. [21:10:29] awwwww [21:10:39] well to make you less jealous: [21:10:46] i spent the last 3 days in meeting rooms [21:11:01] discussing stuff and planning things [21:11:13] good stuff ,3 [21:11:23] hehe well... [21:11:24] :P [21:11:33] (That's my one-eyed cat btw) [21:11:33] it was productive but also exhausting [21:11:37] haha [21:22:15] (03PS1) 10Aude: Clone cached tables before returning them to users in mw.wikibase [extensions/Wikibase] (wmf/1.25wmf10) - 10https://gerrit.wikimedia.org/r/178273 [21:24:27] I thought we exclude redirects? https://www.wikidata.org/w/index.php?title=Q6497686&oldid=prev&diff=179695622 [21:30:14] (03CR) 10Aude: [C: 032] Clone cached tables before returning them to users in mw.wikibase [extensions/Wikibase] (wmf/1.25wmf10) - 10https://gerrit.wikimedia.org/r/178273 (owner: 10Aude) [21:31:39] aude: Still no news from the dump stuff [21:31:51] please wait a bit for creating the build [21:32:04] I still hope to see the bug (and that it's easy to fix) [21:34:44] (03Merged) 10jenkins-bot: Clone cached tables before returning them to users in mw.wikibase [extensions/Wikibase] (wmf/1.25wmf10) - 10https://gerrit.wikimedia.org/r/178273 (owner: 10Aude) [21:37:06] hoo: ok [21:50:26] (03CR) 10Aude: "recheck" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178080 (owner: 10Sn1per) [21:51:23] (03CR) 10Aude: "recheck" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178084 (owner: 10Sn1per) [21:54:39] (03CR) 10jenkins-bot: [V: 04-1] Convert API help to use i18n pt.4 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178084 (owner: 10Sn1per) [21:57:21] aude: Test failures are legit [21:58:03] pt. 5 shoudl work, though [21:58:05] (03CR) 10Aude: [C: 04-1] "Problem is that SetSiteLink and SetReference still need the parent methods at this point, although those usages are removed in the next pa" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178084 (owner: 10Sn1per) [21:58:25] (03CR) 10Aude: "otherwise, all of the patches look good to me now" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178084 (owner: 10Sn1per) [21:58:33] yeah [21:58:50] patch 3 is ready [21:59:01] (03CR) 10Aude: [C: 032] Convert API help to use i18n pt.3 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178080 (owner: 10Sn1per) [22:00:42] I've dumped more than a million items yet (on terbium) still everything is totally fine [22:01:34] did it even start today? [22:01:37] do we know? [22:01:50] Yeah it did [22:01:50] maybe something was broken at the time? [22:01:51] ok [22:01:56] it produced an invalid dump [22:01:58] that's possible [22:02:08] although I can't find anything indicating that [22:03:01] (03Merged) 10jenkins-bot: Convert API help to use i18n pt.3 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178080 (owner: 10Sn1per) [22:03:43] If I could, I would just start the normal dump in an attached screen and watch there [22:03:44] Lydia_WMDE: https://phabricator.wikimedia.org/T75892 <-- could you respond on that ticket? :-) [22:03:55] but I can only dump to /dev/null now :S [22:05:06] wikimedia/mediawiki-extensions-Wikibase/wmf/1.25wmf10/a33baac : Marius Hoch The build passed. http://travis-ci.org/wikimedia/mediawiki-extensions-Wikibase/builds/43396738 [22:07:22] James_F: she's probably swimming now :P [22:07:26] * aude jealous [22:07:59] aude: Ok, two possible causes: Either stuff failed horribly in a way which doesn't get logged (eg. segfault) or the log didn't propagate to fluorine [22:07:59] aude: Oh well. [22:08:15] James_F: done [22:08:21] Just created an exception on snapshoot1003 and it propagated to fluorine [22:08:21] ;-) [22:08:25] hah, it's a lydia! [22:08:29] Lydia_WMDE: Brilliant, thanks. [22:08:29] but that was as "apache" [22:08:33] hey aude! [22:08:36] I don't know what's up with the datasets user [22:08:54] * hoo rages again for not getting access to that damn user [22:09:00] hoo: :( [22:10:30] I don't see any segfaults in dmesg [22:10:48] so either we had an exception/ fatal/ ... and it didn't get propagated because the set up with datasets is wrong [22:11:01] hoo: :( [22:11:03] or things failed in a way that doesn't php error [22:11:30] but usually that only happens for segfaults because PHP is unable to error report in those cases :P [22:11:42] if it's an exception, such as non MWException [22:11:52] maybe it's not handled with logging [22:12:08] we had that issue for simple web requests [22:12:34] just a guess, probably wrong [22:14:06] aude: You might be right [22:14:13] or does eval.php handle those in another way [22:14:14] mh [22:14:41] it's just a maintenance script [22:14:45] but don't know [22:14:55] * hoo pokes more [22:17:06] * aude sees $shard must be smaller than $shardingFactor. [22:17:13] which you must have created [22:17:28] InvalidArgumentException [22:18:38] yeah [22:18:43] I triggered those for testing once [22:18:51] and maybe once because I'm stupid [22:19:03] but the bash script has them right [22:19:43] (03PS10) 10Sn1per: Convert API help to use i18n pt.4 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178084 [22:19:58] (03PS10) 10Sn1per: Convert API help to use i18n pt.5 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178085 [22:20:38] (03PS11) 10Sn1per: Convert API help to use i18n pt.5 [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178085 [22:23:18] aude: I guess I'll just ask someone to restart the cron in a screen [22:23:22] probably it will run through [22:25:03] ori: I need your superpowers! [22:27:41] ori: sudo -u datasets /usr/local/bin/dumpwikidatajson.sh [22:27:47] on snapshoot1003 [22:29:36] hoo: ok, sec [22:30:24] hoo: how long is this going to take? [22:30:51] I think we need 11h now [22:31:36] so, i didn't run it in screen [22:31:42] is it ok to ctrl+c and restart it? [22:32:08] Might that it leaves over temporary files [22:32:12] but despite of that, yeah [22:32:42] It will overwrite those on the next run, so feel free to kill all of it [22:33:05] ok, it's running again now. [22:34:00] looks fine :) Thanks [22:46:53] (03PS1) 10Hoo man: Make ViewEntityAction more robust [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178352 [23:02:17] hoo: can we ever get rid of WikibaseLuaBinding::renumber? [23:02:30] it's expensive, performance wise [23:02:37] It sucks [23:02:41] :( [23:02:41] and no :( [23:02:48] awww [23:03:17] The legacy style doesn't need it... but the new one needs it [23:03:33] Lua starts numbering arrays a 1 which is :S [23:04:22] someone will hate me, but we already serialize once in php [23:04:26] why can't it be an option [23:04:38] let it be numbered the way it needs the first time? [23:04:59] that might mess with caching though [23:05:02] that would force us to adjust every place where we create arrays [23:05:34] aude: https://phabricator.wikimedia.org/T76156 is the true solution to all these slownesses [23:06:02] * aude nods [23:08:16] It will possibly break some exotic things [23:08:35] but who would ever want to foreach key -> value over an entity for example [23:09:05] The possible performance gain is huge... and the API will be nicer to use [23:15:07] (03Draft2) 10Aude: Use one instance of SerializationOptions in WikibaseLuaBindings [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178356 [23:16:51] aude: You need to create one for legacy entity calls and non-legacy ones [23:17:09] both can happen on the same object [23:17:14] * instance [23:17:37] we set the id keys always [23:17:48] since yes, it varies [23:18:03] ah I see [23:18:04] mh [23:18:19] i did it your way the first time [23:18:23] but then this is better [23:18:38] aude: Why not just cache the languages array then? [23:18:53] Would probably be nicer [23:19:06] they get turned into fallback chain objects [23:19:24] in SerializationOptions? [23:19:27] yes [23:19:37] that requires bringing code out of there and duplicating it here :/ [23:19:42] plus other overhead [23:22:07] hoo: suppose you probably won't have a fix for the dumps in the next few minutes? [23:22:24] No [23:22:26] ok [23:22:33] i'll prepare the build update [23:22:41] Maybe they're just fine and the hiccup tonight was something not related to our stuff [23:22:51] hope so [23:26:03] (03PS1) 10Aude: Update Wikibase, fix issue with lua [extensions/Wikidata] (wmf/1.25wmf10) - 10https://gerrit.wikimedia.org/r/178365 [23:27:59] aude: I don't think these fallback chains even get applied [23:29:45] they get serialized, but yes you are probably correct :/ [23:32:40] We should either fix it, or remove it [23:37:03] if we indeed don't use, then we should [23:37:49] Do you know offhand what's missing to fix this? [23:38:03] And how expensive that would be [23:38:12] (performance wise) [23:38:32] as far as i see, $entityArr contains fallbacks applied in the serializaiton [23:39:31] e.g. variants [23:39:46] it does? [23:40:44] that doesn't work for me at all [23:44:33] (03CR) 10Aude: [C: 032] Update Wikibase, fix issue with lua [extensions/Wikidata] (wmf/1.25wmf10) - 10https://gerrit.wikimedia.org/r/178365 (owner: 10Aude) [23:46:38] (03CR) 10Hoo man: " aude: I don't think these fallback chains even get applied" [extensions/Wikibase] - 10https://gerrit.wikimedia.org/r/178356 (owner: 10Aude) [23:46:53] (03Merged) 10jenkins-bot: Update Wikibase, fix issue with lua [extensions/Wikidata] (wmf/1.25wmf10) - 10https://gerrit.wikimedia.org/r/178365 (owner: 10Aude) [23:48:55] aude: Lydia_WMDE: Does JS work for you on https://www.wikidata.org/wiki/Q935 ? [23:49:39] mh, seems to work after a purge [23:49:45] weird [23:58:18] wikimedia/mediawiki-extensions-Wikibase/wmf/1.25wmf11/a33baac : Marius Hoch The build passed. http://travis-ci.org/wikimedia/mediawiki-extensions-Wikibase/builds/43409420