[01:43:50] Aⅼⅼɑh is dⲟⅰng [02:50:37] Alⅼah iѕ ԁoⅰnɡ [02:50:37] sᥙn ⅰѕ nഠt dоiᥒɡ Alⅼɑһ іѕ doinɡ [02:50:37] ⅿοоᥒ iѕ nоt doіᥒg Aⅼlaһ iѕ doinɡ [03:18:55] Aⅼⅼɑһ іѕ ԁoiᥒg [04:16:39] RECOVERY - High lag on wdqs2003 is OK: (C)3600 ge (W)1200 ge 1198 https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen [06:12:23] Αⅼlah is doinɡ [06:12:23] sun іѕ not ԁоⅰng Alⅼah ⅰs ԁoⅰnɡ [06:12:23] moon ⅰs ᥒot ⅾοⅰng Аⅼlaһ is doiᥒɡ [06:37:44] Ꭺⅼlɑh iѕ doіᥒg [06:55:18] Aⅼⅼɑh ⅰѕ ⅾഠⅰᥒɡ [07:21:22] nikki: Lydia_WMDE https://phabricator.wikimedia.org/T195302#4587472 looks like it could be a bug indeed [07:35:45] SMalyshev: does wdqs always provide Retry-after with a 429 response? [07:36:50] addshore: i agree it "looks like it could be a bug indeed". not broken altogether though. and with the (language) item contained in the lemma i am not sure that this is implemented yet [07:37:09] Pablo_WMDE: that's what I was thinking [07:37:27] I guess it is just something to do with the language related parts of the lexeme [07:38:00] oh, you mean even that is not implemented yet? [07:38:08] perhaps, but I havn't checked [07:38:25] nono. i think i saw jakob do that. thats the referenceextractor thing [07:38:48] hmmmmmmm [07:39:07] EntityReferenceExtractors. language is one of them [07:39:24] did you just recenntly do the cache busting? [07:39:33] does it definitely get run after a purge? [07:39:47] i sure hope so. worked under that assumption [07:41:28] i'll be in WMDEHQ in 1.25h, could start digging [07:41:43] *then [08:14:26] addshore: so which piece do you think is not working yet even after an edit? [08:14:35] Then I'll test it and open new ticket if needed [08:14:39] also Hi! :D [08:16:12] hello o/ [08:50:58] KassOtsimine told me in #musicbrainz you'd like suggestions for hiding/removing the spam? [08:51:24] Doesn't look like you have either `+r` or `+z` set, so I guess you haven't done anything about it yet? [08:51:42] Well, feel free to ping me if you still want advice. :) [09:21:02] addshore: I just tried action=purge&forcelinkupdate via the api on a lexeme that wasn't showing up and now it is [09:21:11] maybe you didn't include forcelinkupdate? [09:36:01] nikki: I don't think the maintenance script has that option! Let me have a look again in a bit... [09:45:22] addshore: I vaguely remember playing with https://www.mediawiki.org/wiki/Manual:RefreshLinks.php and commenting something along those lines on the purging ticket, but I guess that's not what ended up being used on prod? [10:01:38] jakob_WMDE: did not spot that in the task [10:01:45] will re purge everything with that script now [10:04:38] addshore: sorry, that should've gone in the description! [10:04:54] no problem, i probably should have read all of the comments too! :D [10:56:16] vandal creating fake nonsens items https://www.wikidata.org/wiki/Special:Contributions/92.184.102.26 [12:59:36] PROBLEM - High lag on wdqs2003 is CRITICAL: 3626 ge 3600 https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen [13:06:04] Aⅼlɑһ іѕ dоіᥒg [13:35:59] Alⅼah is dоinɡ [14:15:18] hello everyone [14:16:07] sup [14:16:34] i have a question regarding the legality of scraping services to automatically add data to wikidata. for example, scraping spotify to add "spotify id" properties to music releases [14:16:49] is that done? [14:17:31] for example with the IMDb link property, was that done in collaboration with IMDb? [14:18:39] IMDb was mostly already used on Wikipedia [14:19:01] i doubt any site was ever asked for permission to link them [14:20:48] so imagine i were to make a bot that continuously queries netflix in various countries, snatches the ids of all videos available in that country, and uploaded the ids to wikidata...is that gonna harm wikidata in anyway? [14:23:14] i read online that some APIs request that you don't cache their data for more than 30 days. would it jeopardize wikidata if data retrieved from a commercial API was added to it permanently? [14:27:23] does a netflix ID allready exist as property? [14:43:21] Aⅼⅼaһ ⅰs doing [14:43:21] ѕᥙn ⅰs not dοing Aⅼlɑһ is dοіᥒɡ [14:43:21] moⲟn is ᥒοt ԁⲟіng Aⅼⅼaһ іѕ dⲟing [14:43:41] yesy [14:44:14] but i assume they're added manually [14:44:54] likely [14:45:41] i see no harm doing it by a bot, if you can make sure that added info is correct [14:46:29] ok [14:47:06] also, netflix ids should be localized to countries [14:47:26] not all items are available in every country [14:47:41] would i do this with a qualifier on the property? [14:47:41] but the ID itself is global? [14:47:53] i believe so [14:53:40] hm. any foreign site gets redirected to the .com site [14:54:17] this is the link to pulp fiction [14:54:18] https://www.netflix.com/title/880640 [14:54:59] it's not available in spain but it has the metadata [14:55:26] it would be great to add the availability to wikidata too [14:57:20] robclouth: do you get a 404 error here? https://www.netflix.com/title/70012571 [14:58:17] no but it immediately redirects [14:58:27] to the home page [14:59:01] what was it? [14:59:08] it has been the ID for "Battle Royale 2" [14:59:21] maybe the ids aren't global then [15:00:08] i think they are, but some have been taken offline [15:00:21] ok [15:00:29] https://webcache.googleusercontent.com/search?q=cache:pJfj2bNO8lsJ:https://www.netflix.com/title/70012571+&cd=1&hl=de&ct=clnk&gl=de [15:01:18] what did you see with the pulp fiction link? [15:01:28] i saw pulp fiction :) [15:01:49] is it available in your country? [15:02:58] do you think the availability should be a whole new property or qualifiers on the netflix id prop? [15:04:34] I don't think anyone here can give you an answer about legality and whether it could cause problems for wikidata, you would presumably have to contact https://meta.wikimedia.org/wiki/Legal for that [15:04:50] i think it is [15:05:23] but I find it unlikely that storing ids would be problematic [15:06:57] if you break their ToS in some way when using their api, they might block *you* from accessing it [15:08:47] but they aren't gonna be like "now take down all references to netflix from wikidata" [15:10:52] companies are very strange, so they might suddenly decide to do that :P but I think it's unlikely [15:12:13] the EU has something called "right to mine" which means you can collect data by scraping for research purposes [15:12:17] anyway removing them if we needed to wouldn't be a problem, we have the tools for that, it would be more problematic if they decided to try and sue us for linking to them [15:12:23] whether this covers then uploading that data, well.. :) [15:14:41] nikki: when linking is advertisement for them... :) [15:15:53] in a similar manner, wikidata should also support all the other stream-on-demand sites :p [15:16:15] amazon ASINs were rejected [15:16:36] ok so i've written a bot that scrapes the entire netflix catalogue in my country using their private api. i would like to add availability status to wikidata. would you recommend adding it as a qualifier to P1874 (netflix id)? i was thinking either Location (https://www.wikidata.org/wiki/Property:P276) or ValidInCountry (https://www.wikidata.org/wi [15:16:36] ki/Property:P3005) [15:17:27] we have asins now, https://www.wikidata.org/wiki/Property:P5749 [15:17:45] "valid in country" seems like the most appropriate [15:18:12] and since qualifiers should work fine, I don't think we would need a separate property [15:18:18] compare that to https://www.wikidata.org/wiki/Wikidata:Property_proposal/Amazon_Standard_Identification_Number [15:18:20] but the id is global [15:18:51] what about IDs that have been taken offline :) [15:20:03] well, if you want to propose a new property, I can't stop you :P (and I don't care enough that I would vote against it either) [15:22:43] maybe i'll use "valid in country" then [15:23:13] i'll share the script, and then if anyone else wants to add the availability of their own country they can [15:24:51] it would be *great* to have a frequently updated catalogue of each entity and where you can watch/listen/read it right now [15:25:34] but maybe that's forcing wikidata to be something it's not meant to be [15:26:50] you would also want to include a qualifier of "valid as of " :) [15:27:28] but then you'd need qualifiers of qualifiers [15:28:28] not a bad thing [15:28:34] https://help.netflix.com/en/node/60541 [15:30:28] titles can be removed from netflix, so storing this information would be valueable, aswell [15:36:05] i'm not sure it would really tell you much though since you'd need to go to the id to see if it's still available anyway [15:36:54] "valid until date" would be amazing but we're never gonna get that [15:38:06] you will know 30 days in advance, if something expires [16:08:36] Maybe use https://www.wikidata.org/wiki/Property:P3301? [16:15:01] hm [16:17:37] but do note https://www.theverge.com/2018/4/2/17187910/netflix-european-users-home-catalog-traveling [16:27:12] sjoerddebruin: for users from the EU it now takes the subscription country, not where you are actually :) [16:27:31] i'm not subscribed to, anyway [16:36:22] bleh [16:37:52] Aⅼlah іs ԁοing [16:37:52] sun is ᥒഠt ԁoiᥒg Αⅼlah is doiᥒg [16:38:16] Either the examples for Isidore ID were wrong from the beginning, or their search form did change: https://www.wikidata.org/wiki/Property:P4491 [16:39:03] https://www.wikidata.org/w/index.php?title=Property:P4491&type=revision&diff=747824049&oldid=731489134&diffmode=source [16:40:46] wow: https://www.wikidata.org/wiki/Wikidata:Database_reports/Constraint_violations/P4491#%22Format%22_violations [16:41:18] but thinking that those format violations DO actually work correctly (: [16:42:42] that whole property is a mess [16:43:00] interestingly, isidorerecherche itself uses the idref-link [16:43:59] if you go to a work and then click on the author to get results for him, it uses not the name, but the link. https://www.rechercheisidore.fr/search/resource/?uri=10670/1.l5mg8l [16:44:35] and the IDREF is nothing else than the SUDOC ID (: [16:45:01] oh, well. I close my eyes and ignore it ^_^ [16:56:27] Alⅼɑh iѕ doiᥒg [17:13:22] Hallo. [17:17:16] I made this patch for Wikibase: https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Wikibase/+/460554/ . Similar patches for some other extensions have already been merged, but Wikibase is bigger, and difficult to install and test. (I don't have it locally.) [17:17:32] If anyone can review it and test that the relevant things appear on Special:ApiSandbox and Special:ApiHelp, I'll be very thankful. It's better to merge it quickly, because Wikibase messages change frequently and rebasing later won't be fun :) [17:17:43] I’ll take a look [17:18:24] addshore: If you want to hide the spam, I'd suggest to either set channel mode `+r`, which will prevent the bots (and other non-authenticated users) from joining in the first place, or `+zq $~a` will make it so only people registered and authenticated will be able to be seen by non-ops (if taking the latter approach, you will also want to `+o Sigyn`). [17:22:20] aharoni: client/i18n/ also has some API messages, do you intend to do those in another commit? [17:22:29] (or want someone else to do them? ^^ ) [17:22:32] Lucas_WMDE: yes, in another commit [17:22:36] ok [17:23:49] ah, now I understand the context for your question a few days ago :) [17:23:59] I was just about to comment on the wikibase-api-* messages when I remembered :D [17:25:52] Lucas_WMDE: Yep, it's related. They are now marked as ignored, which means that they aren't translated at all. It may be OK to allow translating them, unless there's a strong reason not to. [17:26:14] okay [17:26:37] that probably makes sense [17:28:00] i think API docs where marked as not translated because in the past, api self-documentation was always in the wiki's content language, which would have been english anyway. so we wanted to avoid wasted effort. [17:28:25] but the new api documentation interface has been around for a couple of years now, so i guess it would be nice to start translating that stuff now :) [17:28:57] Some people think that it's pointless to translate API messages because they are only useful for developers and developers know English anyway. The first part is true, but the second isn't: I know at least two people who develop gadgets and don't know English well, and they really appreciate the fully translated documentation at Special:ApiSandbox. [17:29:01] actually, i think in the past api self-doc was always english, even. just like api error messages [17:29:41] i think it's very useful. in the past it was just impossibel to even ask mediawiki for translated docusmentation :) [17:30:11] But because these messages are indeed useful only to developers (of gadgets, extensions, tools, etc.) I am gradually separating them from the rest of the extensions' messages. [17:30:37] So that it would be easier for translatewiki volunteers to prioritize their work. [17:31:18] Lucas_WMDE: This is also the context for https://phabricator.wikimedia.org/T204379 [17:31:26] oh nice i was always annoyed by that [17:31:43] aharoni: :+1: [17:33:05] DanielK_WMDE_: the big switch to translatable apihelp messages happened in late 2014. I remember because that's when my son was born, and I was so busy with him that I couldn't keep maintaining translations to Hebrew every day as I had been doing before that. When I came back to it after a few months, I was shocked with about 2000 new messages to translate, but I completed them nevertheless. [17:33:36] aharoni: wow, 2000 messages! [17:34:20] yeah, I think anomie did it back the first big batch, it was huge [17:34:42] Lucas_WMDE: thanks for the merge. I'll find somebody to merge the translatewiki part. [17:35:22] I have 12,3k published Facebook translations :P [17:35:31] evil sjoerddebruin [17:36:07] It reminds me with things like "Your Dutch contributions have been used over 136B times this month." now and then. [17:36:10] Highly doubt that. [17:36:43] Or maybe every pageview counts... [17:39:10] going home now, I’m excited to see what barrage of notifications will be waiting for me tomorrow ^^ [17:39:19] oops, wrong window, sorry :D [17:42:03] do any of you know if translatewiki supports manually entering suggestions for the actual translators to check? [17:57:02] * nikki wonders how we ended up with such a narrow field for entering lexemes when so many german speakers were involved [17:58:59] I can see four and a bit letters, it's ridiculous [18:53:37] Allah iѕ ⅾoing [18:53:37] ѕᥙᥒ іs nⲟt dοing Allɑh is ԁοiᥒɡ [18:53:37] mοoᥒ ⅰs not doіᥒg Aⅼⅼah iѕ dⲟiᥒɡ [20:10:21] Aⅼⅼah iѕ ԁoiᥒg [20:10:21] suᥒ iѕ ᥒot doinɡ Alⅼah ⅰѕ dοіᥒg [20:15:54] Aⅼlaһ is doinɡ [20:15:54] suᥒ is ᥒഠt ԁഠіᥒg Allah іs doіng [20:15:54] ⅿooᥒ is ᥒഠt dοіng Alⅼaһ iѕ doіnɡ [20:27:23] Freso, when the chat is set to only authenticated users some clients (like mine!) first show up banned, as the log-in does not clear before the chat window is opened. I like to lurk and read the discussions. These guys are annoying, but it seems to be the same text, and Sigyn is good at killing them promptly :) [20:28:01] it still shows up as someone talking in the channel though [20:29:53] Back in the old days on message boards and news rooms, you could keep a personal kill file. Persons or messages could be entered, and you were blissfully ignorant of what they were saying :) [20:30:13] they keep changing the letters they use, probably to avoid that [20:30:36] I just saw that. There are unfortunately lots of homoglyphs in Unicode :( [20:30:57] and they don't use the same nicks or hostnames either [20:34:10] They have a twitter account that tweets the same all over the place, and which uploads strange pictures [20:34:36] RECOVERY - High lag on wdqs2003 is OK: (C)3600 ge (W)1200 ge 1150 https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen [20:35:22] May come from here: https://github.com/KristinaEtc/allah-message [20:36:34] They are really all over the place. [20:39:37] it's not exactly the same [20:39:53] mainly, that doesn't use homoglyphs [20:57:07] the cleanest way is to set up a bot that autovoices everyone after 10 seconds [20:57:58] Allɑһ is doinɡ [20:57:58] sᥙn ⅰs ᥒоt doіᥒg Allaһ ⅰѕ doⅰᥒɡ [21:11:16] i'm pretty sure the sun *is* doing stuff, [21:11:16] fusion amonf other things [23:33:39] Allɑh іѕ ԁⲟіᥒɡ