[02:00:39] [[Tech]]; 66.87.121.209; [none]; https://meta.wikimedia.org/w/index.php?diff=15669582&oldid=15666385&rcid=7866767 [02:00:55] [[Tech]]; Defender; Reverted changes by [[Special:Contributions/66.87.121.209|66.87.121.209]] ([[User talk:66.87.121.209|talk]]) to last version by 79.98.159.114; https://meta.wikimedia.org/w/index.php?diff=15669583&oldid=15669582&rcid=7866771 [02:06:27] Holly sport Jeffrey [02:15:33] is this the new horse_ebooks? [02:21:12] Everything happens so much. [07:11:01] @notidy MatmaRex Sorry, I went to sleep. Can you check when the "Show extended metadata" under https://commons.wikimedia.org/wiki/File:Logo-Bogota-Humana_1.jpg#metadata became left-alligned and not centered. I liked it when that text was centered, but some change changed that like 2 weeks ago...can you checkwhich commit [07:11:06] I will let you know when I see MatmaRex and I will deliver that message to them [07:11:06] @notify MatmaRex Sorry, I went to sleep. Can you check when the "Show extended metadata" under https://commons.wikimedia.org/wiki/File:Logo-Bogota-Humana_1.jpg#metadata became left-alligned and not centered. I liked it when that text was centered, but some change changed that like 2 weeks ago...can you checkwhich commit [12:11:53] Josve05a: re metadata table, see https://commons.wikimedia.org/wiki/Commons:Village_pump/Archive/2016/05#binary_metadata_in_exif_causes_corruption_of_exif_table [12:12:32] It looks ugly though... [12:13:50] it always looked ugly :P [12:15:12] yeah, but when collapsed, it looks weird to hav that "big box" for a left justified text, that barely covers half the box [12:15:25] (less than*) [12:15:35] If there is no camera-EXIF etc. [13:52:02] hello, everybody [18:40:06] Krinkle, how well do you know that caching thing? I wonder if it should be a different call - the one for all clusters. [18:40:26] my problem is that DB slaves are not fast enough [18:41:18] so when i generate HTML, browser tries to get the image from graphoid service, which calls MW API to get the data, and the data comes from a stale slave [18:41:30] yurik: Two options [18:42:05] so i store the data in memcached, so the api call checks the memcached first [18:42:17] memcached tends to be much more up to date [18:42:24] yurik: 1) Use WANObjectCache, which is multi cluster aware: Use get(), and populate from db slave on get() miss, never use db master, call delete() if the data changes - only delete() events are broadcasted, get() and set() only uses local cluster. [18:42:54] or, 2) Use local cluster: Call set() when the data changes, and get() from there. [18:43:05] But that is somewhat fragile. [18:43:26] Krinkle, what do you mean by never use db master [18:43:30] since it may fall out of memcached, and then you don't know if it'll be in the slaves yet. [18:43:54] well, i hope that by the time it falls out of memcached, the repl is done [18:44:05] yurik: GET requests must never use DB master. That is an architectural decision that individual applications must embrace if not already. It requires a different way of modelling cache logic. [18:44:40] of course, that's why i was confused - i never use master for get. Were you implying that i should use master for #2? [18:45:00] yurik: No, but I do wonder what the API does if memcached is a miss. [18:45:10] gets data from db slave [18:45:14] and if that's a miss? [18:45:18] fail [18:45:28] Right, so reality is reflected from the slaves (e.g. 404 not found) [18:45:30] that makes sense [18:45:44] yeah, except that its a broken image :) [18:46:11] well, "things" shouldn't know about things that aren't on the slaves yet [18:46:20] also, if on change i do "delete", it won't solve anything, because by the time the new request comes in, slaves haven't updated yet [18:46:33] Yeah, I don't think this data is mutable, right? [18:46:42] You're using verifiable keys. [18:46:42] hash-based [18:46:53] content hash [18:47:03] yurik: So before I continue, read the overview on ObjectCache.php [18:47:14] i did - that's why i was asking you [18:47:18] It described the primary cache models we currently recommend. [18:48:06] i thought that using getMainStashInstance might be better as it is more DC aware [18:48:31] getMainStashInstance() will be replicated just like the DB. It's in fact in the same database right now. [18:48:35] It won't be faster to replicate [18:48:42] bummer [18:49:25] It is the only "cache" that is fully replicated (both set and delete, e.g. values always originate from the master data centre) [18:49:28] one option btw might be to really go against the dbmaster on api cache miss [18:49:38] whereas wanobjectcache is locally populated, but uses purge events to coordinate changes. [18:49:59] if the source of re-population after purge is a slave, it is not good [18:50:10] yurik: Okay, so one other thing to keep in mind - [18:50:11] because the slave is stale [18:50:22] We have ChronologyProtector [18:50:29] Which ensures a user always sees their own changes [18:51:14] yeah, i know about it, but we would have to pass that value all around, from browser to restbase to graphoid to api [18:51:31] It ensures the selected slave is up to date, or otherwise waits briefly in the request to ensure it is up to date [18:52:04] so if the same session used mediawiki to write a value to the database master and cache in memcached. [18:52:18] then you should be fine to read it from memcached, and otherwise (if it fell out immediately) it'll be available through DB_SLAVE [18:52:35] Krinkle, i know about it ^^^ , that protector works fine if you are getting sub-resource from the same origin [18:52:49] but for restbase, it might be different [18:53:08] because you have to pass a header from the browser to restbase and on to the service and to MW API [18:53:29] OK, so summarise for me what this value is (the graph json?), and how it ends up being written to the db master (dont know), and how it ends up retrieved (through api.php from the browser?) [18:54:37] yes and no: case #1 (static images): on parse, graph json -> hash, HTML generated with an image containing that hash, and graph-json stored in pageprops. [18:55:14] also it stores it in memcached [18:55:18] under the hash [18:55:31] browser requests image from restbase, restbase calls graphoid, graphoid calls MW API with the page name + hash, graph API looks in memcached, if not, looks in db slave (pageprops) [18:56:23] case #2 - dynamic images -- same as above, but it is now possible that the user clicks on the image, and the browser loads that same graph data directly from graph API (same as what graphoid uses on backend) [18:57:37] by graph api i mean MW API action provided by the graph ext [19:07:50] Krinkle ^ [19:08:17] * Krinkle was distracted for a sec [19:09:05] yurik: OK. so case #2 is fine. [19:09:47] But yeah, case #1 is interesting. I'm not sure how to stay within the cache/slave-only model and yet ensure somehow that a service is able to see the latest data on a GET request without being in the user's session. [19:10:03] I'd recommend asking Gabriel or Aaron. I think the might know. This sounds familiar. [19:11:23] Ah, got it. [19:11:24] "remoteSkinPath": "WikimediaEvents/modules" [19:23:28] Krinkle, what about it? [19:25:37] Sorry, that last bit was in the wrong channel [19:25:56] yurik: Gabriel/Aaron bit was for you. [19:26:09] gotcha [19:26:13] thx [21:29:18] I'm getting a weird error when trying to push commits to gerrit: "no such identity: /Users/kaldari/.ssh/your_production_ssh_key: No such file or directory" [21:29:30] "your_production_ssh_key" doesn't seem like it should be the name of my ssh file :) [21:32:02] kaldari: check out your ~/.ssh/config [21:32:17] it probably comes from there [21:32:43] it _could_ have that name if it existed, only the content actually matters [21:32:47] kaldari: i guess that at some point, you were configuring https://wikitech.wikimedia.org/wiki/SSH_access#Production ;) [21:37:12] lol, I guess so. I don't remember messing with that any time recently though. [21:38:29] do you know where the actual key is located ? [21:38:44] you can also do: ssh-add /path/to/that/key one time [21:38:54] and remove the IdentifyFile line from config [21:38:57] and it should also just work [21:39:11] eh, gotta go afk , be back soon [23:14:53] works now. Thanks!