[00:01:45] (03PS1) 10Gergő Tisza: Remove czwiki, there is no such thing [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/148896 [00:02:35] (03PS1) 10Gergő Tisza: Remove czwiki, there is no such thing [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/148897 [00:13:27] (03CR) 10Neilk: "To be clearer: the issue with Flickr uploads only happens when we are only uploading from Flickr. Amazingly if you mix it with a non-Flick" [extensions/UploadWizard] - 10https://gerrit.wikimedia.org/r/68835 (https://bugzilla.wikimedia.org/39746) (owner: 10MarkTraceur) [01:55:36] 3MediaWiki extensions / 3TimedMediaHandler: Play button is over top of controls since z-index change - 10https://bugzilla.wikimedia.org/68479#c2 (10Bawolff (Brian Wolff)) ich, that's much worse than the audio issue. Maybe 3bc774a6ccb30ac6 should be reverted. But having the play button not be clickable in a p... [12:10:53] 3MediaWiki extensions / 3GWToolset: GWT freezing up - queues appear to be getting stuck - 10https://bugzilla.wikimedia.org/68506 (10Fæ) 3NEW p:3Unprio s:3normal a:3None I have previously noticed my longer running uploads using GWT may have an hour where no uploads were happening, and put this down to... [14:17:41] (03CR) 10Gilles: [C: 032 V: 032] Remove czwiki, there is no such thing [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/148896 (owner: 10Gergő Tisza) [14:17:49] (03CR) 10Gilles: [C: 032 V: 032] Remove czwiki, there is no such thing [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/148897 (owner: 10Gergő Tisza) [14:25:36] (03CR) 10Gilles: [C: 032 V: 032] Use correct DB for optout queries [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/148838 (owner: 10Gergő Tisza) [14:26:32] (03CR) 10Gilles: [C: 032 V: 032] Record optout stats for users with 1+ edits and 100+ edits [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/148842 (owner: 10Gergő Tisza) [14:42:34] (03PS1) 10Gilles: Make the generate SQL folder if needed [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149014 [14:42:58] (03CR) 10Gilles: [C: 032 V: 032] "Self-merging trivial change" [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149014 (owner: 10Gilles) [14:57:40] (03PS1) 10Gilles: Fix global tsp by generating it like the others [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149017 [14:58:09] (03CR) 10Gilles: [C: 032 V: 032] "Self-merging, I'm saving time by working directly on the machine and trying to make it all work" [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149017 (owner: 10Gilles) [15:01:38] (03PS1) 10Gilles: Make global sql work [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149018 [15:01:49] (03CR) 10Gilles: [C: 032 V: 032] Make global sql work [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149018 (owner: 10Gilles) [15:04:41] (03PS1) 10Gilles: Revert "Make global sql work" [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149022 [15:04:50] (03CR) 10Gilles: [C: 032 V: 032] Revert "Make global sql work" [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149022 (owner: 10Gilles) [15:04:55] (03PS1) 10Gilles: Revert "Fix global tsp by generating it like the others" [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149023 [15:05:03] (03CR) 10Gilles: [C: 032 V: 032] Revert "Fix global tsp by generating it like the others" [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149023 (owner: 10Gilles) [15:07:59] (03PS1) 10Gilles: Copy over global.sql [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149024 [15:08:41] (03PS2) 10Gilles: Copy over global.sql [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149024 [15:09:02] (03CR) 10Gilles: [C: 032 V: 032] Copy over global.sql [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149024 (owner: 10Gilles) [15:26:07] 3MediaWiki / 3Uploading: Generate thumbnails based on buckets - 10https://bugzilla.wikimedia.org/67525#c11 (10Gilles Dubuc) Gergo, I don't have access to deployment-upload, could you give it another try, now that the fixed beta config is out? [15:29:52] (03PS1) 10Gilles: Point optout datasource to the correct server [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149026 [15:30:19] (03CR) 10Gilles: [C: 032 V: 032] Point optout datasource to the correct server [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149026 (owner: 10Gilles) [15:53:00] (03PS1) 10Gilles: Run optout global.sql last and fix queries [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149032 [15:53:12] (03CR) 10Gilles: [C: 032 V: 032] Run optout global.sql last and fix queries [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149032 (owner: 10Gilles) [15:56:17] tgr: I think that the git hook doesn't work [15:56:44] json files that existed before still exist, but the optout ones are blank [15:57:07] I've also noticed that locally it wasn't running on git pull, or not doing what it's supposed to anyway [15:57:42] ah, wait, I hadn't installed it locally. let's try [16:01:37] ah, it's just another trailing comma somewhere. unrelated to the git hook, which seems to work fine [16:03:21] (03PS1) 10Gilles: Fix trailing comma in JSON [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149034 [16:03:34] (03CR) 10Gilles: [C: 032 V: 032] Fix trailing comma in JSON [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149034 (owner: 10Gilles) [16:14:40] (03PS1) 10Gilles: Fix datasource field order [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149037 [16:14:52] (03CR) 10Gilles: [C: 032 V: 032] Fix datasource field order [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149037 (owner: 10Gilles) [16:20:02] (03PS1) 10Gilles: Add very active users to tsvs [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149039 [16:20:15] (03CR) 10Gilles: [C: 032 V: 032] Add very active users to tsvs [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149039 (owner: 10Gilles) [16:31:56] (03PS1) 10Gilles: Add very active users to graphs [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149040 [16:32:07] (03CR) 10Gilles: [C: 032 V: 032] Add very active users to graphs [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149040 (owner: 10Gilles) [16:33:11] tgr: http://multimedia-metrics.wmflabs.org/dashboards/mmv_enwiki#opt_in_opt_out-graphs-tab [16:47:32] (03PS7) 10Gilles: Track loading time for MediaViewer and the file page [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/148021 (owner: 10Gergő Tisza) [16:52:12] (03PS8) 10Gilles: Track loading time for MediaViewer and the file page [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/148021 (owner: 10Gergő Tisza) [16:52:54] (03CR) 10Gilles: [C: 032 V: 032] Track loading time for MediaViewer and the file page [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/148021 (owner: 10Gergő Tisza) [16:55:21] (03PS1) 10Gilles: Create duration sql folder when needed [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149041 [16:55:37] (03CR) 10Gilles: [C: 032 V: 032] Create duration sql folder when needed [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149041 (owner: 10Gilles) [16:57:43] (03PS1) 10Gilles: Fix %wiki% clause in duration template [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149043 [16:57:54] (03CR) 10Gilles: [C: 032 V: 032] Fix %wiki% clause in duration template [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149043 (owner: 10Gilles) [17:12:46] (03PS1) 10Ori.livneh: Fix use of undefined const NS_TIMEDTEXT [extensions/TimedMediaHandler] - 10https://gerrit.wikimedia.org/r/149046 [18:00:04] (03PS3) 10Gilles: Track loading time for MediaViewer and the file page [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/148024 (owner: 10Gergő Tisza) [18:07:22] (03PS4) 10Gilles: Track loading time for MediaViewer and the file page [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/148024 (owner: 10Gergő Tisza) [18:07:39] (03CR) 10Gilles: [C: 032 V: 032] Track loading time for MediaViewer and the file page [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/148024 (owner: 10Gergő Tisza) [18:21:15] (03PS1) 10Gilles: Restore graph that was incorrectly deleted + git ignored [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149072 [18:21:28] (03CR) 10Gilles: [C: 032 V: 032] Restore graph that was incorrectly deleted + git ignored [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149072 (owner: 10Gilles) [18:26:05] 3MediaWiki / 3Uploading: Generate thumbnails based on buckets - 10https://bugzilla.wikimedia.org/67525#c12 (10Tisza Gergő) Works as expected. [18:29:07] gi11es: to access deployment-*, you can ask project membership for the deployment-prep labs project on the qa or ops channel [18:29:21] it is a labs project, so no red tape involved [18:32:26] fabriceflorin: FYI reedy just merged the config change to turn MMV off on private wikis [18:32:45] I'm not going to rush to change it; it seems OK for now, I think we still have that CORS bug [18:33:03] (03PS1) 10Gilles: Restore global graph that was deleted/git ignored accidentally [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149079 [18:33:13] (03CR) 10Gilles: [C: 032 V: 032] Restore global graph that was deleted/git ignored accidentally [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149079 (owner: 10Gilles) [18:33:32] marktraceur: Thanks for the update. So does that mean Media Viewer is now disabled on private wikis? If someone wants it for a private wiki, how can they enable it? [18:34:07] They can't right now; we talked about adding a different config setup for "disabled, but with an opt-in preference" but never did it [18:34:12] Or made a card for it that I remember [18:34:44] marktraceur: OK, thanks for the clarification. I will add a card about this for the distant future. [18:34:47] Sure [18:35:04] we also haven't made a bug report for the bug that the remove patch refers to :/ [18:35:08] marktraceur: fabriceflorin: 2 new graphs on the limn dashboards, courtesy of tgr. last one in the optout section and first one in "media viewer vs file page" are new [18:35:33] gi11es: Thanks for letting us know. Let me check them and get back to you with any questions. [18:36:29] I don't think it makes sense to disable an extension saying "it has bugs!" when no one seems to know what said bugs are [18:36:46] tgr: We've had reports on the staff list, at least [18:37:24] then can we have a bugzilla ticket which contains those, at least? [18:37:34] Ahhhh, yes. [18:37:44] although ideally you would reproduce and identify the bug before you disable [18:38:23] tgr: https://bugzilla.wikimedia.org/show_bug.cgi?id=63282 ? [18:38:28] thanks [18:38:48] I'm not sure that's the same issue, it popped up in the "similar bugs" [18:40:18] I'm asking because I want to fix the CORS bug once I have some free time [18:40:30] it's not limited to private wikis [18:40:48] people who have sucky firewalls don't see Wikipedia images because of it [18:41:01] so if that's the only problem, we can reenable them [18:41:12] I filed https://bugzilla.wikimedia.org/show_bug.cgi?id=68522 [18:41:22] 3MediaWiki extensions / 3MultimediaViewer: Private wikis don't play nice with Media Viewer - 10https://bugzilla.wikimedia.org/68522 (10Mark Holmquist) 3NEW p:3Unprio s:3normal a:3None In some browsers (Chrome?), a private wiki's images are hidden from media viewer, because the img_auth.php request is... [18:43:02] marktraceur tgr : I defer to you guys as to whether or not it’s safe to re-enable once bugs have been identified and severity assessed. It’s not a top priority for me, but if the technology works, we should allow sites who want it to try it out. For now, I filed this Mingle ticket, which we can take on when the time comes: Allow private wikis to enable Media Viewer on their sites - [#810] : [18:43:02] https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/810 [18:59:11] gi11es: those performance graphs are baffling [18:59:34] anon vs loggedin you mean? [19:00:21] presumably loggedin are very likely to have a warm cache [19:00:29] there is much less difference between anon and logged-in than I expected, but that's ok [19:00:47] I mean the comparision between MediaViewer and file pages [19:01:26] clearly MediaViewer can't affect file page loading times [19:01:49] so maybe only users with slow connections use file pages now? [19:02:19] this just confirms what we were seeing on the e2e test [19:02:32] why is it surprising? [19:02:48] tgr gi11es : Could you guys please explain in laymen’s terms what has changed in the performance graphs, compared to what we had before? Is ‘global network performance’ still the time it takes to load an image after you click on an image? Is that based on global data from all image loads on production? Or just a sample? http://multimedia-metrics.wmflabs.org/dashboards/mmv#overall_network_performance-graphs-tab [19:03:05] because the median times for file pages before MediaViewer was deployed is much faster than the median for MediaViewer now [19:03:35] not sure how that's possible unless MediaViewer is actually slower [19:04:05] Sorry, I see now that the new performance graph is on this URL: http://multimedia-metrics.wmflabs.org/dashboards/mmv#media_viewer_vs_file_page-graphs-tab [19:05:46] also, a hundred logged-in file page views a day globally? that seems unrealistically low [19:06:01] eh, crap, I didn't take sampling into account [19:07:48] anyway, this doesn't seem to be the right kind of graph to answer which one is faster [19:08:28] maybe sw should sum loading times and see how the deployment changed that sum? [19:10:10] tgr: It would still be useful to describe in laymen’s terms how the ‘Loading time’ is calculated now, compared to before. Also, it seems that the image load times are much closer now in the 90th percentile for media viewer (5 secs.) and file page (7 secs. logged-in, 6 secs logged-out). What do you think changed to make that happen? Is it because we’re tracking on production? or tracking different things? [19:10:49] tgr: re: sampling, are different wikis sampled differently for navigationtiming and duration logger? [19:11:15] fabriceflorin: MediaViewer loading time is time measured between thumbnail click and replacing the blurry image with the real one [19:11:16] we adjusted actions to be different from one wiki to another, but not duration afaik [19:11:47] file page loading times are whatever the browser reports for how long it took to load the page [19:12:16] gi11es: not sure, but anyway, I don't think that explains any strangeness in the graphs [19:12:33] it just means the reported sample sizes are incorrect [19:12:51] or, well, misleading because we should report the population size instead [19:12:57] anyway the mix of people who were using media viewer vs the file page was also non-standard before the launch. it's hard to know what mix of user groups we're looking at before and after [19:13:28] also applies to their bandwidth, etc. [19:13:58] tgr gi11es : Thanks for the clarifications, very helpful. So do you think these new graphs still need more work before we share them? Or do we have enough evidence to suggest that load times are in fact closer than we thought? [19:14:09] the june 20th jump is a bit more concerning, I wonder what happened that day [19:14:11] the point is that if you look early data where no one used MediaViewer, the file page loading times were much faster than either current file page loading times or current MediaViewer loading times [19:14:42] fabriceflorin: I honestly have no idea what the graphs suggest [19:14:54] tgr: that's misleading because it ignores the very active users who signed up for media viewer and thus were taken out of the file page measurement [19:15:22] and beta users are generating a lot more pageviews, thus are more likely to be sampled [19:15:37] a lot of factors make the before/after a bit meaningless [19:17:27] tgr gi11es: OK, it sounds like we should hold off on sharing widely — or even in Erik’s report, which is due today, before the German RfC starts. Is there a way to put the questionable graphs on a different ‘Beta’ tab until we have vetted them? Right now, every community members is using this URL to check our data, and they are likely to be confused by the discrepancy between what we are saying and what the graphs show. I would [19:17:28] recommend making that change in the next few hours, before Erik posts his report at end of day. It would be very embarrassing for the foundation to have a mismatch between our statements and the data. [19:17:52] it's just difficult to draw conclusions from these graphs [19:18:13] the data is nonetheless interesting [19:18:42] maybe the pages where media viewer doesn't ever kick in, even when media viewer is on, are slow to load on average? [19:19:24] gi11es: Yes, it is interesting, but I think we should stick to the previous graphs for now, for consistency, until we have confidence in the new graphs. How hard would that be to do today? [19:19:31] the reversal isn't what worries me most, it's the jump on june 20th [19:20:02] fabriceflorin: the new graphs provide reliable data, it's just that they can't tell us the causes [19:20:15] they're as reliable as the other user-measured ones we had before [19:22:00] gi11es: do you remember the sampling ratio for duration logs? [19:22:18] not on the top of my head, it's in the code [19:22:32] gi11es: So can we now say with confidence that image loads for file pages and media viewer are about the same — and a lot closer than we thought? Should I revise our public pages to say that before Erik posts? Should I let him know as well, so he can edit his report accordingly? [19:23:15] fabriceflorin: we can't draw that conclusion. we can't say which one is faster which this reversal on the graph that doesn't have an explanation [19:23:20] *with [19:23:54] in addition to that, it looks like media viewer got slower durably on june 20th and so far I can't find anything that happened around that time that would explain it [19:24:24] gi11es: that was when MediaViewer got deployed to all wikis [19:24:35] the conclusion of seeing this data is "needs more digging before saying X is faster than Y" [19:24:43] although that still does not explain why [19:24:53] tgr: so, somehow that affected enwiki? doesn't make sense [19:25:00] oh no sorry [19:25:05] I'm looking at the global graph [19:25:06] gi11es: OK, so maybe we should put a disclaimer in the new graphs, to say that they are new, with different results than before, and that we are studying them to understand the discrepancy? Also, could we explain in laymen’s terms how our methodology changed between the old and new graphs? [19:25:24] old graph is measured on a test server [19:25:33] new graph comes from data collected on users' machiens [19:25:39] machines [19:26:17] fabriceflorin: the old graph compares MediaViewer and the file page under identical circumstances [19:26:29] like gergo said, one possible explanation is that media viewer is slower, and that somehow the remaining file page impressions (which are likely to be overwhelmingly non-image pages visited by anons) happen to be slower than media viewer [19:26:30] the new graph looks at actual usage [19:27:03] gi11es: Thanks, that’s very helpful. I can write a quick update and let Erik and Howie know, if you like. Or you can do it, as you prefer. We have a meeting at 3pm to plan our next steps before posting the report for German Wikipedia RfC, which starts tomorrow. [19:27:24] tgr: in navigationtiming data, can we see which pages they are exactly? we could check if it's mostly video pages and the like for recent anon traffic [19:27:25] also, the new graph does not contain all image views since we only log the first loading time per page [19:27:36] that could throw off things somehow [19:27:51] gi11es: good point [19:28:04] they have page ids, but that only works on Commons [19:28:05] fabriceflorin: please write it, I've already done my hours for today, I have personal things to attend to [19:28:08] they do not have page names [19:28:10] tgr: Does this mean that if I click next and previous in MV, those image load times are no longer counted? That could make a difference. [19:28:36] fabriceflorin: yes, those are not counted [19:28:46] gi11es: OK, I will write something short, let you guys elaborate as a follow up. [19:28:51] closing and reopening the lightbox does not count either [19:29:24] tgr: OK, that helps. What would it take to count these events as well? Do we think they might increase performance? [19:29:41] fabriceflorin: I'm halfway writing a short mail already [19:29:48] gi11es tgr : [19:29:52] it's not counted because media viewer would be kind of "unfairly" advantaged on those. although in practice it's arguable that the true average time to load an image (next or otherwise) is what we might decide matters [19:30:23] Can I assume that the opt-in/out graph by wiki at the bottom of that tab is OK? http://multimedia-metrics.wmflabs.org/dashboards/mmv#media_viewer_vs_file_page-graphs-tab [19:30:27] there is nothing unfair in being quick :) [19:31:06] we didn't measure it because we were worried about the time of the first load, not the time of the prev/next navigation [19:31:16] it depends on whether you want to optimize the first experience or the average experience. slower initial experience might make people leave and not have a 2nd at all [19:32:12] gi11es tgr : I’m not an expert in what is the best practice for reporting these data, but it would seem helpful to have two measurements, maybe on separate graphs: 1) how long it takes for the first image 2) how long it takes if you count all image loads, including next/previous, etc. [19:32:12] ideally the first load would be comparable to the file page and the subsequent ones faster, then you have the best of both worlds [19:32:31] but it's hard to justify that media viewer is faster if its first image load isn't [19:32:43] because you create a more likely visit dropoff for anons [19:32:58] well, ideally the first load would be much faster than the file page, and subsequent loads even faster than that :) [19:33:12] and that seems to be the case right now [19:33:40] but the graph kind of suggests that might be an artifact and not actual speedup [19:33:45] another factor is primed cache for thumbnails. maybe if the thumbnails had been pregenerated, media viewer would have been faster pre-launch. [19:33:57] gi11es: Yes, that is a good point, and I think we need to be upfront that the first image takes about as long to load as the file page, if the data confirms that. But we could also point out in the next statement that if you count all the subsequent load events, then it appears that the media viewer loads faster, if the data shows that. [19:34:04] we can't know that, and the launch is what triggered the thumbnail generation for most images on enwiki [19:34:52] file pages, pre-launch, on the other hand, were extremely likely to have their thumbnail already generated compared to media viewer [19:35:54] and as we know the difference there can be quite striking. unfortunately with the 30 day limit on the overall network performance graphs, we can't see those dates anymore. we might want to increase that [19:36:47] gi11es tgr: Sounds like we still have a lot of questions, and limited resources to answer them. Not sure how to proceed next, I guess we could create one card for each research question and try to tackle them over time. Let’s keep the conversation going to get more confidence in what we can state based on the current data, then see if we can pick a task or two that could shed more light on open questions.Thanks for getting us to that [19:36:47] point. At least, we can say that there is uncertainty about how the two methods compare. [19:39:33] tgr: I've just thought of something else. for media viewer duration, do we filter out loads where the image is already in the browser's cache? [19:39:36] we can't do that for the file page [19:40:04] that's something that could create a huge imbalance in favor of file pages if revisits of file pages with browser-cached images are common [19:40:07] I don't think we do that [19:40:25] also, not sure we should, from a user experience point of view [19:40:36] we do filter that on the network graphs [19:40:47] but let me check to be sure [19:42:01] if we don't, and we start to measure subsequent (prev/next) image loads, then the data will be artifically in favor of media viewer... you're much more likely to prev/next to an image you've arleady seen in media viewer than you are with a file page [19:42:04] tgr gi11es : Just to confirm, the opt-in/out graph by wiki is reliable data, right? It’s just the load time graphs that we are not sure about? http://multimedia-metrics.wmflabs.org/dashboards/mmv#media_viewer_vs_file_page-graphs-tab [19:42:49] gi11es: true [19:42:53] fabriceflorin: yes. tgr wrote the optout one, there's nothing on there that looks surprising to me [19:43:00] anyway, we don't filter correctky [19:43:04] *currently* [19:43:35] gi11es: I think having two image load graphs would be useful, as proposed above: 1) just the first image 2) all the images. Both measurements have value, IMHO. [19:43:36] and we only track the very first lightbox opening per page load, so I don't think that's unfair [19:43:41] ok, right now that's probably acceptable as the likelyhood to open an image first in media viewer that is browser-cached is probably the same as for the file page [19:44:19] fabriceflorin: yes but the question is whether or not we filter out image loads for prev/next that are images which were already cached by the browser [19:44:25] those load almost instantly, which skews the results [19:44:50] i.e. it would be really easy for media viewer to look like it's super fast just because tend to prev/next a bunch across images they've just seen [19:44:57] *people [19:45:01] prev/next is hard to compare [19:45:11] it will be usually zero, since it is preloaded [19:45:17] that too [19:45:44] but in the file page version, maybe the user would never click on that image since it is not interesting for them [19:45:44] arguably, people can open file pages as tabs and they won't experience the loading time as a nuisance either [19:45:51] so not entirely fair to count [19:45:53] "manual preloading" [19:46:30] gi11es: Thanks for explaining all this so patiently :) I guess if we had two numbers, we could show the best case and worst case side-by-side. The real answer maybe somewhere in the middle, and we could state that. [19:46:32] where the user can actually be a lot more agressive about the preloading that media viewer is [19:47:14] fabriceflorin: I think the bottom line is that it's going to be difficult to claim it one way or another without exposing ourselves to easy criticism on the methodology that led us to that conclusion [19:47:51] it's probably going to prove to be faster on average thanks to preloading, but that's not exactly fair as people could preload file pages as well, and don't take that "manual preloading" into account in the file page performance [19:48:13] we almost need to measure the preloading time that happens behind the scenes to make a fair comparison [19:48:45] gi11es tgr : I wish the analytics team had more bandwith for us, this is a case where having some guidance on best practices would be helpful. But I don’t think that they have a lot of time for us due to Wikimania. Still, it would be worth it to ask the question. From a priority standpoint, the opt-in/out data questions are probably a bit more important now than performance questions, but they are both valuable. [19:50:35] I'd be more inclined to believe the results of the E2E test which we already had, just because it's testing both tools in a sort of isolated bubble (as much as that is possible anyway). we just have to figure out what's going on with the real world measurements and maybe we'll find useful information while we do that digging [19:50:40] but that doesn't have to happen now [19:51:40] we've just discussed at least 3 big factors that can affect the data. it's going to be difficult to isolate them and make a true real world comparison. we'd probably have to run an A/B test on real users, wait for thumbnail prerendering to be rolled out, etc. [19:52:02] it's a big undertaking to run a better measurement where we eliminate all these factors [19:52:30] gi11es: Is there any chance we could put the E2E graph back up on the URL everyone is using, before or after the new graph? With a short statement saying what each does, and explaining which is the new one and the old one? I am worried that we are introducing confusion at a time when the community is already on edge. How hard would it be to put that graph back up, ideally at the top? [19:52:44] fabriceflorin: it's there, below the new one [19:53:05] I think tgr didn't put a description on the new one, though [19:53:16] tgr: could you add one? [19:53:27] in a sec [19:54:20] I don't mind switching them, but then we're starting to delve into fiddling to pick the presentation that makes media viewer look the best [19:55:11] (03PS1) 10Gergő Tisza: Take sampling factor into account for duration data [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149101 [19:55:23] gi11es: Oh, OK. Does LIMN let us put a short sentence below each graph, to explain what they are? Or can we add something in the title to separate them? (e.g. ‘Image Load Times - Media Viewer vs. file page (new experimental data)’ and ‘Image Load Times - Media Viewer vs. file page (old E2E data)’) [19:56:02] the title can be changed and a description below each graph can be written, if there isn't one already [19:56:30] (03CR) 10Gilles: [C: 032 V: 032] Take sampling factor into account for duration data [analytics/multimedia] - 10https://gerrit.wikimedia.org/r/149101 (owner: 10Gergő Tisza) [19:56:52] gi11es tgr : Good. Want me to take a stab at writing titles and descriptions for each? Does this require a SWAT, or can we do it anytime? [19:57:05] we can do it any time [19:57:13] we have control over the limn deployment [19:57:32] (03PS1) 10Gergő Tisza: Take sampling factor into account in duration data [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149102 [19:58:42] OK, I will prepare a Mingle ticket with proposed titles and descriptions now. Then you guys can tweak as needed and make the change when ready. [20:01:17] (03CR) 10Gilles: [C: 032 V: 032] Take sampling factor into account in duration data [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149102 (owner: 10Gergő Tisza) [20:01:30] tgr: I'm regenerating the tsvs right now [20:04:54] thanks [20:05:09] (03PS1) 10Gergő Tisza: More informative description for duration graph [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149109 [20:19:40] tgr gi11es : I just created a card for updating the title and description of the Media Viewer image load dashboards here — feel free to edit and make the change later today : https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/811 [21:37:25] gi11es: https://plot.ly/~tgr/1 [21:37:38] this is all loading times, union all'd [21:38:11] the deploy days still show a noticeable increase [21:41:02] (june 4 and june 20) [21:42:20] (03CR) 10Gergő Tisza: [C: 032] "Self-merge, trivial text change" [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149109 (owner: 10Gergő Tisza) [21:47:30] (03CR) 10Gergő Tisza: [V: 032] More informative description for duration graph [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149109 (owner: 10Gergő Tisza) [21:56:13] (03PS1) 10Gergő Tisza: Clarify description of performance Selenium test [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149172 [21:56:41] (03CR) 10Gergő Tisza: [C: 032 V: 032] "Trivial text change." [analytics/multimedia/config] - 10https://gerrit.wikimedia.org/r/149172 (owner: 10Gergő Tisza) [22:24:33] gi11es: just checked, there is a HUGE difference between domready and mediawikiloadend in NavigationTiming [22:24:42] like, +70%