[04:10:29] community discussion on speeding up [[en:Donald Trump]] by removing repetitive wikilinks, in case anyone `here would like to provide some guidance: https://en.wikipedia.org/wiki/Talk:Donald_Trump#Wikilinks_to_works/publications_in_refs [19:00:20] dpifke: Here's the Github issue: https://github.com/WPO-Foundation/webpagetest/issues/1395 [19:01:05] So if I understand correctly: When you looked the https://wikitech.wikimedia.org/wiki/Performance/WebPageTest#Setup_EBS setup was broken right, so you fixed it? [19:28:52] When I looked at it, the disk was full and it was throwing 500 errors because of that. There was an empty 300G partition mounted on /data, so I used that for /var/www/webpagetest/www/results. [19:29:25] If /data is now full, that means we went from ~70G in results to ~300G, so it definitely implies something isn't removing old data. [19:30:48] I'll dig into it a bit more after lunch. [19:33:04] The other problem I noticed was /var/www/webpagetest/www/tmp was a ~500M tmpfs partition, which was also full. After moving stuff to /data, I turned that back into a regular directory. [20:35:26] Interesting, when codfw is primary, redis mainstash still sees about 5K/s queries (normally ~15Ks) [20:35:31] https://grafana.wikimedia.org/d/000000174/redis?orgId=1&from=now-1y&to=now&var-datasource=eqiad%20prometheus%2Fops&var-job=redis_sessions&var-instance=All [20:35:39] https://grafana.wikimedia.org/d/000000174/redis?orgId=1&from=now-1y&to=now&var-datasource=codfw%20prometheus%2Fops&var-job=redis_sessions&var-instance=All [20:35:51] I guess that's all monitoring traffic or something [20:36:29] 4500/s from abusefilter-profile, 500/s from ChronologyProtector. [20:36:52] anyway, do ignore this snipe by all means. it doesn't matter :)