[12:00:40] Hi, is there a nice visual tool to display the "subclass of" relations (upwards)? I have doubts on the current results of CLAIM[131:(TREE[5907][150][131])] AND CLAIM[31:(TREE[56061][][279])] ... which currently returns Rivers and Motorways, which it didn't do a few days ago. [12:13:41] aeroid: http://sergestratan.bitbucket.org/ [12:13:49] not sure how up-to-date it is though [12:14:28] ah actually it says [12:14:30] 19th [13:09:39] thanks ... I'll try my luck ... [14:35:10] hoo: Lag thing is happening again [14:35:20] didn't get paged yet [14:35:26] hm [14:35:32] b [14:35:33] readonlyreason:Waiting for 3 lagged database(s) [14:35:46] Exact time? [14:35:51] Feels like micro bursts [14:35:59] Indeed [14:36:06] Sleeping for 7.0 seconds, 2016-01-23 15:34:43 [14:36:11] Just after that one [14:36:36] Only 1 of 3 bots noticed, the other bots were waiting for replag to recover ;-) [14:38:37] Nothing in tendril [14:38:46] but if its just one bad query it might not get caught by it [14:43:04] I'm seeing my own bot in tendril :P [14:49:33] aude: How's the weather? [15:01:50] hoo: still have power :) [15:13:03] aude: Shall we run the rebuild items per site script maybe? [15:13:12] because of https://phabricator.wikimedia.org/T47839 [15:14:54] hoo: maybe [15:15:01] but the pages i found already had entries there [15:15:06] * aude just found another page [15:15:09] hm, very weird [15:15:35] seeing when it was parsed and checking that it's in ips [15:16:24] it's in ips [15:16:32] item was edited on jan 12 [15:16:41] Can you link the item? [15:16:52] yeah [15:17:00] wikipedia page cached on 20160123045938 [15:17:09] (suppose we did rollback to wmf10 last night) [15:17:29] https://en.wikipedia.org/wiki/Leduc_County and https://www.wikidata.org/wiki/Q6512776 [15:17:37] please nobody purge [15:17:51] thanks [15:17:52] * hoo wont [15:17:54] ok [15:18:31] and i foudn a whole batch of pages of other municipal districts like this (but not all including ones whose items were edited0 [15:18:52] the only common thing is they were edited and afaik related templates were not edited [15:21:08] aude: do they have the page prop set? [15:22:35] ones i checked, yes at least in page props table [15:22:50] you'll notice it also has the lua stuff [15:23:00] [[Category:Coordinates on Wikidata]] [15:23:01] 10[1] 04https://www.wikidata.org/wiki/Category:Coordinates_on_Wikidata [15:23:12] https://gist.github.com/filbertkm/364e52f028e65b314d86 [15:23:17] * aude kicks the bot :P [15:23:50] it has entity usage T + X + O [15:24:02] aude: You sure coordinates are from Wikidata? [15:24:06] and Number of Wikibase entities loaded: 1 [15:24:07] | latd = 53 |latm= 20|lats= 16|latNS=N [15:24:07] | longd = 113 |longm= 31|longs= 53|longEW=W [15:24:15] that's in the template [15:24:18] it means the coordinates are also on wikidata [15:24:22] ah ok [15:24:43] this helps tracks articles that have coordinates but are missing on wikidta [15:24:47] so we can import them [15:25:45] probably in wbc_entity_usage, it now also has S usage [15:27:50] hoo: https://gist.github.com/filbertkm/fc2022f97791188a1bef [15:28:05] it has T + X + O + S (but S has a different touched timestamp) [15:28:09] hm [15:32:57] Maybe it's possible to reproduce locally now [15:34:41] can you? [15:34:53] Not yet, but I have some idea [15:34:57] ok [15:35:27] I bet in the end it's going to be a stupid one line thing again [15:35:36] could be [15:36:23] thing is when i view, say action=history (uncached) then the links are in the sidebar etc [15:36:37] aude: Sure... these are not cached [15:36:51] Because we don't ahve the ParserOutput there [15:37:03] * hoo implemented that because there was a huge demand [15:37:04] so generating new parseroutput, it probbly works [15:37:17] or purging (please don't do) would probably fix it [15:37:28] Yeah, pretty sure it would [15:39:02] * aude needs to go shovel snow in a few minutes [15:42:37] uh… how much snow is there? [15:43:10] probably 50 cm [15:43:22] and it's still snowing until maybe tomorrow morning [15:43:30] or maybe 40 cm [15:43:36] wow, I don't think I ever saw that much snow [15:43:40] * aude has [15:43:57] e.g. https://www.flickr.com/photos/kmf164/4346885247/in/album-72157623280004843/ :) [15:44:27] hehe :) [15:44:56] * aude will go take some photos for commons, but won't be as nice as walking around the city in the snow [15:45:01] or skiing [15:46:19] or staying indoors where electricity makes it warm... woops :p [15:47:00] Alphos: quite possible will lose power, which is ok [15:47:26] ;) [15:47:53] where are you at btw ? no snow here in paris :-( [15:47:54] luckily that very barely happens in Germany... but also there are very few places here that actually have tha tmuch snow [15:47:57] dc [15:48:23] Q61 [15:48:30] aude: Still your dev summit trip? [15:48:34] multichill: yep [15:48:45] us east coast gets all the winter fun, so unfair :-( [15:49:05] * aude wish berlin would get more snow [15:49:55] anyway, need to go :) [15:50:03] in paris an inch of snow is considered catastrophic :/ [15:50:04] if still have power, then back later :) [15:50:08] take care [15:50:10] later ! [15:50:21] * aude hopes hoo can repl [15:50:25] reproduce the bug [15:50:37] yeah, that would be nice [17:35:14] * hoo wonders if aude still has power [17:37:44] * aude back :) [17:38:09] with power too :D [17:38:28] yep :) [17:38:34] Nice :) [17:39:02] I spent most of the time digging through Wikibase client/ jobs/ ... [17:39:44] we do power lines like https://commons.wikimedia.org/wiki/File:Snow_in_Pittsburgh_2010.JPG soemtimes :) [17:40:02] ours are not as bad, but there probably are above ground ones that we still depend on [17:40:34] hoo: any ideas about the bug? [17:41:41] No... :/ I suspected something is re-saving incomplete stuff to parser cache again [17:41:48] but can't see how this would happen here [17:42:22] I also looked into race conditions between different jobs touching pages [17:46:26] hmm, resaving [17:46:43] i think mobile was having an issue with that [17:46:50] * aude looks [17:47:41] https://phabricator.wikimedia.org/T124356 with the OutputPageParserOutput hook [17:47:48] aude: Interestingly enough, the entity usage rows for the page you showed me earlier on changed [17:48:01] but not due to a purge [17:48:09] but I guess a job ran just now [17:49:10] could be [17:49:25] that's how all the pages i found were, regarding entity usage [17:49:27] I'll look through the runJobs logs [17:49:52] they had page_links_updated as more recent [17:49:55] (see the page table) [17:50:01] vs. page_touched [17:51:02] aude: On that one page_touched is more recent [17:51:06] than links_touched [17:53:43] oh really? [17:53:51] page_touched: 20160123011956 [17:53:51] page_links_updated: 20160122060710 [17:54:04] hmmm [17:54:26] ones i saw had page_touched on jan 12 (soon after the wikidata item was edited) [17:54:34] and page_links_updated on jan 18 [17:55:16] if i view action=history, does that touch the page? [17:55:31] since it requires generating parser output? [17:55:35] shouldn't [17:55:38] ok [17:55:54] also 011956 would be last night (for me) [17:58:39] oh, wow [17:58:40] I got it [17:58:57] but not sure what exactly triggered it [18:00:20] either action=render or viewing diffs [18:01:29] oh [18:02:18] diff of the wikipedia page? [18:02:24] No, locally [18:02:27] oh [18:02:32] I reproduced it locally [18:03:44] what is "it"? [18:04:08] langlinks disappearing [18:04:11] oh [18:04:26] diff in the item? [18:05:03] I played with diffs of my client page and action=render [18:05:09] not sure what of that triggered it [18:05:28] render doesn't [18:05:37] for me [18:06:21] and doubt someone did action=render or diffs of all the pages (quite many, but not in the same category and whose items were edited) [18:06:47] which pretty certain is what caused page_touched to be updated for the ones i looked at [18:06:55] True [18:07:07] it was like 18 seconds later [18:07:07] but finding one cause will probably lead us to the others [18:07:11] hm [18:07:20] page_touched on enwiki vs page_touched of the item [18:07:41] diff and render don't reproduce the isseu for me [18:08:08] my theory is that in updating sitelinks [18:08:12] we delete tehm all [18:08:15] and readd them all [18:08:25] and then site link lookup in wikipedia uses slave [18:08:48] but then we use caching site link lookup [18:09:02] * aude not sure but related to that somehow [18:09:19] hm... SiteLinkTable does diffs and only saves what's needed [18:09:21] i think lua gets parsed first [18:09:22] well, it should [18:09:32] hoo: it should [18:09:38] it's not done in one transaction [18:10:42] no, it looks like we do diff [18:10:43] https://github.com/wikimedia/mediawiki-extensions-Wikibase/blob/master/lib/includes/store/sql/SiteLinkTable.php#L96-L126 [18:11:08] yeah [18:11:43] but all the lookups use DB_SLAVE [18:11:46] e.g. https://github.com/wikimedia/mediawiki-extensions-Wikibase/blob/master/lib/includes/store/sql/SiteLinkTable.php#L241 [18:12:27] anyway, lua gets parsed first and does site link lookup (and finds the item) [18:12:32] then it gets cached apparently [18:12:54] then the parseroutput hooks run (for the page_prop, lang links, other projects, etc) [18:13:36] and strange that the site link lookup would then be different if all of these steps are done [18:14:07] Indeed [18:31:46] hoo: found another page https://en.wikipedia.org/wiki/Lac_Ste._Anne_County [18:31:55] (please don't purge it) [18:33:37] they have the same page_touched timestamp [18:37:33] https://gist.github.com/filbertkm/e2b69df9be11f97b538e (and found one with the same timestamp but has links) [18:39:08] page_links_updated is older than page_touched for the ones that have links [18:40:33] (but don't think that matters) [18:42:39] :/ [18:42:45] * aude wonders how these 4 pages were touched at the smae time [18:42:52] and are different [18:43:00] maybe a common template? [18:43:03] yeah [18:43:19] It's time to make diner... I'll probably look further after that [18:43:37] oh, template infobox settlement was edited [18:43:44] https://en.wikipedia.org/w/index.php?title=Template:Infobox_settlement&curid=1406921&action=history [18:43:49] clearly explains that [18:44:04] aha [18:44:30] that touches quite a lot of pages [18:44:33] Indeed [18:44:42] maybe refreshlinks jobs or so going weir [18:44:43] d [18:44:47] with multiple pages [18:44:56] Keep me updated :) [18:45:01] yeah [18:45:05] (while away) [18:45:05] * aude has to go shovel again [18:46:33] https://gist.github.com/filbertkm/3f850865afbd2777a2cf shows one of the pages is missing entity usage [18:48:45] and is missing entry in page_props [18:55:28] * aude tired of nobody uploading photos to commons [18:59:37] I do occassionally :) [19:00:10] aude: I did! About 2 million ;-) [19:01:33] https://www.wikidata.org/w/index.php?title=Q22222608&type=revision&diff=295011519&oldid=294901469 [19:01:40] multichill: not many of the snowstorm [19:01:44] maybe everyone has no power [19:01:50] (except me) [19:02:11] kirill's photo is nice too though [19:02:24] Yeah, https://commons.wikimedia.org/wiki/Category:January_2016_United_States_winter_storm looks a bit empty [19:02:46] LOL, the power system in the US always seems to break [19:03:34] I hope people do have heating.... [19:04:00] we wouldn't if we lose power, but would manage [19:04:06] e.g. fireplace [19:04:15] anyway, time to shovel again [19:04:24] What's the point again? [19:04:55] multichill: so we don't have to shovel everything at once :) [19:05:09] and also at least need to shovel around the door so we can open it and go outside [19:05:29] * aude back later hopefully :) [19:05:33] Haha, free gym and it's kind of fun to do [19:05:39] yeah [19:05:46] this time i will measure how much we have [19:05:53] later :) [19:06:00] It's fun to make a central pile with your neighbours [19:06:05] And see how big you can get it ;-) [19:55:42] aude: Any new findings? [22:37:19] [19:06:18] and doubt someone did action=render or diffs of all the pages (quite many, but not in the same category and whose items were edited) /// sorry for the late reply, but a gadget maybe ? [22:38:20] if it's recent, *maybe* magnus' gadget to drap and drop refs from wikipedia ? [22:39:06] (heavy on the "maybe", but a gadget would make sense :) ) [22:46:49] Alphos: these are generally obscure items and pages [22:47:06] and incidents of the bug have been reported for a while :/ [22:47:22] while it does preclude magnus' gadget, it doesn't preclude all gadgets [22:47:36] the "obscurity" of items and pages doesn't preclude gadgets either [22:47:58] sure [22:48:47] * aude wonders what property to use to list snowfall amounts in various places? [22:49:01] action=render is really quite specific, and not really widely used ; aside from an external tool (not a gadget) that calls it (heck, maybe search engines), or through a local gadget, it's really unlikely to be called [22:49:30] aude: https://twitter.com/MariahCBlake/status/690923715048026113 :D [22:49:41] Alphos: definitely some tool might use action=render [22:49:43] that's just my internals-noob take on it, of course :) [22:49:53] Alphos: any ideas are good :) [22:50:02] cheers ^^ [22:50:03] hoo: lol :D [23:00:25] hoo: While you were away another read-only spike :-( [23:01:16] :( [23:01:27] When exactly? [23:01:57] I didn't record the time stamp this time [23:02:24] * aude would like to specify time values with precision of minute + timezone [23:02:29] I really wish we had logging for that in place [23:03:04] aude: hm... do we except that via the API? [23:07:26] * accept [23:22:25] hoo: not yet [23:37:40] aude so the only time offset we can use for now is Z/Zulu/UTC ? [23:38:38] We probably still filter everything else during data validation [23:39:05] given not all offsets have a single-letter abbreviation, why not also allow semi-arbitrary offsets expressed as literals ? [23:39:53] I think hte current internal representation is supposed to allow offsets as hours:minutes [23:40:52] pity for https://en.wikipedia.org/wiki/Calcutta_Time ;p [23:41:21] (+05:30:21, plus/minus error) [23:42:23] not in effect anymore, but people were indeed born or died in calcutta while it was in effect ;) [23:43:59] Alphos: Yay, more lovely edge cases [23:44:11] oh please, i eat those for breakfast ! [23:45:22] still with indian 19th/20th-century edge cases, https://en.wikipedia.org/wiki/Port_Blair_mean_time roughly +06:19:51 [23:46:59] there are other smaller edge cases with non-dizenal non-sesquidizenal offsets, but all the ones i know of can be expressed in minutes, or hours:minutes [23:47:51] Luckily, I don't need to wrap my head around taht now [23:47:58] ^^ [23:48:06] have some breakfast :D [23:48:15] I should rather go to bed :P [23:48:23] me too... [23:51:48] aude: Think I'll give up on the langlinks bug for today :( [23:52:07] I have not been able to get my installation do to the same thing again [23:52:32] Even after looking at my access logs and doing the exact same things [23:52:58] ooh, which langlinks bug ? working on a bit of one myself ! [23:53:11] ("ooh" was an expression of surprise here :D ) [23:53:22] T47839 [23:53:25] the one with action=render ? [23:53:37] Maybe [23:53:42] ok [23:53:48] not working on the same then ^^ [23:53:54] the only time I was able verify it locally, it happend after action=render