[00:02:49] <wikibugs>	 10Growth-Team, 10Thanks, 10DiscussionTools, 10OWC2020, 10Editing-team (Tracking): Add "Thanks" to talk page comments - https://phabricator.wikimedia.org/T249893 (10Sophivorus)
[00:03:07] <wikibugs>	 10Growth-Team, 10Thanks, 10DiscussionTools, 10OWC2020, 10Editing-team (Tracking): Add "Thanks" to talk page comments - https://phabricator.wikimedia.org/T249893 (10Sophivorus)
[00:05:53] <wikibugs>	 10Growth-Team, 10Thanks, 10DiscussionTools, 10OWC2020, 10Editing-team (Tracking): Add "Thanks" to talk page comments - https://phabricator.wikimedia.org/T249893 (10Sophivorus) Many replies consist of simple "Thank you" messages. These messages clutter talk pages and increase the nesting depth of discussi...
[00:07:17] <wikibugs>	 10Growth-Team, 10Thanks, 10DiscussionTools, 10OWC2020, 10Editing-team (Tracking): Add "Thanks" to talk page comments - https://phabricator.wikimedia.org/T249893 (10Sophivorus)
[00:33:10] <wikibugs>	 10Growth-Team, 10GrowthExperiments: [WelcomeSurvey] Links on the "Getting started" module are in English, hence creating redlinks - https://phabricator.wikimedia.org/T270250 (10Tgr) 05Open→03Declined Per Slack discussion, this doesn't really need fixing - new users are extremely unlikely to see Special:Wel...
[02:18:28] <wikibugs>	 (03PS1) 10Jforrester: Use User->isRegistered(), not deprecated isLoggedIn() [extensions/WikiLove] - 10https://gerrit.wikimedia.org/r/650337 (https://phabricator.wikimedia.org/T270450)
[02:24:16] <wikibugs>	 (03PS1) 10Jforrester: Use User->isRegistered(), not deprecated isLoggedIn() [extensions/Thanks] - 10https://gerrit.wikimedia.org/r/650341 (https://phabricator.wikimedia.org/T270450)
[02:26:25] <wikibugs>	 (03CR) 10DannyS712: [C: 03+2] Use User->isRegistered(), not deprecated isLoggedIn() [extensions/WikiLove] - 10https://gerrit.wikimedia.org/r/650337 (https://phabricator.wikimedia.org/T270450) (owner: 10Jforrester)
[02:32:45] <wikibugs>	 (03PS1) 10Jforrester: Use User->isRegistered(), not deprecated isLoggedIn() [extensions/Echo] - 10https://gerrit.wikimedia.org/r/650371 (https://phabricator.wikimedia.org/T270450)
[02:44:36] <wikibugs>	 (03PS1) 10Jforrester: Use User->isRegistered(), not deprecated isLoggedIn() [extensions/PageTriage] - 10https://gerrit.wikimedia.org/r/650381 (https://phabricator.wikimedia.org/T270450)
[02:46:33] <wikibugs>	 (03Merged) 10jenkins-bot: Use User->isRegistered(), not deprecated isLoggedIn() [extensions/WikiLove] - 10https://gerrit.wikimedia.org/r/650337 (https://phabricator.wikimedia.org/T270450) (owner: 10Jforrester)
[02:49:00] <wikibugs>	 (03PS1) 10Jforrester: Use User->isRegistered(), not deprecated isLoggedIn() [extensions/Flow] - 10https://gerrit.wikimedia.org/r/650385 (https://phabricator.wikimedia.org/T270450)
[02:52:41] <wikibugs>	 (03CR) 10DannyS712: [C: 03+2] Use User->isRegistered(), not deprecated isLoggedIn() [extensions/PageTriage] - 10https://gerrit.wikimedia.org/r/650381 (https://phabricator.wikimedia.org/T270450) (owner: 10Jforrester)
[02:56:29] <wikibugs>	 (03CR) 10DannyS712: [C: 03+2] Use User->isRegistered(), not deprecated isLoggedIn() [extensions/Thanks] - 10https://gerrit.wikimedia.org/r/650341 (https://phabricator.wikimedia.org/T270450) (owner: 10Jforrester)
[02:59:45] <wikibugs>	 (03PS1) 10Jforrester: Use User->isRegistered(), not deprecated isLoggedIn() [extensions/GrowthExperiments] - 10https://gerrit.wikimedia.org/r/650391 (https://phabricator.wikimedia.org/T270450)
[03:03:23] <wikibugs>	 (03CR) 10DannyS712: [C: 03+2] Use User->isRegistered(), not deprecated isLoggedIn() [extensions/Echo] - 10https://gerrit.wikimedia.org/r/650371 (https://phabricator.wikimedia.org/T270450) (owner: 10Jforrester)
[03:07:40] <wikibugs>	 (03Merged) 10jenkins-bot: Use User->isRegistered(), not deprecated isLoggedIn() [extensions/PageTriage] - 10https://gerrit.wikimedia.org/r/650381 (https://phabricator.wikimedia.org/T270450) (owner: 10Jforrester)
[03:07:46] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Use User->isRegistered(), not deprecated isLoggedIn() [extensions/GrowthExperiments] - 10https://gerrit.wikimedia.org/r/650391 (https://phabricator.wikimedia.org/T270450) (owner: 10Jforrester)
[03:12:44] <wikibugs>	 (03CR) 10Jforrester: "recheck" [extensions/GrowthExperiments] - 10https://gerrit.wikimedia.org/r/650391 (https://phabricator.wikimedia.org/T270450) (owner: 10Jforrester)
[03:18:05] <wikibugs>	 (03CR) 10DannyS712: [C: 03+2] Use User->isRegistered(), not deprecated isLoggedIn() [extensions/Flow] - 10https://gerrit.wikimedia.org/r/650385 (https://phabricator.wikimedia.org/T270450) (owner: 10Jforrester)
[03:21:40] <wikibugs>	 (03Merged) 10jenkins-bot: Use User->isRegistered(), not deprecated isLoggedIn() [extensions/Thanks] - 10https://gerrit.wikimedia.org/r/650341 (https://phabricator.wikimedia.org/T270450) (owner: 10Jforrester)
[03:32:50] <wikibugs>	 (03Merged) 10jenkins-bot: Use User->isRegistered(), not deprecated isLoggedIn() [extensions/Echo] - 10https://gerrit.wikimedia.org/r/650371 (https://phabricator.wikimedia.org/T270450) (owner: 10Jforrester)
[03:34:27] <wikibugs>	 (03CR) 10DannyS712: [C: 03+2] Use User->isRegistered(), not deprecated isLoggedIn() [extensions/GrowthExperiments] - 10https://gerrit.wikimedia.org/r/650391 (https://phabricator.wikimedia.org/T270450) (owner: 10Jforrester)
[03:47:12] <wikibugs>	 (03Merged) 10jenkins-bot: Use User->isRegistered(), not deprecated isLoggedIn() [extensions/Flow] - 10https://gerrit.wikimedia.org/r/650385 (https://phabricator.wikimedia.org/T270450) (owner: 10Jforrester)
[03:49:08] <wikibugs>	 (03Merged) 10jenkins-bot: Use User->isRegistered(), not deprecated isLoggedIn() [extensions/GrowthExperiments] - 10https://gerrit.wikimedia.org/r/650391 (https://phabricator.wikimedia.org/T270450) (owner: 10Jforrester)
[03:58:18] <wikibugs>	 (03CR) 10Gergő Tisza: Make Task, TaskType and Topic JsonUnserializable (031 comment) [extensions/GrowthExperiments] - 10https://gerrit.wikimedia.org/r/645221 (https://phabricator.wikimedia.org/T269366) (owner: 10Gergő Tisza)
[11:50:36] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10Expiring-Watchlist-Items: ClearUserWatchlistJob bad database peformance on enwiki, commons, causing database lag? - https://phabricator.wikimedia.org/T270481 (10jcrespo)
[12:11:21] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10Expiring-Watchlist-Items: ClearUserWatchlistJob bad database peformance on enwiki, commons, causing database lag? - https://phabricator.wikimedia.org/T270481 (10jcrespo) Timestamps correlate well with deletions (not selects) on logstash errors:  htt...
[12:11:28] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10Expiring-Watchlist-Items: ClearUserWatchlistJob bad database peformance on enwiki, commons, causing database lag? - https://phabricator.wikimedia.org/T270481 (10jcrespo) This is enwiki lag- up to 1 second is ok (note the log scale), at 11:20 goes ov...
[12:19:31] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10Expiring-Watchlist-Items, 10Wikimedia-production-error: ClearUserWatchlistJob bad database peformance on enwiki, commons, causing database lag? - https://phabricator.wikimedia.org/T270481 (10jcrespo)
[12:21:47] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10Expiring-Watchlist-Items, 10Wikimedia-production-error: ClearUserWatchlistJob bad database peformance on enwiki, commons, causing database lag? - https://phabricator.wikimedia.org/T270481 (10jcrespo)
[13:10:27] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10Expiring-Watchlist-Items, 10Wikimedia-production-error: ClearUserWatchlistJob bad database peformance on enwiki, commons, causing database lag? - https://phabricator.wikimedia.org/T270481 (10Marostegui) p:05Triage→03High @ifried this needs to...
[13:16:36] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10Expiring-Watchlist-Items, 10Wikimedia-production-error: ClearUserWatchlistJob bad database peformance on enwiki, commons, causing database lag? - https://phabricator.wikimedia.org/T270481 (10Marostegui)
[13:17:52] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10Expiring-Watchlist-Items, 10Wikimedia-production-error: ClearUserWatchlistJob bad database peformance on enwiki, commons, causing database lag? - https://phabricator.wikimedia.org/T270481 (10Marostegui)
[13:18:02] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10Expiring-Watchlist-Items, 10Wikimedia-production-error: ClearUserWatchlistJob bad database peformance on enwiki, commons, causing database lag? - https://phabricator.wikimedia.org/T270481 (10Marostegui)
[13:18:17] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10Expiring-Watchlist-Items, 10Wikimedia-production-error: ClearUserWatchlistJob bad database peformance on enwiki, commons, causing database lag? - https://phabricator.wikimedia.org/T270481 (10Marostegui)
[13:18:48] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10DBA, and 2 others: ClearUserWatchlistJob bad database peformance on enwiki, commons, causing database lag? - https://phabricator.wikimedia.org/T270481 (10Marostegui)
[13:30:45] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10DBA, and 2 others: ClearUserWatchlistJob bad database peformance on enwiki, commons, causing database lag? - https://phabricator.wikimedia.org/T270481 (10Marostegui) The master had a huge spike on deletes at 11:20, which matches the above graphs {F3...
[13:46:47] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10DBA, and 2 others: ClearUserWatchlistJob bad database peformance on enwiki, commons, causing database lag? - https://phabricator.wikimedia.org/T270481 (10kostajh) > @Joe told me those were (possibly) jobs generated by newly deployed feature "watchli...
[13:49:40] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10DBA, and 2 others: ClearUserWatchlistJob bad database peformance on enwiki, commons, causing database lag? - https://phabricator.wikimedia.org/T270481 (10kostajh) cc @Samwilson @MusikAnimal
[14:30:40] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10DBA, and 2 others: ClearUserWatchlistJob bad database peformance on enwiki, commons, causing database lag? - https://phabricator.wikimedia.org/T270481 (10jcrespo) To summarize all my findings above, there is 3 "bad" things happening right now:  * Lo...
[14:31:25] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10DBA, and 2 others: ClearUserWatchlistJob/WatchedItemStore::removeWatchBatchForUser bad database peformance on enwiki and others, causing database lag - https://phabricator.wikimedia.org/T270481 (10jcrespo)
[14:33:06] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10DBA, and 2 others: ClearUserWatchlistJob/WatchedItemStore::removeWatchBatchForUser bad database peformance on enwiki and others, causing database lag - https://phabricator.wikimedia.org/T270481 (10Marostegui) >>! In T270481#6701497, @jcrespo wrote:...
[17:35:36] <wikibugs>	 (03CR) 10Urbanecm: [C: 03+2] "works for me locally => merged" [extensions/GrowthExperiments] - 10https://gerrit.wikimedia.org/r/649948 (https://phabricator.wikimedia.org/T270297) (owner: 10Gergő Tisza)
[17:38:17] <wikibugs>	 (03CR) 10Urbanecm: [C: 03+1] "sounds sane" [extensions/GrowthExperiments] - 10https://gerrit.wikimedia.org/r/649285 (owner: 10Gergő Tisza)
[17:43:35] <wikibugs>	 (03PS1) 10Urbanecm: Fix Wikilove for smaller screens [extensions/WikiLove] - 10https://gerrit.wikimedia.org/r/650539 (https://phabricator.wikimedia.org/T268534)
[17:48:58] <wikibugs>	 (03Merged) 10jenkins-bot: Fix message fallback for homepage banner [extensions/GrowthExperiments] - 10https://gerrit.wikimedia.org/r/649948 (https://phabricator.wikimedia.org/T270297) (owner: 10Gergő Tisza)
[17:54:03] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10DBA, and 2 others: ClearUserWatchlistJob/WatchedItemStore::removeWatchBatchForUser bad database peformance on enwiki and others, causing database lag - https://phabricator.wikimedia.org/T270481 (10aezell) The engineers on CommTech are looking into t...
[18:45:01] <wikibugs>	 10Growth-Team, 10Image-Recommendations, 10User-Urbanecm_WMF: Ambassador discussions on "add an image" - https://phabricator.wikimedia.org/T270222 (10Urbanecm_WMF) I just marked https://www.mediawiki.org/wiki/Growth/Personalized_first_day/Structured_tasks/Add_an_image for translation, so it can be translated...
[19:02:51] <wikibugs>	 (03PS1) 10Kosta Harlan: Remove variant A code [extensions/GrowthExperiments] - 10https://gerrit.wikimedia.org/r/650558 (https://phabricator.wikimedia.org/T238887)
[19:03:16] <wikibugs>	 (03CR) 10Kosta Harlan: "Warning! Untested code" [extensions/GrowthExperiments] - 10https://gerrit.wikimedia.org/r/650558 (https://phabricator.wikimedia.org/T238887) (owner: 10Kosta Harlan)
[19:10:15] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10DBA, and 2 others: ClearUserWatchlistJob/WatchedItemStore::removeWatchBatchForUser bad database peformance on enwiki and others, causing database lag - https://phabricator.wikimedia.org/T270481 (10MusikAnimal) `removeWatchBatchForUser()` is only cal...
[19:25:28] <wikibugs>	 (03CR) 10Gergő Tisza: "Nice! I think the Start module and its submodules (other than StartEditing) can also be removed." (034 comments) [extensions/GrowthExperiments] - 10https://gerrit.wikimedia.org/r/650558 (https://phabricator.wikimedia.org/T238887) (owner: 10Kosta Harlan)
[19:26:46] <wikibugs>	 (03CR) 10Kosta Harlan: Remove variant A code (032 comments) [extensions/GrowthExperiments] - 10https://gerrit.wikimedia.org/r/650558 (https://phabricator.wikimedia.org/T238887) (owner: 10Kosta Harlan)
[19:34:01] <wikibugs>	 (03CR) 10Bartosz Dziewoński: [C: 03+2] Move Vector-specific top:-5px badge offset to Vector style file [extensions/Echo] - 10https://gerrit.wikimedia.org/r/612954 (owner: 10Isarra)
[19:35:46] <wikibugs>	 (03CR) 10DannyS712: [C: 04-1] Remove variant A code (034 comments) [extensions/GrowthExperiments] - 10https://gerrit.wikimedia.org/r/650558 (https://phabricator.wikimedia.org/T238887) (owner: 10Kosta Harlan)
[19:43:26] <wikibugs>	 (03CR) 10Kosta Harlan: Remove variant A code (031 comment) [extensions/GrowthExperiments] - 10https://gerrit.wikimedia.org/r/650558 (https://phabricator.wikimedia.org/T238887) (owner: 10Kosta Harlan)
[20:01:37] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10DBA, and 2 others: ClearUserWatchlistJob/WatchedItemStore::removeWatchBatchForUser bad database peformance on enwiki and others, causing database lag - https://phabricator.wikimedia.org/T270481 (10Marostegui) >>! In T270481#6702282, @MusikAnimal wro...
[20:16:47] <wikibugs>	 10Growth-Team, 10GrowthExperiments-NewcomerTasks: FAQ before asking questions - https://phabricator.wikimedia.org/T270523 (10Ladsgroup)
[20:26:59] <wikibugs>	 (03Merged) 10jenkins-bot: Move Vector-specific top:-5px badge offset to Vector style file [extensions/Echo] - 10https://gerrit.wikimedia.org/r/612954 (owner: 10Isarra)
[21:06:47] <wikibugs>	 10Growth-Team, 10MediaWiki-Watchlist, 10Community-Tech, 10DBA, and 2 others: ClearUserWatchlistJob/WatchedItemStore::removeWatchBatchForUser bad database peformance on enwiki and others, causing database lag - https://phabricator.wikimedia.org/T270481 (10MusikAnimal) > Is there a way to throttle those dele...