[01:59:38] 06Data-Engineering, 10MediaWiki-File-management, 07Schema-change: Future of filearchive table - https://phabricator.wikimedia.org/T425897 (10Bugreporter) 03NEW [02:00:16] 06Data-Engineering, 10MediaWiki-File-management, 07Schema-change: Future of filearchive table - https://phabricator.wikimedia.org/T425897#11906184 (10Bugreporter) [02:00:26] 06Data-Engineering, 06Data-Engineering-Radar, 06Commons, 06Data-Persistence, and 5 others: Migrate file tables to a modern layout (image/oldimage; file/filerevision; add primary keys) - https://phabricator.wikimedia.org/T28741#11906185 (10Bugreporter) [02:33:26] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform, 13Patch-For-Review: mediawiki.page_change.v1 event - Add revision revert details - https://phabricator.wikimedia.org/T423583#11906209 (10Ottomata) [02:33:48] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform, 13Patch-For-Review: mediawiki.page_change.v1 event - Add revision revert details - https://phabricator.wikimedia.org/T423583#11906210 (10Ottomata) [07:30:00] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 13Patch-For-Review: Support for Java 25 and Flink 2 - https://phabricator.wikimedia.org/T412978#11906412 (10brouberol) [07:37:36] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE (2026-04-24 - 2026-05-15), 13Patch-For-Review: Support for Java 25 and Flink 2 - https://phabricator.wikimedia.org/T412978#11906423 (10brouberol) 05Open→03In progress [07:37:37] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE (2026-04-24 - 2026-05-15), 13Patch-For-Review: Support for Java 25 and Flink 2 - https://phabricator.wikimedia.org/T412978#11906422 (10brouberol) [07:37:57] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental: mediawiki_history_incremental_v1: schema specification for stakeholder review - https://phabricator.wikimedia.org/T425573#11906425 (10Pablo) Similar to retrieving revisions from the same page across multiple languages, having the global... [09:25:56] (03PS3) 10A-pizzata: change create table for mediawiki_content to become private [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1285337 (https://phabricator.wikimedia.org/T424355) [09:42:44] (03CR) 10Gkyziridis: [C:03+1] expand_event_sanitized_analytics_allowlist: Add revertrisk-multilingual predictions to allowlist. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1283749 (https://phabricator.wikimedia.org/T415892) (owner: 10Gkyziridis) [09:50:24] (03CR) 10Gkyziridis: [C:03+1] "Please deploy whenever is possible." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1283749 (https://phabricator.wikimedia.org/T415892) (owner: 10Gkyziridis) [09:50:44] 06Data-Engineering, 06Data-Engineering-Radar, 10Event-Platform, 06Machine-Learning-Team (Q4 FY2025-26), 13Patch-For-Review: Add Multilingual RevertRisk predictions to mediawiki.page_revert_risk_prediction_change - https://phabricator.wikimedia.org/T415892#11906837 (10gkyziridis) I `+1` the patch event-sa... [10:08:44] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Engineering-Radar, 06ServiceOps new, 10ServiceOps-Services-Oids, and 3 others: Make eventstreams-internal available to WMF staff without an ssh tunnel - https://phabricator.wikimedia.org/T348763#11906877 (10atsuko) Result output is https://phabr... [10:20:01] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental, 10Event-Platform: Incremental MWH - MediaWiki event data source improvements - https://phabricator.wikimedia.org/T423935#11906900 (10Ottomata) [10:20:13] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental: mediawiki_history_incremental_v1: schema specification for stakeholder review - https://phabricator.wikimedia.org/T425573#11906901 (10Ottomata) > global user ID event_user_central_id should be easy to get as a point-in-time field from pa... [10:25:05] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE (2026-04-24 - 2026-05-15), 13Patch-For-Review: Support for Java 25 and Flink 2 - https://phabricator.wikimedia.org/T412978#11906925 (10Ottomata) > Change #1285540 merged by Brouberol [...] flink: build the flink 2.0.0 image 😍 [12:09:21] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental: mediawiki_history_incremental_v1: schema specification for stakeholder review - https://phabricator.wikimedia.org/T425573#11907306 (10Pablo) Thank you, @Ottomata! [12:10:07] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-Mediawiki-Content: Missing/inconsistent page_redirect_target field for redirects in Mediawiki content current v1 dumps - https://phabricator.wikimedia.org/T400632#11907307 (10MGerlach) >>! In T400632#11855621, @xcollazo wrote: >>>! In T400632#1169716... [12:41:46] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Backfill datasets affected by Nov 2025 automated traffic incident - https://phabricator.wikimedia.org/T421735#11907444 (10mforns) [12:44:08] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental, 10Event-Platform, 13Patch-For-Review: Create mediawiki.user_change event stream - https://phabricator.wikimedia.org/T423952#11907453 (10PatchDemoBot) Test wiki **created** on [[ https://patchdemo.wmcloud.org | Patch demo ]] by Ottoma... [12:48:08] (03CR) 10Ottomata: [C:03+2] expand_event_sanitized_analytics_allowlist: Add revertrisk-multilingual predictions to allowlist. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1283749 (https://phabricator.wikimedia.org/T415892) (owner: 10Gkyziridis) [12:48:21] (03CR) 10Ottomata: [V:03+2 C:03+2] expand_event_sanitized_analytics_allowlist: Add revertrisk-multilingual predictions to allowlist. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1283749 (https://phabricator.wikimedia.org/T415892) (owner: 10Gkyziridis) [12:50:25] 06Data-Engineering, 06Data-Engineering-Radar, 10Event-Platform, 06Machine-Learning-Team (Q4 FY2025-26), 13Patch-For-Review: Add Multilingual RevertRisk predictions to mediawiki.page_revert_risk_prediction_change - https://phabricator.wikimedia.org/T415892#11907475 (10Ottomata) Merged and added to https:/... [13:14:23] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-Mediawiki-Content: Missing/inconsistent page_redirect_target field for redirects in Mediawiki content current v1 dumps - https://phabricator.wikimedia.org/T400632#11907558 (10APizzata-WMF) > However, I find 1,308,751 pages in English Wikipedia main n... [13:16:42] 06Data-Engineering, 06Data-Platform-SRE (2026-04-24 - 2026-05-15), 07Epic, 13Patch-For-Review: Upgrade Spark to a version with long term Iceberg support - https://phabricator.wikimedia.org/T338057#11907563 (10xcollazo) [13:26:52] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental: mediawiki_history_incremental_v1: schema specification for stakeholder review - https://phabricator.wikimedia.org/T425573#11907613 (10Milimetric) @Ottomata and @xcollazo: love most of this, I'm not sure the revert 48h window stuff works.... [14:04:57] (03CR) 10Xcollazo: change create table for mediawiki_content to become private (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1285337 (https://phabricator.wikimedia.org/T424355) (owner: 10A-pizzata) [14:05:31] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10CheckUser-SuggestedInvestigations, 10MWH-Incremental, 10Event-Platform, 06Product Safety and Integrity (Sprint lily-of-the-valley (May 4 - May 22)): CheckUser extension should use EventBus.UserEnt... - https://phabricator.wikimedia.org/T425801#11907840 [14:05:35] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10CheckUser-SuggestedInvestigations, 10MWH-Incremental, 10Event-Platform, 06Product Safety and Integrity (Sprint lily-of-the-valley (May 4 - May 22)): CheckUser extension should use EventBus.UserEnt... - https://phabricator.wikimedia.org/T425801#11907841 [14:09:32] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental, 13Patch-For-Review: Accelerate sqoop landing for MediaWiki History private tables - https://phabricator.wikimedia.org/T424355#11907867 (10xcollazo) >With these results the maximum waiting time would be 14 h if we decide to move the con... [14:15:46] 06Data-Engineering, 06Data-Engineering-Radar, 10Event-Platform, 06Machine-Learning-Team (Q4 FY2025-26), 13Patch-For-Review: Add Multilingual RevertRisk predictions to mediawiki.page_revert_risk_prediction_change - https://phabricator.wikimedia.org/T415892#11907925 (10gkyziridis) Thank you very much @Otto... [14:17:37] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Engineering-Radar, 06ServiceOps new, 10ServiceOps-Services-Oids, and 3 others: Make eventstreams-internal available to WMF staff without an ssh tunnel - https://phabricator.wikimedia.org/T348763#11907943 (10atsuko) Going to push updates for turn... [14:44:28] 06Data-Engineering: Set up a rolling retention window for event_sanitized.netflow - https://phabricator.wikimedia.org/T424904#11908177 (10xcollazo) [14:51:03] 06Data-Engineering: Set up a rolling retention window for event_sanitized.netflow - https://phabricator.wikimedia.org/T424904#11908222 (10xcollazo) >>! In T424904#11885395, @ayounsi wrote: > It would be best to keep all historical data but aggregate it further as time goes. Do we already drop/aggregate all the f... [14:52:36] 06Data-Engineering: Set up a rolling retention window for event_sanitized.netflow - https://phabricator.wikimedia.org/T424904#11908234 (10xcollazo) For context, we need this {T311739} [14:56:33] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Engineering-Radar, 06ServiceOps new, 10ServiceOps-Services-Oids, and 2 others: Make eventstreams-internal available to WMF staff without an ssh tunnel - https://phabricator.wikimedia.org/T348763#11908267 (10atsuko) Plan # update openstream cha... [14:58:38] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 07Epic: Attribution Research: Instrument Donation Attempts - https://phabricator.wikimedia.org/T419569#11908279 (10tchin) [15:01:00] 06Data-Engineering, 05FY2025-26 KR 5.1, 06MediaWiki-Platform-Team (Kanban Board), 07OKR-Work: redioscope: periodically publish top clients to the data lake - https://phabricator.wikimedia.org/T424823#11908298 (10JTweed-WMF) [15:08:24] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Backfill datasets affected by Nov 2025 automated traffic incident - https://phabricator.wikimedia.org/T421735#11908359 (10mforns) [15:11:02] !log Test Kitchen mw-user experiment (poll 225789) - adds: none; removes: none; fields: ab-test-email-confirmation-banner - xLab/MPIC/TK tips at https://w.wiki/FwuD [15:11:03] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [15:12:22] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MediaWiki-Core-Revision-backend, 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, and 4 others: MediaWiki\Revision\RevisionAccessException: Unable to load fresh row for rev_id: {rev_id} - https://phabricator.wikimedia.org/T400380#11908367 (10Ottomata)... [15:14:40] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06MW-Interfaces-Team, 10Event-Platform: mediawiki.page_change.v1 event stream - Investigate mismatched meta.dt and dt (and rev_dt) fields - https://phabricator.wikimedia.org/T409105#11908377 (10Ottomata) Possibly related? {T425735} [15:22:25] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 07Epic: Attribution Research: Instrument Donation Attempts - https://phabricator.wikimedia.org/T419569#11908426 (10tchin) Took a look at this, here's what I found: - Main menu sidebar: Has a class `n-sitesupport` on the `
  • `. This is default to wikis wit... [15:58:40] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental, 10Event-Platform: Emit comprehensive mediawiki user block change information in an event stream - https://phabricator.wikimedia.org/T424685#11908745 (10Ottomata) a:05Ottomata→03None [16:05:35] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental, 10Event-Platform: Incremental MWH - MediaWiki event data source improvements - https://phabricator.wikimedia.org/T423935#11908809 (10Ottomata) [16:22:07] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental, 10Event-Platform: Incremental MWH - MediaWiki event data source improvements - https://phabricator.wikimedia.org/T423935#11908925 (10Ottomata) [16:23:20] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental, 10Event-Platform: mediawiki.page_change event - add access_method (AKA platform) field - https://phabricator.wikimedia.org/T424887#11908929 (10Ottomata) [16:23:38] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental, 10Event-Platform: mediawiki.page_change event - add access_method (AKA platform) field - https://phabricator.wikimedia.org/T424887#11908937 (10Ottomata) [16:26:31] 06Data-Engineering, 06Data-Engineering-Radar, 10MediaWiki-extensions-EventLogging, 07Essential-Work, and 3 others: Migrate "WikiLambda API" instrument to use the Test Kitchen SDK - https://phabricator.wikimedia.org/T415254#11908962 (10Sfaci) Patches above represent some additional work I didn't realize bef... [16:27:49] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to analytics_privatedata_users and SQL Lab for AnnieKim_WMDE - https://phabricator.wikimedia.org/T420500#11908967 (10Dzahn) a:03AnnieKim_WMDE [16:45:06] 06Data-Engineering: implement script to move data from P&T data lake to FR Tech data lake - https://phabricator.wikimedia.org/T425133#11909079 (10AStein-WMF) This looks good to me! thanks for writing it up! [16:47:34] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental: mediawiki_history_incremental_v1: schema specification for stakeholder review - https://phabricator.wikimedia.org/T425573#11909108 (10Ottomata) Here are my comments! Let's discuss, I don't think I am strongly opinionated (am I? Let's fi... [16:49:26] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental: mediawiki_history_incremental_v1: schema specification for stakeholder review - https://phabricator.wikimedia.org/T425573#11909118 (10Ottomata) Question for MWH users: Should we consider renaming the various `_historical` fields? To me... [17:33:25] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Backfill datasets affected by Nov 2025 automated traffic incident - https://phabricator.wikimedia.org/T421735#11909395 (10mforns) [17:41:55] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental: Implement MERGE INTO writers for mediawiki_history_incremental_v1 - https://phabricator.wikimedia.org/T425729#11909429 (10xcollazo) 05Open→03In progress p:05Triage→03High [17:58:21] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Backfill datasets affected by Nov 2025 automated traffic incident - https://phabricator.wikimedia.org/T421735#11909526 (10mforns) [18:39:26] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Add new API rate limiting fields from webrequest_logs to Turnilo view - https://phabricator.wikimedia.org/T419736#11909682 (10amastilovic) Turnilo has been updated this morning, so I believe that completes this task. [18:40:10] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Backfill datasets affected by Nov 2025 automated traffic incident - https://phabricator.wikimedia.org/T421735#11909685 (10mforns) [18:41:51] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental, 10Event-Platform: mediawiki.page_change event - add access_method & platform details - https://phabricator.wikimedia.org/T424887#11909693 (10Ottomata) [18:41:52] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental: Add log_id to mediawiki.page_change event schema and wmf.mediawiki_history - https://phabricator.wikimedia.org/T425986 (10xcollazo) 03NEW [18:47:03] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Backfill datasets affected by Nov 2025 automated traffic incident - https://phabricator.wikimedia.org/T421735#11909740 (10mforns) [18:53:38] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental: Add log_id to mediawiki.page_change event schema and wmf.mediawiki_history - https://phabricator.wikimedia.org/T425986#11909783 (10Ottomata) See also: https://www.mediawiki.org/wiki/Manual:Domain_events/Hierarchy#Include_LogEntry_objects... [18:54:08] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Data-Platform-SRE: Presto cluster improvements for concurrency and workload - https://phabricator.wikimedia.org/T424112#11909784 (10amastilovic) a:03amastilovic [19:05:04] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental, 10Event-Platform: mediawiki.page_change event - add access_method & platform details - https://phabricator.wikimedia.org/T424887#11909803 (10Ottomata) This task looks more difficult than I originally expected. [[ https://wikimedia.sla... [19:05:27] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental, 10Event-Platform: Incremental MWH - MediaWiki event data source improvements - https://phabricator.wikimedia.org/T423935#11909805 (10Ottomata) [19:11:00] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Backfill datasets affected by Nov 2025 automated traffic incident - https://phabricator.wikimedia.org/T421735#11909831 (10mforns) [19:11:47] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental, 10Event-Platform, 05MW-1.47-notes (1.47.0-wmf.2; 2026-05-12): Create mediawiki.user_change event stream - https://phabricator.wikimedia.org/T423952#11909833 (10xcollazo) A new consideration: {T425986}. For this new `user_change` stre... [19:15:21] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental: Implement MERGE INTO writers for mediawiki_history_incremental_v1 - https://phabricator.wikimedia.org/T425729#11909843 (10xcollazo) (This work is now blocked by {T425986}.) [19:29:47] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Requesting access to analytics_privatedata_users & Kerberos & SQL Lab for catherinekelsey - https://phabricator.wikimedia.org/T425565#11909884 (10Dzahn) [20:06:16] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental: Add log_id to mediawiki.page_change event schema and wmf.mediawiki_history - https://phabricator.wikimedia.org/T425986#11910002 (10Ottomata) > 1. Add log_id to mediawiki.page_change event schema @daniel this would be very simple if we c... [20:38:26] (03PS1) 10Xcollazo: Add event_log_id to mediawiki_history DDL [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1285903 (https://phabricator.wikimedia.org/T425986) [20:39:25] (03PS1) 10Xcollazo: Add event_log_id to wmf.mediawiki_history (T425986) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1285904 (https://phabricator.wikimedia.org/T425986) [20:45:22] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental, 13Patch-For-Review: Add log_id to mediawiki.page_change event schema and wmf.mediawiki_history - https://phabricator.wikimedia.org/T425986#11910124 (10xcollazo) [20:48:52] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental, 13Patch-For-Review: Add log_id to wmf.mediawiki_history - https://phabricator.wikimedia.org/T425986#11910143 (10xcollazo) 05Open→03In progress p:05Triage→03High [20:49:22] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental, 13Patch-For-Review: Add log_id to wmf.mediawiki_history - https://phabricator.wikimedia.org/T425986#11910148 (10xcollazo) [20:50:24] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10MWH-Incremental, 13Patch-For-Review: Add log_id to wmf.mediawiki_history - https://phabricator.wikimedia.org/T425986#11910151 (10xcollazo) >>! In T425986#11910000, @Ottomata wrote: >> 1. Add log_id to mediawiki.page_change event schema > > @daniel thi... [21:20:43] !log Test Kitchen edge-unique experiments (poll 226882) - adds: none; removes: none; fields: logged-out-retention-round8 - xLab/MPIC/TK tips at https://w.wiki/FwuD [21:20:44] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [22:23:20] 06Data-Engineering, 06Data-Engineering-Radar, 10MediaWiki-extensions-EventLogging, 07Essential-Work, and 3 others: Migrate "WikiLambda API" instrument to use the Test Kitchen SDK - https://phabricator.wikimedia.org/T415254#11910404 (10Sfaci) [22:26:03] 06Data-Engineering, 06Data-Engineering-Radar, 10MediaWiki-extensions-EventLogging, 07Essential-Work, and 3 others: Migrate "WikiLambda API" instrument to use the Test Kitchen SDK - https://phabricator.wikimedia.org/T415254#11910405 (10Sfaci) [22:35:53] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Load Google Search Console data into the Data Lake - https://phabricator.wikimedia.org/T420996#11910417 (10Ahoelzl) After evaluating various technical download options and considering cost profiles (thanks @nshahquinn-wmf for all the considerations and refer...