[03:19:25] FIRING: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [05:46:18] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add cl_timestamp_id index to categorylinks table - https://phabricator.wikimedia.org/T399249#11007661 (10Marostegui) [05:47:04] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add cl_timestamp_id index to categorylinks table - https://phabricator.wikimedia.org/T399249#11007663 (10Marostegui) [07:19:25] FIRING: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [09:19:18] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add cl_timestamp_id index to categorylinks table - https://phabricator.wikimedia.org/T399249#11008312 (10Marostegui) [09:45:18] (03PS2) 10Aqu: Refine: Force partition creation at end of refine [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1169720 (https://phabricator.wikimedia.org/T369845) [09:48:08] (03CR) 10Aqu: [V:03+2 C:03+2] Refine: Force partition creation at end of refine (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1169720 (https://phabricator.wikimedia.org/T369845) (owner: 10Aqu) [09:59:39] (03Merged) 10jenkins-bot: Refine: Force partition creation at end of refine [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1169720 (https://phabricator.wikimedia.org/T369845) (owner: 10Aqu) [10:03:44] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add cl_timestamp_id index to categorylinks table - https://phabricator.wikimedia.org/T399249#11008470 (10Marostegui) [10:43:39] 10Data-Engineering (Q4 2025 April 1st - June 30th): Define Event Platform Essential Work FY26 - https://phabricator.wikimedia.org/T399661#11008645 (10Reedy) [11:19:25] FIRING: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [11:40:44] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add cl_timestamp_id index to categorylinks table - https://phabricator.wikimedia.org/T399249#11008809 (10Marostegui) [13:13:56] 06Data-Engineering, 06Data-Engineering-Radar, 10ConfirmEdit (CAPTCHA extension), 10MediaWiki-extensions-Campaigns, and 2 others: Send hCaptcha API response data to event platform - https://phabricator.wikimedia.org/T379179#11009190 (10kostajh) This is pending some discussions with hCaptcha before we can re... [13:14:37] 06Data-Engineering, 06Data-Engineering-Radar, 10ConfirmEdit (CAPTCHA extension), 10MediaWiki-extensions-Campaigns, and 2 others: Send hCaptcha API response data to event platform - https://phabricator.wikimedia.org/T379179#11009195 (10kostajh) [13:56:38] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add cl_timestamp_id index to categorylinks table - https://phabricator.wikimedia.org/T399249#11009337 (10Marostegui) [14:00:23] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add cl_timestamp_id index to categorylinks table - https://phabricator.wikimedia.org/T399249#11009362 (10Marostegui) [14:06:40] (03CR) 10Gmodena: [C:03+2] Remove TwoColConflict* sanitization allowlist entries [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1164339 (https://phabricator.wikimedia.org/T397611) (owner: 10Phuedx) [14:10:28] (03CR) 10Gmodena: [V:03+2 C:03+2] Remove TwoColConflict* sanitization allowlist entries [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1164339 (https://phabricator.wikimedia.org/T397611) (owner: 10Phuedx) [14:23:03] 06Data-Engineering, 06Data-Engineering-Radar, 10AbuseFilter, 06Trust and Safety Product Team, and 2 others: AbuseFilter abuse_filter_log table: Store IP addresses as hex values - https://phabricator.wikimedia.org/T395612#11009519 (10Tchanders) [14:56:57] 06Data-Engineering, 10CheckUser, 06Trust and Safety Product Team, 07Schema-change-in-production: Add '*_actor_ip_hex_time' indexes to 'cu_changes', 'cu_log_event', and 'cu_private_event' on WMF wikis - https://phabricator.wikimedia.org/T399728 (10Dreamy_Jazz) 03NEW [14:57:35] 06Data-Engineering, 06Data-Engineering-Radar, 10CheckUser, 06Trust and Safety Product Team, and 4 others: Add '*_actor_ip_hex_time' indexes to 'cu_changes', 'cu_log_event', and 'cu_private_event' - https://phabricator.wikimedia.org/T395683#11009691 (10Dreamy_Jazz) 05Open→03Resolved #schema-change-i... [14:58:20] 06Data-Engineering, 10CheckUser, 06DBA, 06Trust and Safety Product Team, 07Schema-change-in-production: Add '*_actor_ip_hex_time' indexes to 'cu_changes', 'cu_log_event', and 'cu_private_event' on WMF wikis - https://phabricator.wikimedia.org/T399728#11009698 (10Dreamy_Jazz) [15:13:08] 06Data-Engineering, 06Data-Engineering-Radar, 10AbuseFilter, 07Schema-change, and 2 others: AbuseFilter abuse_filter_log table: Store IP addresses as hex values - https://phabricator.wikimedia.org/T395612#11009754 (10Tchanders) [15:15:30] !log Deploying Refinery at dc1ba0e3abe78050317a00de89a12554b34a9118 / Remove TwoColConflict* sanitization allowlist entries [15:15:34] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [15:19:25] FIRING: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [15:26:30] Starting build #39 for job analytics-refinery-maven-release [15:44:28] still seeing 41 failed nodes starting with an* [15:57:43] Project analytics-refinery-maven-release build #39: 09SUCCESS in 31 min: https://integration.wikimedia.org/ci/job/analytics-refinery-maven-release/39/ [16:09:39] 06Data-Engineering, 06Data-Engineering-Radar, 10CheckUser, 06DBA, and 2 others: Add '*_actor_ip_hex_time' indexes to 'cu_changes', 'cu_log_event', and 'cu_private_event' on WMF wikis - https://phabricator.wikimedia.org/T399728#11009986 (10Ottomata) [16:12:54] 06Data-Engineering: Update documentation for banner_activity_minutely dataset. - https://phabricator.wikimedia.org/T398958#11010024 (10Ottomata) Q: perhaps this stuff should go in datahub instead of wikitech? https://datahub.wikimedia.org/dataset/urn:li:dataset:(urn:li:dataPlatform:druid,banner_activity_minut... [16:14:07] 10Data-Engineering-Roadmap: Update documentation for banner_activity_minutely dataset. - https://phabricator.wikimedia.org/T398958#11010030 (10Ottomata) [16:15:04] 06Data-Engineering, 06Data-Engineering-Radar: Update documentation for banner_activity_minutely dataset. - https://phabricator.wikimedia.org/T398958#11010035 (10Ottomata) [16:15:06] 06Data-Engineering, 06Data-Engineering-Radar: Update documentation for banner_activity_minutely dataset. - https://phabricator.wikimedia.org/T398958#11010036 (10Ottomata) [16:18:45] 06Data-Engineering, 06Experimentation Lab, 10Event-Platform: EventGate: Add Prometheus metric for hoisting errors - https://phabricator.wikimedia.org/T398922#11010045 (10Ottomata) Dan and I put this in DE Next Up. @phuedx what is the priority of this? Do you have a deadline of when you'd like it done and... [16:20:38] 06Data-Engineering: Sqoop block and block_target as replacement to the ipblocks table - https://phabricator.wikimedia.org/T398828#11010071 (10Ottomata) FWIW, event.mediawiki_user_blocks_change is a Hive table! :) [16:22:55] 06Data-Engineering, 10Data-Platform, 06Growth-Team, 10Event-Platform: EventBus throws TypeError: array_map(): Argument #2 ($array) must be of type array, int given when the event is too large - https://phabricator.wikimedia.org/T398506#11010089 (10Ottomata) p:05Triage→03Medium [16:26:12] 06Data-Engineering: Adapt Sqoop for categorylinks schema change - https://phabricator.wikimedia.org/T397923#11010111 (10Ottomata) p:05Triage→03High [16:28:28] 06Data-Engineering: Adapt Sqoop for categorylinks schema change - https://phabricator.wikimedia.org/T397923#11010125 (10Ottomata) @xcollazo , Dan and I are grooming, and this needs to happen soon. We'd like to spread a bit of knowledge around, would you be willing to make this happen? @Milimetric can help. [16:28:34] 10Data-Engineering (Q4 2025 April 1st - June 30th): Adapt Sqoop for categorylinks schema change - https://phabricator.wikimedia.org/T397923#11010127 (10Ottomata) [16:30:02] 06Data-Engineering, 10Data-Platform, 05http-client-hints: Client hints not recorded in the webrequest table - https://phabricator.wikimedia.org/T397464#11010135 (10Ottomata) p:05Triage→03Medium [16:30:45] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10EventStreams, 10Event-Platform: SSE events from offline DC topics - https://phabricator.wikimedia.org/T396564#11010137 (10Ottomata) p:05Triage→03High [16:30:54] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10EventStreams, 10Event-Platform: SSE events from offline DC topics - https://phabricator.wikimedia.org/T396564#11010141 (10Ottomata) [18:19:28] 10Data-Engineering (Q4 2025 April 1st - June 30th), 06Experimentation Lab, 10Event-Platform: EventGate: Add Prometheus metric for hoisting errors - https://phabricator.wikimedia.org/T398922#11010509 (10Ahoelzl) a:03tchin [19:19:25] FIRING: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [22:04:58] 10Data-Engineering (Q4 2025 April 1st - June 30th): Avoid Gobblin SimpleSkein DAG runs queuing up by disabling "catchup" - https://phabricator.wikimedia.org/T393618#11011235 (10amastilovic) 05Open→03Resolved [22:07:48] 10Data-Engineering (Q4 2025 April 1st - June 30th): Update airflow-dags wmf_airflow_common library rename analytics to main - https://phabricator.wikimedia.org/T393111#11011254 (10amastilovic) This has been addressed in the following MR: https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_re... [22:08:00] 10Data-Engineering (Q4 2025 April 1st - June 30th): Update airflow-dags wmf_airflow_common library rename analytics to main - https://phabricator.wikimedia.org/T393111#11011255 (10amastilovic) 05Open→03Resolved [23:19:25] FIRING: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem