Skip to content

fix: drop shared meeting host accounts in the meetings source query - #5064

Merged
mbani01 merged 1 commit into
mainfrom
fix/snowflake-meetings-shared-host-emails-source-query
Oct 21, 2026
Merged

mbani01 merged 1 commit into
mainfrom
fix/snowflake-meetings-shared-host-emails-source-query

Conversation

@mbani01

@mbani01 mbani01 commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Summary

Since #5043 the meetings transformer drops rows for the 9 shared meeting host accounts (zoom@hyperledger.org, meetings@dpdk.org and the rest). Those drops are counted as transformSkippedCount, which trips the Metaplane monitor "[SNOWFLAKE INTEGRATIONS] Skipped activities due to invalid transformation" (expected = 0) every night, and keeps the daily S3 cleanup from deleting the affected batches since it only cleans jobs with skippedCount = 0.

The first export on the new image (2026-10-21 00:20Z) had 19 such skips across 14 of 48 meetings batches, out of 511,884 transformed rows. No other source had any. Datadog shows no "Failed to transform row" warnings, so none of them are real transformation failures.

This filters the shared host accounts in the meetings source query itself, so the rows never leave Snowflake and the skip counters go back to 0.

Changes

  • SHARED_MEETING_ACCOUNT_EMAILS is exported from core/personalEmail.ts so the SQL and the transformer read the same list.
  • The meetings buildSourceQuery adds LOWER(TRIM(t.INVITEE_EMAIL)) NOT IN (...) next to the existing empty-email predicate, rendered from that set.
  • cleanPersonalEmail in the transformer is unchanged. A row that still reaches it is now a genuine skip the monitor should report.
  • One assertion in the source query test covers the new predicate.

Other sources are untouched: none of them produced a skip, and the listed accounts are meeting hosts.

Type of change

  • Bug fix
  • New feature
  • Refactor / cleanup
  • Performance improvement
  • Chore / dependency update
  • Documentation

Related issue

Follow-up to linuxfoundation/lfx-cdp-insights-ops#853 (closed by #5043).

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>
Copilot AI balanced review requested due to automatic review settings October 21, 2026 09:34
@cursor

cursor Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

PR Summary

Low Risk
Narrow meetings export filter using an existing shared-email list; transformer fallback unchanged and scope is meetings-only.

Overview
Filters shared meeting host invitee emails in Snowflake so those rows are never exported, instead of being dropped later in the transformer as “skipped” transforms (which was tripping Metaplane and blocking S3 batch cleanup).

SHARED_MEETING_ACCOUNT_EMAILS is exported from personalEmail.ts and the meetings buildSourceQuery adds LOWER(TRIM(t.INVITEE_EMAIL)) NOT IN (...) built from that set. Transformer behavior via cleanPersonalEmail is unchanged as a safety net. A test asserts the new SQL predicate.

Reviewed by Cursor Bugbot for commit e93d6f6. Bugbot is set up for automated code reviews on this repo. Configure here.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The narrow change consistently reuses the existing exclusion list, with no blocking issues identified; tests were not executed.

No open issues 🎉

What changed in this PR

Moves shared meeting-host filtering into the Snowflake source query, preventing intentional exclusions from inflating skip counters and blocking S3 cleanup.

Changes:

  • Reuses the transformer’s existing shared-account list in SQL.
  • Excludes matching emails before export, leaving transformer behavior unchanged.
  • Adds an assertion covering the query predicate.
File Description
services/​apps/​snowflake_connectors/​src/​integrations/​meetings/​meeting-attendance/​buildSourceQuery.ts Filters shared meeting accounts before export.
services/​apps/​snowflake_connectors/​src/​integrations/​meetings/​meeting-attendance/​buildSourceQuery.test.ts Verifies the exclusion predicate.
services/​apps/​snowflake_connectors/​src/​core/​personalEmail.ts Exports the existing shared-account list.

Review effort: Balanced

@mbani01
mbani01 merged commit 5a83725 into main Oct 21, 2026
14 checks passed
@mbani01
mbani01 deleted the fix/snowflake-meetings-shared-host-emails-source-query branch October 21, 2026 09:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants