Skip to content

Use native Iceberg reference expiration - #714

Merged
mkuchenbecker merged 1 commit into
mkuchenbecker/bdp-106337-default-max-ref-agefrom
mkuchenbecker/bdp-106337-native-ref-expiration
Sep 4, 2026
Merged

mkuchenbecker merged 1 commit into
mkuchenbecker/bdp-106337-default-max-ref-agefrom
mkuchenbecker/bdp-106337-native-ref-expiration

Conversation

@mkuchenbecker

Copy link
Copy Markdown
Collaborator

Stack

Summary

Replace the custom branch-reference removal pass from PR 708 with Iceberg's native reference expiration.

Snapshot-expiration maintenance backfills history.expire.max-ref-age-ms to seven days when the property is missing. New tables receive the same default during creation, while an existing configured value is preserved. Iceberg then expires eligible branches and tags through its standard snapshot-expiration commit.

Changes

  • Client-facing API Changes
  • Internal API Changes
  • Bug Fixes
  • New Features
  • Performance Improvements
  • Code Style
  • Refactoring
  • Documentation
  • Tests

Testing Done

  • Manually Tested on local docker setup. Please include commands ran, and their output.
  • Added new tests for the changes made.
  • Updated existing tests to reflect the changes made.
  • No tests added or updated. Please explain why. If unsure, please feel free to ask for help.
  • Some other form of testing like staging or soak time in production. Please explain.

A real Iceberg table integration test removes the property to simulate a legacy table, creates a branch and tag on an older snapshot, runs snapshot expiration, and verifies that the property is backfilled and both references expire. The integration test passes with Iceberg 1.2 and Iceberg 1.5.

The full OperationsTest suite passes for both application variants. OpenHouseInternalRepositoryImplTest verifies the table-creation default and preservation of a configured value. The affected Spotless checks pass.

Additional Information

  • Breaking Changes
  • Deprecations
  • Large PR broken into smaller PRs, and PR plan linked in the description.

Stamp the seven-day maximum reference age on new tables and backfill it when snapshot maintenance encounters a legacy table. Delegate branch and tag expiration to native Iceberg reference handling.

BDP-106337

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mkuchenbecker
mkuchenbecker marked this pull request as ready for review September 4, 2026 17:11
@mkuchenbecker
mkuchenbecker merged commit 3c00bcc into mkuchenbecker/bdp-106337-default-max-ref-age Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant