Skip to content

Avoid rewriting namespace tombstones during repeated deletion - #12009

Draft
taylan-oai wants to merge 1 commit into
temporalio:mainfrom
taylan-oai:dev/taylan/namespace-mutation-consistency
Draft

taylan-oai wants to merge 1 commit into
temporalio:mainfrom
taylan-oai:dev/taylan/namespace-mutation-consistency

Conversation

@taylan-oai

Copy link
Copy Markdown

Summary

Skip the namespace state update when a repeated deletion finds the namespace already deleted, preventing that activity from recreating a tombstone removed by an earlier reclaim workflow.

Problem

A repeated DeleteNamespace request can run while the earlier asynchronous reclaim workflow is still active. MarkNamespaceDeletedActivity writes DELETED even if it reads that state from persistence. In Cassandra, physical deletion does not advance the notification version, so a reclaim between the activity's read and update allows the stale update to recreate the name row while its ID mapping remains absent.

Approach

Return success when the persisted namespace state is already DELETED. The activity performs no write in that case. Registered and deprecated namespaces still transition to deleted using the existing notification-version check.

Validation

  • Native changed-code lint passed.
  • Unit regression covers registered, deprecated and already-deleted namespaces, preserves metadata-before-namespace read ordering, and checks ID, name, global status and notification version on state transitions.
  • A temporary native Cassandra 5.0.9 regression invokes the actual activity and physically deletes the namespace after its read. It failed before the change because name lookup still succeeded after reclaim, then passed three times after the change and three times with the race detector.
  • All deletion-worker package tests, their race-enabled runs, and the full Cassandra metadata persistence suite passed.

Risks, rollout, and scope

This change prevents the repeated-deletion activity from resurrecting an already-deleted namespace. It does not make Cassandra's separate name and ID mutations atomic or repair existing inconsistent rows. A complete persistence-level fix needs an ownership and recovery protocol across those mappings. No schema migration is required for this change. The native interleaving fixture was run separately; the permanent unit regression guards the no-write behavior without adding a database dependency to the worker unit suite.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants