Skip to content

Docs: document the Transaction write API (#1008) - #3809

Open
iSparshP wants to merge 2 commits into
apache:mainfrom
iSparshP:docs-write-transaction-api-1008
Open

Docs: document the Transaction write API (#1008)#3809
iSparshP wants to merge 2 commits into
apache:mainfrom
iSparshP:docs-write-transaction-api-1008

Conversation

@iSparshP

Copy link
Copy Markdown

Addresses #1008. The write docs already cover append / overwrite / overwrite_filter / delete / dynamic_partition_overwrite / upsert, but the Transaction API was undocumented in this section. This adds a "Write API modes: Table and Transaction" subsection explaining both modes, the single-atomic-commit guarantee, and combining schema + data changes in one commit. It also adds a ### Delete heading so each operation is its own subsection, per the issue.

Builds on the direction from the earlier #2770 and #3647 (both closed by the stale bot). Handed off by @pramila-bishnoi on the issue thread.

Closes #1008

@iSparshP

Copy link
Copy Markdown
Author

@kevinjqliu this picks up #1008 after the handoff from @pramila-bishnoi. It's a small docs-only change: adds the Transaction write API (which was missing from this section) and a Delete subsection so each operation has its own heading. Would appreciate a review when you get a chance. Thanks!

@iSparshP

Copy link
Copy Markdown
Author

Gentle bump on this one. CI is green and it's a small docs-only change picking up #1008 after @pramila-bishnoi's handoff. @kevinjqliu @Fokko whenever one of you has a moment for a quick review, would really appreciate it. Happy to adjust anything.

Comment thread mkdocs/docs/api.md Outdated

Every write operation is available through two APIs.

The **`Table` API** exposes each operation directly on the table object: `tbl.append(...)`, `tbl.overwrite(...)`, `tbl.delete(...)`, `tbl.dynamic_partition_overwrite(...)` and `tbl.upsert(...)`. Each call opens a transaction, applies the single operation, and commits it as one atomic snapshot. This is the simplest mode and the right default when you only need a single write.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A small nit in the wording. The tbl.upsert(...) could potentially generate more than one snapshot, but it will be written atomically. Maybe we should be less strict in the wording:

Suggested change
The **`Table` API** exposes each operation directly on the table object: `tbl.append(...)`, `tbl.overwrite(...)`, `tbl.delete(...)`, `tbl.dynamic_partition_overwrite(...)` and `tbl.upsert(...)`. Each call opens a transaction, applies the single operation, and commits it as one atomic snapshot. This is the simplest mode and the right default when you only need a single write.
The **`Table` API** exposes each operation directly on the table object: `tbl.append(...)`, `tbl.overwrite(...)`, `tbl.delete(...)`, `tbl.dynamic_partition_overwrite(...)` and `tbl.upsert(...)`. Each call opens a transaction, applies the operation, and commits atomically. This is the simplest mode and the right default when you only need a single write.

@Fokko

Fokko commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

@iSparshP I left one comment, apart from that, this looks good. Thanks for picking this up 👍

upsert may produce more than one snapshot while still committing
atomically, so describe the Table API as committing atomically rather
than as a single snapshot.
@iSparshP

iSparshP commented Sep 2, 2026

Copy link
Copy Markdown
Author

thanks @Fokko, good catch. you are right that upsert can end up as more than one snapshot even though its still atomic, so the old wording was too strict. i took your suggestion and reworded it to just say it commits atomicaly instead of as one snapshot. pushed the change, let me know if you want anything else tweaked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DOCS: Improve Documentation on Write Support

2 participants