Repository navigation
sql_store: S3 and filesystem object stores (slice 1b PR B) - #441
Merged
Merged
Conversation
added 7 commits
October 24, 2026 03:51
Adds the S3 and filesystem backends of SqlObjectStore. Objects are
created at most once and never replaced.
- S3: conditional create (If-None-Match: *) with the SHA-256 as user
metadata, a startup probe that selects conditional or check-then-create
mode, range reads and head.
- Filesystem: temp file, fsync, hard link create under {datadir}/sql-store,
with boot checks for the root.
- Conformance tests shared by the in-memory, filesystem and S3 stores;
S3 tests run against an in-process S3 API and, in the at-rest-harness
job, against the harness MinIO.
- The S3 block store's start-up size scan lists the tinycloud: key prefix.
Details: TC-1491
Details: TC-1491
Details: TC-1491
Details: TC-1491
Details: TC-1491
…t conditional PUTs on the provider Details: TC-1491
…Spec Details: TC-1491
Contributor
Author
|
APPROVE: see TC-1491 (reviewed exact head 8d69358) |
- The probe maps only 501 and NotImplemented to check-then-create; any other status is an error. The deciding answer's status and code are logged with the mode, and a rejected conditional create reports its status and code. - The per-key gate is released by a guard, so a cancelled create leaves no entry behind. - Relative filesystem roots are made absolute before the block store root check; the parent of an existing directory is fsynced on create. - The S3 scan prefix and the space id text share one constant, SPACE_ID_PREFIX. - Tests that planted different bytes derive them from the wanted bytes. - The new CI step has a 10 minute timeout. Contract section 12 updated. Details: TC-1491
Contributor
Author
|
APPROVE: see TC-1491 (reviewed exact head 635d73c) |
1 similar comment
Contributor
Author
|
APPROVE: see TC-1491 (reviewed exact head 635d73c) |
samgbafa
marked this pull request as ready for review
October 24, 2026 06:28
# Conflicts: # CHANGELOG.md
# Conflicts: # CHANGELOG.md # tinycloud-node-server/src/storage/mod.rs # tinycloud-node-server/src/storage/s3.rs
Contributor
Author
|
APPROVE: see TC-1491 (reviewed exact head ec5ee9b) |
# Conflicts: # CHANGELOG.md
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the S3 and filesystem backends of the SQL object store trait. Neither is started by the node yet.
S3SqlObjectStore: conditional create (If-None-Match: *) with the SHA-256 as user metadata; a 412 is accepted only after the stored length and hash match. A startup probe undersql/v1/_probe/selects conditional or check-then-create mode and logs the mode once. Range reads and head.FsSqlObjectStore: temp file, fsync, hard-link create under{datadir}/sql-store; boot checks for the root (absolute, outside the block store root).at-rest-harnessjob against the harness MinIO (new step,--test sql_object_store -- --ignored).tinycloud:key prefix; the sizes it computes are unchanged (test included).docs/sql-block-store-contract.mdsection 12. CHANGELOG line.The conditional-create conformance test against MinIO takes about two minutes; per-call timeouts for the uploader are chosen with this in mind.
Details: TC-1491