docs: park the file-level backup design drafts - #345
Draft
onel wants to merge 1 commit into
Draft
Conversation
Design work from a pass that was not taken forward: hosted v1 uses a control-plane-side provider snapshot instead, which needs no OS-side code. Parked rather than dropped because the design is the destination once the provider snapshot cap or whole-disk granularity starts to bite, and because the pass surfaced several findings that are true regardless of which backup mechanism ships -- two doc/code drifts, the unparsed data_volumes field that also blocks the specced pre-update snapshot, and the brain-state identity split that makes a naive restore fail silently. docs/drafts/ is deliberately not one of the three doc homes; this branch is not meant to merge. See docs/drafts/backup-file-level/README.md.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What the design is
A box captures its own data — user content, shared content, app instances, brain state — deduplicated and encrypted client-side, and pushes it off-box daily, because nothing can reach into a hosted box. Databases are dumped rather than file-copied. Restore lands the bundle on a fresh, current box. The artifact is a logical bundle rather than a disk image, which is what would let one mechanism serve both disaster recovery and the "move it home" migration promise.
Why it was parked, and what brings it back
The chosen v1 mechanism is whole-disk, crash-consistent, one-day-of-history, hosted-only, and bounded by a provider cap that is account-wide rather than per-server. This design has none of those limits. It comes back when any of them starts to bite — most likely the cap, or the first customer who wants one deleted file back.
The part worth reading even if this never lands
docs/drafts/backup-file-level/README.md# Findings worth keeping. These came from checking the drafts against the code and are true regardless of which backup mechanism ships:ENVIRONMENT.md# Admin bootstrap describes a seed carryingadmin_bootstrap_secret;internal/profile/seed.goshowsassertion_verification_key, andinternal/store/store.gorecords that it replaced that secret. SeparatelySTORAGE.md's layout diagram hasbrain/state.dbandmanaged-services/where the code has<stateDir>/malmo.dband<stateDir>/services/.data_volumes/cache_volumesare authored but unparsed, with nesting rules nothing enforces. This blocks more than backup: the pre-update snapshot inUPDATES.md— the v1 rollback safety net, a locked decision — is unbuilt for exactly this missing field.instances/<id>/restores a database-backed app with its rows gone.box_metaholds box-scoped identity and the brain ignores the seed when abox_idis present, so a wholesale restore makes a fresh box adopt a destroyed box's identity — failing silently until the first certificate operation.The first two are worth fixing independently of any backup work.
Do not merge
This is parked design work, not a proposal. It is opened as a draft PR so the work is findable and reviewable, not because it is meant to land. Nothing here is decided, implemented, or scheduled.
The hosted product went with a simpler v1 mechanism that needs no OS-side code at all, so none of this is on the critical path. The branch exists so the design — and more importantly the corrections that came out of writing it — survive until the design is revisited.
docs/drafts/is deliberately not one of the three doc homes (docs/specs/,docs/progress/,docs/dev/). If this is ever adopted, the files move:BACKUP.mdtodocs/specs/, the box-identity section intodocs/specs/ENVIRONMENT.md, the issue drafts into GitHub issues.