Skip to content

docs: park the file-level backup design drafts - #345

Draft
onel wants to merge 1 commit into
devfrom
docs/backup-design-drafts
Draft

docs: park the file-level backup design drafts#345
onel wants to merge 1 commit into
devfrom
docs/backup-design-drafts

Conversation

@onel

@onel onel commented Jul 19, 2026

Copy link
Copy Markdown
Collaborator

What the design is

A box captures its own data — user content, shared content, app instances, brain state — deduplicated and encrypted client-side, and pushes it off-box daily, because nothing can reach into a hosted box. Databases are dumped rather than file-copied. Restore lands the bundle on a fresh, current box. The artifact is a logical bundle rather than a disk image, which is what would let one mechanism serve both disaster recovery and the "move it home" migration promise.

Why it was parked, and what brings it back

The chosen v1 mechanism is whole-disk, crash-consistent, one-day-of-history, hosted-only, and bounded by a provider cap that is account-wide rather than per-server. This design has none of those limits. It comes back when any of them starts to bite — most likely the cap, or the first customer who wants one deleted file back.

The part worth reading even if this never lands

docs/drafts/backup-file-level/README.md # Findings worth keeping. These came from checking the drafts against the code and are true regardless of which backup mechanism ships:

  • Two live doc/code drifts. ENVIRONMENT.md # Admin bootstrap describes a seed carrying admin_bootstrap_secret; internal/profile/seed.go shows assertion_verification_key, and internal/store/store.go records that it replaced that secret. Separately STORAGE.md's layout diagram has brain/state.db and managed-services/ where the code has <stateDir>/malmo.db and <stateDir>/services/.
  • data_volumes / cache_volumes are authored but unparsed, with nesting rules nothing enforces. This blocks more than backup: the pre-update snapshot in UPDATES.md — the v1 rollback safety net, a locked decision — is unbuilt for exactly this missing field.
  • Managed-service data lives outside the instance directories, so any backup copying only instances/<id>/ restores a database-backed app with its rows gone.
  • Restoring the brain's SQLite cannot be a file copy. box_meta holds box-scoped identity and the brain ignores the seed when a box_id is present, so a wholesale restore makes a fresh box adopt a destroyed box's identity — failing silently until the first certificate operation.
  • An include list of user folders is unsafe, because users may rename, delete, and add folders.
  • Password hashes are in no backup set by default — PAM is the source of truth and the brain holds no hash.

The first two are worth fixing independently of any backup work.

Do not merge

This is parked design work, not a proposal. It is opened as a draft PR so the work is findable and reviewable, not because it is meant to land. Nothing here is decided, implemented, or scheduled.

The hosted product went with a simpler v1 mechanism that needs no OS-side code at all, so none of this is on the critical path. The branch exists so the design — and more importantly the corrections that came out of writing it — survive until the design is revisited.

docs/drafts/ is deliberately not one of the three doc homes (docs/specs/, docs/progress/, docs/dev/). If this is ever adopted, the files move: BACKUP.md to docs/specs/, the box-identity section into docs/specs/ENVIRONMENT.md, the issue drafts into GitHub issues.

Design work from a pass that was not taken forward: hosted v1 uses a
control-plane-side provider snapshot instead, which needs no OS-side code.

Parked rather than dropped because the design is the destination once the
provider snapshot cap or whole-disk granularity starts to bite, and because
the pass surfaced several findings that are true regardless of which backup
mechanism ships -- two doc/code drifts, the unparsed data_volumes field that
also blocks the specced pre-update snapshot, and the brain-state identity
split that makes a naive restore fail silently.

docs/drafts/ is deliberately not one of the three doc homes; this branch is
not meant to merge. See docs/drafts/backup-file-level/README.md.
@onel onel changed the title docs: park the file-level backup design drafts (DO NOT MERGE) docs: park the file-level backup design drafts Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant