Skip to content

Add support for on-disk SQLite store in computerd - #123

Open
aron-cf wants to merge 1 commit into
mainfrom
computerd-on-disk-store
Open

Add support for on-disk SQLite store in computerd#123
aron-cf wants to merge 1 commit into
mainfrom
computerd-on-disk-store

Conversation

@aron-cf

@aron-cf aron-cf commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

computerd keeps the workspace in memory. When it restarts, the store is gone and the durable object sends every path again before any command can run. For unsynced directories like node_modules this requires a full re-build.

Set COMPUTERD_DB to an absolute path to use a SQLite DB on disk, this can then be backed up and restored across container instances.

COMPUTERD_DB=memory                      # default, unchanged
COMPUTERD_DB=/var/lib/computerd/state.db # on disk, survives a restart

The path must be absolute and must sit outside MOUNT_POINT. A database the mount also shows would feed its writes back to itself. computerd refuses to start on either mistake.

The cloudflared process then starts back knowing what it was already sent, so only the difference needs to be synced.

import { Database, initializeSchema } from "@cloudflare/dofs";
import { NodeSQLiteStorage } from "@cloudflare/dofs/node";

const storage = new NodeSQLiteStorage({ location: "/var/lib/computerd/state.db" });
const db = new Database(storage);
initializeSchema(db, Date.now);

storage.checkpoint(); // { walFrames, sizeBytes, durationMs }
storage.close();

POST /__computerd/checkpoint collapses the write-ahead log back into the database file, so a snapshot holds one file instead of two. The same happens on SIGTERM, after the mount is released — the mount writes buffered bytes as it closes each file, so it has to go first. GET /__computerd/info reports the store in use, and GET /__computerd/stats reports its size.

Restart by hand
PORT=8080 MOUNT_POINT=/workspace FUSE_MOUNT=fuse \
  COMPUTERD_DB=/var/lib/computerd/state.db computerd &

echo hello > /workspace/a.txt
curl -X POST localhost:8080/__computerd/checkpoint
kill %1

PORT=8080 MOUNT_POINT=/workspace FUSE_MOUNT=fuse \
  COMPUTERD_DB=/var/lib/computerd/state.db computerd &

cat /workspace/a.txt                    # hello
curl -s localhost:8080/api/watermarks   # positions kept, not zeroed

Tests cover the storage layer, the new setting, and the point of the change: restart on a file store and the files and sync positions are still there, restart in memory and they are not. The unmount-then-checkpoint order has its own test, since reversing it would lose the last writes quietly. The computerd README covers the setting and its limits, and the performance document carries the numbers.


Devin Review

computerd keeps its workspace in memory, so a restart loses it and the
durable object has to send every path again. On a large workspace that
replay is most of the time it takes to get back to work.

Set COMPUTERD_DB to an absolute path and the store goes on the
container's disk instead. The sync positions live in the same database,
so a restarted daemon still knows what it was sent and the durable
object only sends the difference: about 25ms instead of a full resend,
whatever the size of the tree.

dofs gains a ./node export with NodeSQLiteStorage, which runs
node:sqlite against a file or memory. It stays out of the main entry
point because that has to load under workerd, which has no node:sqlite.
SQLiteTestStorage is now the in-memory pinning of the same class.

Also adds POST /__computerd/checkpoint for folding the write-ahead log
back before a disk snapshot, reports the store on /__computerd/info,
and adds store size and free pages to /__computerd/stats.

Measured with script/store-compare.mjs and script/restore-time.mjs;
numbers in packages/computerd/bench-results.md. Reads are not slower on
disk. Writes cost 10 to 20 percent more through a real FUSE mount. The
in-memory store stays the default.
@changeset-bot

changeset-bot Bot commented Aug 27, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: aa4f984

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@cloudflare/dofs Minor
@cloudflare/computer Minor
@cloudflare/computer-rpc Minor
@cloudflare/computerd Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new

pkg-pr-new Bot commented Aug 27, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@cloudflare/computer@123

commit: aa4f984

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant