Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/document-symlink-surface.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@cloudflare/computer": patch
---

Document the symbolic-link filesystem surface. `docs/04_filesystem_interface.md` claimed that symbolic links were internal and that `Workspace.fs` had no `symlink`, `readlink`, `lstat`, or `chmod`, none of which matched the shipped API. Those four methods now have sections of their own covering return values and the `ENOENT`, `EINVAL`, and `ELOOP` cases, the comparison with `node:fs/promises` maps them, and the specification explains that `stat` follows a trailing link while `lstat` reports the link itself.
6 changes: 6 additions & 0 deletions .changeset/expose-fs-rename.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
"@cloudflare/dofs": minor
"@cloudflare/computer": minor
---

`Workspace.fs` gains `rename(oldPath, newPath)`, exposing the store's existing transactional move through the public surface and through `WorkspaceFilesystemStub`. An existing destination is replaced when the two ends agree on kind — a file or symbolic link for a file or symbolic link, an empty directory for a directory — and the operation reports `ENOENT`, `ENOTEMPTY`, `EISDIR`, `ENOTDIR`, `EINVAL`, and `EROFS` as documented in `docs/04_filesystem_interface.md`. The Worker shell's `mv` now calls it, so an interrupted move no longer leaves the entry at both paths or a directory half copied.
12 changes: 12 additions & 0 deletions .changeset/find-exclude-globs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
"@cloudflare/dofs": minor
"@cloudflare/computer": minor
---

`find` accepts `exclude`, a list of glob patterns matched against the same directory-relative path as the inclusion glob. Exclusion is decided first, so it always wins, and an excluded directory is pruned during traversal: neither it nor anything below it is read. `limit` and `offset` apply to the matches that survive. The option travels through `WorkspaceFilesystem`, `WorkspaceFilesystemStub`, and the public find tool.

```ts
const sources = await workspace.fs.find("/workspace", "**/*.ts", {
exclude: ["node_modules", "node_modules/**", ".git", ".git/**"],
});
```
5 changes: 5 additions & 0 deletions .changeset/mkdir-follows-symlinks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@cloudflare/dofs": patch
---

`mkdir` now follows symbolic links in intermediate path segments, so a link to a directory resolves transparently instead of failing with `ENOTDIR`. Creating `/alias/new-directory` where `/alias` points at `/real` creates `/real/new-directory`, and recursive creation places its missing ancestors under the resolved parent. A resolved parent that is a file still reports `ENOTDIR`, a dangling parent link reports `ENOENT`, and a chain longer than the shared forty-hop budget reports `ELOOP`.
177 changes: 155 additions & 22 deletions docs/04_filesystem_interface.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,8 +190,10 @@ stat(path: string): Promise<{
`name` is the last segment of the canonicalized path. For the workspace
root this is the empty string: `(await fs.stat("/")).name === ""`.

`stat` follows symlinks transparently; there is no `lstat`. See the
note on internal symlink support in the appendix.
`stat` follows a trailing symbolic link, so it reports the file or
directory the link points at. Use [`lstat`](#lstat) to inspect the link
itself. A dangling link makes `stat` report `ENOENT` while `lstat`
succeeds.

> When a parent path segment is itself a file, `stat` reports `ENOENT`
> (because resolution returns `null` for that case) rather than
Expand All @@ -203,6 +205,117 @@ const s = await fs.stat("/workspace/build/out.wasm");
console.log(`${s.size} bytes, modified ${new Date(s.mtime).toISOString()}`);
```

### `lstat`

```ts
lstat(path: string): Promise<{
name: string;
mode: number;
mtime: number; // ms since epoch
size: number;
isFile: boolean;
isDirectory: boolean;
isSymbolicLink: boolean;
}>
```

Same shape as `stat`, but a trailing symbolic link is reported as the
link rather than followed. `size` is then the byte length of the stored
target string, and `isSymbolicLink` is true. Intermediate segments are
still followed, so a link in the middle of the path resolves as usual.
Throws `ENOENT` when the path does not exist and `ELOOP` when an
intermediate chain exceeds 40 hops.

```ts
await fs.symlink("/workspace/real.txt", "/workspace/alias.txt");
(await fs.stat("/workspace/alias.txt")).isSymbolicLink; // false
(await fs.lstat("/workspace/alias.txt")).isSymbolicLink; // true
```

### `symlink`

```ts
symlink(target: string, path: string): Promise<void>
```

Creates a symbolic link at `path` pointing at `target`, with the
argument order of `node:fs/promises`. The target is stored verbatim: it
may be absolute or relative, and it is allowed to dangle. Reads and
writes that walk through the link follow it, with the same 40-hop cap as
every other resolution.

Throws `EEXIST` when `path` already exists (the link is never replaced
silently), `ENOENT` when the parent directory is missing, `ENOTDIR` when
a parent segment is a file, and `EROFS` under a read-only mount.

```ts
await fs.symlink("../shared/config.json", "/workspace/app/config.json");
```

### `readlink`

```ts
readlink(path: string): Promise<string>
```

Returns the stored target of a symbolic link, exactly as it was
written — relative targets are not resolved. Throws `EINVAL` when `path`
is not a symbolic link and `ENOENT` when it does not exist.

```ts
await fs.readlink("/workspace/app/config.json"); // "../shared/config.json"
```

### `chmod`

```ts
chmod(path: string, mode: number): Promise<void>
```

Changes the permission bits of an existing path without rewriting its
bytes. The mode is masked to twelve bits. Like POSIX `chmod`, a trailing
symbolic link is followed, so the change lands on the target rather than
the link. Throws `ENOENT` for a missing path and `EROFS` under a
read-only mount.

```ts
await fs.chmod("/workspace/bin/run.sh", 0o755);
```

### `rename`

```ts
rename(oldPath: string, newPath: string): Promise<void>
```

Moves a file, directory, or symbolic link in a single transaction, so an
interrupted call can never leave the entry at both paths or a directory
partly copied. The moved entry keeps its inode, its bytes, and its mode;
a directory move carries its whole subtree.

Overwrite behavior follows POSIX `rename(2)`: an existing destination is
replaced when the two ends agree on kind. A file or symbolic link
replaces a file or symbolic link, and a directory replaces an *empty*
directory. Nothing else is replaced.

| Code | When |
| --- | --- |
| `ENOENT` | `oldPath` does not exist, or `newPath`'s parent directory is missing. |
| `ENOTEMPTY` | `newPath` is a directory with children. |
| `EISDIR` | `newPath` is a directory and `oldPath` is not. |
| `ENOTDIR` | `oldPath` is a directory and `newPath` is not. |
| `EINVAL` | Either end is the root, or a directory would be moved inside itself. |
| `EROFS` | Either end falls under a read-only mount. |

```ts
// Publish a build atomically.
await fs.writeFile("/workspace/site/index.html.tmp", html);
await fs.rename("/workspace/site/index.html.tmp", "/workspace/site/index.html");

// Move a whole tree.
await fs.rename("/workspace/draft", "/workspace/published");
```

### `find`

```ts
Expand All @@ -212,6 +325,7 @@ find(
options?: {
limit?: number;
offset?: number;
exclude?: string[];
},
): Promise<Array<{ path; type: "file" | "dir" }>>
```
Expand All @@ -225,12 +339,24 @@ its absolute path — so `**/*.ts` under `/workspace/src` matches
The glob supports `*`, `**`, `**/`, and `?`. Character classes and
brace expansions are matched literally.

`exclude` takes globs of the same shape, matched against the same
relative path. An exclusion is decided before the inclusion glob, so it
always wins. When an excluded entry is a directory the walk prunes it:
neither the directory nor anything beneath it is read, which is what
makes skipping `node_modules` or `.git` cheap rather than merely quiet.
`limit` and `offset` then paginate whatever survives.

```ts
// Every TypeScript file in the project.
const ts = await fs.find("/workspace/src", "**/*.ts");

// Everything under a directory (no pattern).
const all = await fs.find("/workspace/notes");

// Skip generated trees without descending into them.
const sources = await fs.find("/workspace", "**/*.ts", {
exclude: ["node_modules", "node_modules/**", ".git", ".git/**"],
});
```

### `ls`
Expand Down Expand Up @@ -321,12 +447,12 @@ so handlers from Node code port over directly.
| Code | When |
| --- | --- |
| `ENOENT` | Path does not exist and `force` is not true. Also raised by `stat` when a parent segment turns out to be a file. |
| `ENOTEMPTY` | Path is a non-empty directory and `recursive` is not true. |
| `ENOTDIR` | A parent path segment is a file (raised explicitly by `mkdir` and `writeFile`; `find` raises it when its `directory` argument is a file). |
| `EISDIR` | Expected a file, got a directory (e.g. `readFile` on a dir, `writeFile` on `/`). |
| `EEXIST` | `mkdir` without `recursive: true` on an existing path. |
| `EINVAL` | Invalid path or unsupported options. |
| `ELOOP` | Symlink traversal exceeded 40 hops. Thrown by the internal resolver when the `node:vfs` adapter wires up a cycle. |
| `ENOTEMPTY` | Path is a non-empty directory and `recursive` is not true. Also raised by `rename` when the destination directory has children. |
| `ENOTDIR` | A parent path segment is a file (raised explicitly by `mkdir` and `writeFile`; `find` raises it when its `directory` argument is a file; `rename` raises it when a directory would replace a non-directory). |
| `EISDIR` | Expected a file, got a directory (e.g. `readFile` on a dir, `writeFile` on `/`, `rename` of a file onto a directory). |
| `EEXIST` | `mkdir` without `recursive: true` on an existing path, or `symlink` onto an existing path. |
| `EINVAL` | Invalid path or unsupported options: `readlink` on something that is not a symbolic link, `rename` of the root or of a directory into itself. |
| `ELOOP` | Symbolic-link traversal exceeded 40 hops. Every path-walking method shares that budget, so a cycle surfaces from `stat`, `readFile`, `writeFile`, `mkdir`, and the rest alike. |
| `EPERM` | Operation is forbidden, e.g. deleting the workspace root. |
| `EIO` | Backing storage failed unexpectedly. |
| `EACCES` | *Reserved for future mount layer (see [06. Mount Interface](./06_mount_interface.md)).* No code path in `workspace-fs` currently throws it. |
Expand Down Expand Up @@ -379,28 +505,35 @@ maps to `Workspace.fs`:
| `rm` | `rm` | `{ recursive: true }` for non-empty dirs. |
| `unlink` | `rm` | Same. |
| `readdir` | `readdir` | Always returns dirent-shaped entries. |
| `stat` / `lstat` | `stat` | No `lstat`; `stat` follows symlinks. See note below. |
| `stat` / `lstat` | `stat` / `lstat` | `stat` follows a trailing symbolic link; `lstat` reports the link. |
| `truncate` | — | Read, slice, write. |
| `chmod` | | Pass `mode` to `writeFile` / `mkdir` at create time. There is no way to chmod an existing file without rewriting its bytes. |
| `chmod` | `chmod` | Mode masked to twelve bits; follows a trailing symbolic link. `mode` can also be passed to `writeFile` / `mkdir` at create time. |
| `chown` | — | No ownership model. |
| `utimes` | — | `mtime` is managed by the VFS. |
| `cp` / `copyFile` | — | Read + write. |
| `rename` | | Read + write + delete. |
| `rename` | `rename` | One transaction; replaces a destination of the same kind. |
| `realpath` | — | Paths are already canonical. |
| `symlink` / `readlink` | — | Not on the public surface; see note below. |
| `symlink` / `readlink` | `symlink` / `readlink` | Same argument order as Node. Targets are stored verbatim and may dangle. |
| `watch` | — | Low-level primitive in `fs/watch.ts` (`createWatcher`, `createWatchAsyncIterable`, `WatchHandle`, `WatchOptions`); not exposed on the `WorkspaceFilesystem` class. |
| `open` / `FileHandle` | — | Use streams instead. |
| `glob` | `find` | Limited glob support (`*`, `**`, `**/`, and `?`). |
| `glob` | `find` | Limited glob support (`*`, `**`, `**/`, and `?`), plus `exclude` for pruning subtrees. |
| — | `grep` | Not in `node:fs`; literal by default, with optional regular expressions. |
| — | `find` | Recursive directory walk with an optional glob, relative-rooted. |
| — | `ls` | Flat list of file paths under a directory (segment-aware). |

### Note: symlinks

Symlinks exist as an **internal primitive** used by the `node:vfs`
adapter — the schema supports a `'symlink'` node type with a
`link_target`, and the resolver in `fs/resolve.ts` follows them with a
40-hop cap (throws `ELOOP` on overflow). They are **not** part of the
public `WorkspaceFilesystem` surface: there are no `fs.symlink` or
`fs.readlink` methods on `Workspace.fs`, and callers should treat all
visible paths as if they pointed straight at real files.
### Note: symbolic links

Symbolic links are part of the public surface. The schema carries a
`'symlink'` node type with a `link_target`, the resolver in
`fs/resolve.ts` follows them with a 40-hop cap (throws `ELOOP` on
overflow), and `Workspace.fs` exposes `symlink`, `readlink`, and
`lstat` on top of that. `WorkspaceFilesystemStub` mirrors all three
across the Workers RPC boundary, which is how the Dynamic Worker
filesystem adapters provide the `node:fs` behavior a shell expects from
`ln -s`, `readlink`, and `test -L`.

Two rules cover the whole surface. Intermediate segments are always
followed, so a link to a directory behaves like the directory for every
method, `mkdir` included. A trailing link is followed by everything
except `lstat` and `readlink`, which are the two methods whose purpose
is to describe the link itself.
7 changes: 5 additions & 2 deletions docs/09_tool_interface.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,15 +166,18 @@ Entries are in name order. A non-final page includes `nextOffset`; pass it as th

```ts
{
path?: string; // default /workspace
path?: string; // default /workspace
pattern: string;
limit?: number; // default 200, maximum 1000
exclude?: string[];
limit?: number; // default 200, maximum 1000
offset?: number;
}
```

The pattern is relative to `path`. `*` stays within one path segment, `**` crosses directories, and `?` matches one non-separator character. Results contain `path` and `type`; a non-final page includes `nextOffset`. Pagination reaches `workspace.fs.find`, which walks directory children in fixed-size pages and stops after collecting the requested page instead of materializing every match.

`exclude` takes globs of the same shape, matched against the same relative path, and beats the inclusion pattern. An excluded directory is pruned rather than filtered, so `exclude: ["node_modules", "node_modules/**"]` keeps the walk out of a package tree instead of walking it and discarding the results.

## `grep`

```ts
Expand Down
4 changes: 3 additions & 1 deletion docs/12_worker_backend.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,9 @@ policy. The backend does not own the external runtime's lifecycle.
- **No hard links, no `utimes`.** The adapter throws `ENOSYS` on
`link` (the store has no hard-link model) and no-ops on
`utimes` (no atime column). `chmod`, `symlink`, `readlink`,
and `lstat` all work end-to-end against the DO's store.
`lstat`, and `rename` all work end-to-end against the DO's
store, so `mv` moves an entry in one operation instead of
copying and then deleting it.
- **No cross-request reattach.** `ShellWorker.getExec` always
returns ENOENT; `killExec` is a no-op. Each exec is scoped to
its own call. The previous in-isolate event log shape didn't
Expand Down
28 changes: 28 additions & 0 deletions packages/computer/src/backends/worker-shell/adapter.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,34 @@ describe("WorkspaceFsAdapter — composites", () => {
expect(await workspace.fs.readFile("/dst", "utf8")).toBe("hello");
await expect(workspace.fs.stat("/src")).rejects.toMatchObject({ code: "ENOENT" });
});

it("mv moves through the store's rename rather than copy and delete", async () => {
await workspace.fs.writeFile("/src", "hello");
const rename = vi.spyOn(stub, "rename");
const writeFile = vi.spyOn(stub, "writeFile");
await adapter.mv("/src", "/dst");
expect(rename).toHaveBeenCalledWith("/src", "/dst");
expect(writeFile).not.toHaveBeenCalled();
});

it("mv moves a directory tree in one operation", async () => {
await workspace.fs.mkdir("/src/inner", { recursive: true });
await workspace.fs.writeFile("/src/inner/b", "b");
await adapter.mv("/src", "/dst");
expect(await workspace.fs.readFile("/dst/inner/b", "utf8")).toBe("b");
await expect(workspace.fs.stat("/src")).rejects.toMatchObject({ code: "ENOENT" });
});

it("mv falls back to copy and delete when rename cannot replace the destination", async () => {
await workspace.fs.mkdir("/src", { recursive: true });
await workspace.fs.writeFile("/src/a", "a");
await workspace.fs.mkdir("/dst", { recursive: true });
await workspace.fs.writeFile("/dst/keep", "keep");
await adapter.mv("/src", "/dst");
expect(await workspace.fs.readFile("/dst/a", "utf8")).toBe("a");
expect(await workspace.fs.readFile("/dst/keep", "utf8")).toBe("keep");
await expect(workspace.fs.stat("/src")).rejects.toMatchObject({ code: "ENOENT" });
});
});

describe("WorkspaceFsAdapter — pure utilities", () => {
Expand Down
22 changes: 16 additions & 6 deletions packages/computer/src/backends/worker-shell/adapter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
//
// The adapter is a thin façade. Operations that map one-for-one
// (writeFile, readdir, mkdir, rm, chmod, symlink, readlink, stat,
// lstat) forward directly. Operations the stub doesn't expose —
// appendFile, cp, mv, exists — synthesize from the available
// lstat, rename) forward directly. Operations the stub doesn't
// expose — appendFile, cp — synthesize from the available
// primitives. Hard links and utimes aren't supported: link throws
// ENOSYS so a script that depends on them fails loudly; utimes
// is a documented no-op because the store has no atime column.
Expand Down Expand Up @@ -50,6 +50,7 @@ export interface WorkspaceFs {
rm(path: string, options?: RmOptions): Promise<void>;
chmod(path: string, mode: number): Promise<void>;
symlink(target: string, path: string): Promise<void>;
rename(oldPath: string, newPath: string): Promise<void>;
}

// Matches the subset of just-bash's IFileSystem the adapter
Expand Down Expand Up @@ -213,10 +214,19 @@ export class WorkspaceFsAdapter {
}

async mv(src: string, dest: string): Promise<void> {
// The store doesn't have a native rename today, so model mv as
// copy+delete. POSIX mv is atomic when src and dest live on
// the same filesystem; this approach isn't, but it matches
// what just-bash's other adapters do.
// The store renames in one transaction, so an interrupted move can
// no longer leave the bytes at both paths or a directory half
// copied. A destination that rename refuses to replace — a
// non-empty directory, or a directory and a non-directory in
// either order — still falls back to copy-then-delete, which is
// what the shell's own `mv` expects when it merges a tree.
try {
await this.#fs.rename(src, dest);
return;
} catch (err) {
const code = (err as { code?: string }).code;
if (code !== "ENOTEMPTY" && code !== "EISDIR" && code !== "ENOTDIR") throw err;
}
await this.cp(src, dest, { recursive: true });
await this.#fs.rm(src, { recursive: true });
}
Expand Down
Loading
Loading