Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 97 additions & 4 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ members = [
"src/wasix/runtime/engine/crates/wasmer-wasix-types",
"src/wasix/runtime/engine/crates/wasmer-journal",
"src/wasix/runtime/engine/crates/wasmer-wasix",
"src/wasix/runtime/engine/crates/virtual-fs",
"src/wasix/runtime/engine/crates/v8-windows-x64-msvc",

]
Expand Down
44 changes: 40 additions & 4 deletions src/docs/architecture/database-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,32 @@ PostgreSQL backend; its SDK-owned temporary directory must remain until that
backend's physical lifetime ends. It is not durable storage and the operating
system may reclaim it after process exit.

## Synchronization

Persistent storage selects where files live, not whether every commit is forced
to disk. Embedded Native and WASIX default PostgreSQL's `fsync` to `off`.
Caller startup settings take precedence over saved PostgreSQL configuration,
which takes precedence over this default. No SDK durability flag or environment
variable duplicates that control. Full native servers and the separate WASIX
Postmaster retain PostgreSQL's `fsync=on` default.

For disk synchronization, use the existing startup GUC (`fsync: 'on'` in
TypeScript or `.startup_guc("fsync", "on")` in Rust). With it off, a machine or
OS crash can lose committed data or corrupt the cluster. Turning it on later
does not retroactively make earlier unsynchronized writes safe; follow
PostgreSQL's procedure when changing a running database's durability policy.
WASIX host directories distinguish ordinary write flushing from explicit
`fdatasync` (data) and `fsync` (data and metadata); errors from explicit requests
are not suppressed by the default. Initial installation and physical restore
keep their checked publication barriers, independently of query settings.
Host directory barriers are checked on Unix; Windows helpers flush regular
files but do not implement a POSIX directory-sync barrier.

Browser providers still complete their existing publication/flush boundary
before returning. IndexedDB's transaction guarantees and OPFS's per-file flushes
are different contracts; `fsync=on` alone does not make either a POSIX disk.
Memory storage has no durable medium to synchronize.

## SDK spellings

Each language keeps its native conventions instead of importing a cross-language
Expand Down Expand Up @@ -98,6 +124,9 @@ staging generation; it cannot be mistaken for an existing database.
Restore uses `destination`, not `root`. A restore destination is an external
filesystem location receiving backup bytes; it is not an open database's
storage selector. The current API accepts only a new or empty destination.
If native restore fails its final durability check after publication, it retains
the populated destination and reports that outcome. Do not blindly retry or
delete that destination: an error does not always mean nothing was published.

## WASIX persistence contract

Expand All @@ -114,15 +143,22 @@ namespace never becomes ready. The initializing/ready phase and full-flush
selector are provider details, not public operations or configuration.
Applications that need a PostgreSQL checkpoint call `execute("CHECKPOINT")`;
that statement completes the same ordinary provider boundary as another
successful operation.
successful operation. With the embedded default `fsync=off`, PostgreSQL skips
its file-sync barriers, so `CHECKPOINT` alone does not guarantee that every
database file has been flushed to host storage. Set PostgreSQL's `fsync=on` to
request those barriers; the browser provider's persistence guarantees still
apply rather than native filesystem guarantees. A checkpoint may also replace
metadata and trigger a full namespace publication; applications must not rely
on that incidental flush as a substitute for PostgreSQL's file-sync barriers.

Each logical IndexedDB name owns an independent physical IndexedDB database.
Compatibility metadata and path rows change in one atomic read-write
transaction using the browser's default durability policy, so an aborted
publication leaves the prior generation current. OPFS stores an opaque logical
namespace over flat backing files. Its direct path honors PostgreSQL file flushes
and drains WAL at operation boundaries; checkpoint, close, and namespace
publication flush WAL before ordinary files and `global/pg_control`. Its
namespace over flat backing files. Its direct path honors explicit PostgreSQL
file flushes and normally drains only WAL at operation boundaries. Initialization,
clean close, and namespace publication flush WAL before ordinary files and
`global/pg_control`, independently of the PostgreSQL `fsync` setting. Its
portable path uses copy-on-write backing files and publishes the namespace
state last. The direct path keeps a private preopened fast-path reserve. A larger
creation burst is staged until the mandatory host boundary, where every staged
Expand Down
7 changes: 6 additions & 1 deletion src/examples/wasix/browser/opfs-crash-probe-worker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,12 @@ scope.addEventListener('message', (event: MessageEvent<ProbeRequest>) => {
});

async function prepareDurableState(name: string): Promise<void> {
database = await Oliphaunt.open({ seed: standardSeed, storage: opfs(name) });
// This probe requires checkpoint file barriers, not the embedded fsync=off default.
database = await Oliphaunt.open({
seed: standardSeed,
storage: opfs(name),
startupGUCs: { fsync: 'on' },
});
await database.queryRaw('CREATE TABLE opfs_crash_probe (answer integer NOT NULL)');
await database.queryRaw('INSERT INTO opfs_crash_probe VALUES (73)');
await database.queryRaw(`
Expand Down
5 changes: 3 additions & 2 deletions src/native/runtime/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,8 +113,9 @@ need different PostgreSQL settings do not need a new C ABI; they pass validated
`-c name=value` startup arguments through `OliphauntConfig.startup_args`. Later
arguments win, so SDKs and benchmark harnesses can apply concrete PostgreSQL
GUC overrides above the stable C boundary without inventing tuning profiles.
Direct startup defaults to `-F` (`fsync=off`). Persistent storage
alone does not provide crash safety: pass `-c fsync=on` when it is required.
Embedded startup defaults to `fsync=off`, without overriding saved PostgreSQL
configuration. Persistent storage alone does not provide crash safety: pass
`-c fsync=on` when disk synchronization is required. Full-server defaults are unchanged.

SDKs must hydrate PGDATA from a packaged cluster seed before calling
`oliphaunt_init`; the C boundary never runs `initdb` or initializes an empty
Expand Down
5 changes: 4 additions & 1 deletion src/native/runtime/include/oliphaunt.h
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,10 @@ typedef int32_t (*OliphauntStreamCallback)(void *context, const uint8_t *data, s
/* Returns zero and sets read_len (zero at EOF), or nonzero on failure.
* Read/write callbacks run synchronously and must not reenter this database.
* A failed backup sink still runs backup-mode cleanup before returning.
* Restore writes only to private staging until the entire archive validates. */
* Restore writes only to private staging until the entire archive validates.
* If final durability fails after publication, restore returns an error but
* retains the populated destination; retry must not overwrite it. This applies
* to both buffered and streamed restore. */
typedef int32_t (*OliphauntArchiveReadCallback)(
void *context, uint8_t *data, size_t capacity, size_t *read_len);
typedef struct OliphauntRestoreStreamOptions {
Expand Down
1 change: 1 addition & 0 deletions src/native/runtime/moon.yml
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,7 @@ tasks:
- /src/native/runtime/include/oliphaunt.h
- /src/native/runtime/src/liboliphaunt_internal.h
- /src/native/runtime/src/liboliphaunt_platform.h
- /src/native/runtime/src/liboliphaunt_archive.c
- /src/native/runtime/src/liboliphaunt_archive_tar.c
- /src/native/runtime/src/liboliphaunt_fs.c
- /src/native/runtime/src/liboliphaunt_error.c
Expand Down
1 change: 1 addition & 0 deletions src/native/runtime/postgres/series
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,4 @@ src/native/runtime/patches/postgresql-18.4/0019-liboliphaunt-link-windows-embedd
src/native/runtime/patches/postgresql-18.4/0020-liboliphaunt-enforce-embedded-signal-boundary.patch
src/native/runtime/patches/postgresql-18.4/0021-liboliphaunt-model-trusted-embedded-sessions.patch
src/native/runtime/patches/postgresql-18.4/0022-liboliphaunt-preserve-host-process-boundaries.patch
src/third-party/postgres/patches/common/embedded-fsync-default.patch
Loading
Loading