Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
70dfdf4
feat(persistence): add optional bounded journal paging
Quaternion8192 Oct 7, 2026
913ba7c
feat(local-changes): expose bounded typed change scans
Quaternion8192 Oct 7, 2026
47b6048
fix(local-changes): preserve reconciliation fences during scans
Quaternion8192 Oct 7, 2026
d7b1b7a
test(local-changes): cover native backlog scans across roots
Quaternion8192 Oct 7, 2026
7c350a4
refactor(items): share transactional subtree path projection
Quaternion8192 Oct 7, 2026
74b0e9d
feat(items): expose complete native bindings for directories
Quaternion8192 Oct 7, 2026
82b483a
fix(sqlite): fence durable directory recovery metadata
Quaternion8192 Oct 7, 2026
fa600f7
chore: clarify MirrorPulse Team code ownership
Quaternion8192 Oct 7, 2026
f050cfa
feat(items): prepare durable directory move proofs
Quaternion8192 Oct 7, 2026
0bc8edf
fix(items): retain directory bindings during identity projection
Quaternion8192 Oct 7, 2026
444b29d
fix(items): reconcile verified external directory moves
Quaternion8192 Oct 7, 2026
d50e096
fix(items): verify directory move replay with retained proofs
Quaternion8192 Oct 7, 2026
56454c4
fix(local-changes): project verified directory rename subtrees atomic…
Quaternion8192 Oct 7, 2026
ead18f5
test(items): cover directory reconciliation object races
Quaternion8192 Oct 7, 2026
1a4fa55
test(items): verify directory move recovery across process crashes
Quaternion8192 Oct 7, 2026
0804a7e
test(compat): preserve preview3 state and public behavior
Quaternion8192 Oct 7, 2026
b99c5e5
docs: explain bounded scans and proven directory recovery
Quaternion8192 Oct 7, 2026
90df9be
build(release): verify preview capabilities in isolated consumers
Quaternion8192 Oct 7, 2026
97e2443
fix(items): preserve operations beyond bounded directory evidence
Quaternion8192 Oct 7, 2026
78da2e8
test(sqlite): drain and reap abrupt crash subprocesses
Quaternion8192 Oct 7, 2026
74e2d9b
test(local-changes): verify scans recover from SQLite query faults
Quaternion8192 Oct 8, 2026
20a8b96
test(local-changes): await durable echo processing deterministically
Quaternion8192 Oct 8, 2026
4b08f32
fix(lifecycle): drain owned rename notifications before shutdown
Quaternion8192 Oct 8, 2026
42430f1
fix(namespace): prepare fresh intents for repeated directory moves
Quaternion8192 Oct 8, 2026
774e691
test(sqlite): observe durable boundary before replay restart
Quaternion8192 Oct 8, 2026
9742b84
fix(namespace): finish admitted directory moves during shutdown
Quaternion8192 Oct 8, 2026
2877acd
fix(local-changes): fence batches across persisted loss signals
Quaternion8192 Oct 8, 2026
828b590
fix(local-changes): capture durable loss boundaries for feed reads
Quaternion8192 Oct 8, 2026
ec0f005
test(integration): verify reference protection across sharing conflicts
Quaternion8192 Oct 8, 2026
05b3056
fix(local-changes): detach source consumption from caller context
Quaternion8192 Oct 8, 2026
7b4c33a
test(local-changes): detach persistence barriers from caller context
Quaternion8192 Oct 8, 2026
a37ed4e
test(local-changes): scope cancellation budgets to verification phases
Quaternion8192 Oct 8, 2026
b9926ea
perf(persistence): refresh directory membership once per batch
Quaternion8192 Oct 8, 2026
c0aa286
fix(file-system): use native parent spelling for directory moves
Quaternion8192 Oct 8, 2026
8c90987
fix(persistence): refresh ancestor membership after directory moves
Quaternion8192 Oct 8, 2026
57392dd
fix(persistence): refresh ancestor membership after file moves
Quaternion8192 Oct 8, 2026
cb4c6bc
fix(local-changes): refresh membership for discovered items
Quaternion8192 Oct 8, 2026
019a39d
fix(persistence): relocate live provenance under ordinary moves
Quaternion8192 Oct 8, 2026
170442f
test(items): support checkpoint reads in commit fault fixture
Quaternion8192 Oct 8, 2026
8ecab28
fix(file-system): honor cancellation after move preparation
Quaternion8192 Oct 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Default owner for repository changes. Replace with an organization team when
# a dedicated MirrorPulse maintainers team is created.
# MirrorPulse Team maintains the project. Keep the individual GitHub owner until
# a visible organization team is available; team entries require @org/team-slug.
* @Quaternion8192
22 changes: 22 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,21 @@ jobs:
- name: Pack SQLite state provider
run: dotnet pack src/CfSharp.Storage.Sqlite/CfSharp.Storage.Sqlite.csproj --configuration Release --no-build --output artifacts/packages -p:Version=0.0.0-ci.${{ github.run_number }} -p:PackageVersion=0.0.0-ci.${{ github.run_number }}

- name: Verify isolated package capabilities on x64
shell: pwsh
run: ./eng/prepare-preview.ps1 -Version '0.0.0-preview.${{ github.run_number }}' -Channel Preview -OutputDirectory artifacts/package-capabilities

- name: Upload x64 package capability evidence
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: package-capabilities-x64
path: |
artifacts/package-capabilities/**/manifest.json
artifacts/package-capabilities/**/consumer-smoke/packages.lock.json
artifacts/package-capabilities/**/packages/*
if-no-files-found: ignore

- name: Upload test results
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
Expand Down Expand Up @@ -146,6 +161,10 @@ jobs:
- name: Test ARM64 solution
run: dotnet test CfSharp.sln --configuration Release --no-build --no-restore --disable-build-servers -m:1 --filter "Category!=AbiProbe&Category!=ApiBaseline&Category!=LongSoak" --logger "trx;LogFileName=arm64-test-results.trx" --blame-hang-timeout 2m

- name: Verify isolated package capabilities on ARM64
shell: pwsh
run: ./eng/prepare-preview.ps1 -Version '0.0.0-preview.${{ github.run_number }}' -Channel Preview -OutputDirectory artifacts/package-capabilities

- name: Upload ARM64 test results
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
Expand All @@ -154,6 +173,9 @@ jobs:
path: |
**/TestResults/**
artifacts/runtime-boundaries/**
artifacts/package-capabilities/**/manifest.json
artifacts/package-capabilities/**/consumer-smoke/packages.lock.json
artifacts/package-capabilities/**/packages/*
if-no-files-found: error

native-abi-x64:
Expand Down
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,11 @@ The dependency direction is one way: `CfSharp.Native` is the interop layer, `CfS
the safe domain API, and the SQLite package depends on `CfSharp`. Remote transport, authentication,
content bytes, and business conflict policy remain application-owned.

The source API supports [finite typed scans of pending local changes](docs/local-change-feed.md)
without acknowledging a deferred head, and [verified directory move recovery](docs/placeholders.md)
that preserves subtree identities, current revisions, and pending journal operations. Applications
retain their own dispatch policy and durable move intents.

## Quick start

### Requirements
Expand Down
99 changes: 99 additions & 0 deletions docs/local-change-feed.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,13 @@
The local change feed turns Windows directory notifications into an explicit, bounded application
queue. It is a coordination aid, not a lossless replacement for reconciliation.

Disposal stops notification production and drains accepted events while the state store remains
open, including queue-tail renames during owner shutdown. If draining times out or fails, the feed
persists a full-rescan marker before canceling its worker. Persisting that fence can wait for an
active store transaction beyond the worker's `ShutdownTimeout`. If fence persistence fails, disposal
reports the failure, leaves the worker uncanceled, and retains the owner's store for a disposal retry;
the retry must complete the drain and persist the fence before the store is closed.

## Start one feed per sync root

```csharp
Expand All @@ -27,13 +34,105 @@ else
The feed normalizes paths relative to the sync root, pairs renames, and persists the ordered
journal with its watcher checkpoint in one transaction. Acknowledgement advances durable progress
only after the application has accepted the batch.
Background notification processing does not require pumping the synchronization context that
started the feed; loss persistence and shutdown drain continue independently of that context.

## Scan a finite backlog without acknowledging its head

Use typed scans when some pending changes cannot currently be dispatched. A scan captures the
highest pending sequence once; each page uses an exclusive lower bound and that inclusive upper
bound. Reads never ACK a deferred row, wait for a new event, or extend the scan with later writes.

```csharp
CloudLocalChangeScan scan = await feed.BeginScanAsync(cancellationToken);
long after = 0;
while (true)
{
CloudLocalChangePage page = await feed.ReadPageAsync(scan, after, 4, cancellationToken);
if (page.RequiresFullRescan)
{
// Stop dispatch. Replay pending remote creations, reconcile, explicitly acknowledge
// the rescan, then begin a new scan from zero.
break;
}

IReadOnlyList<Guid> acceptedIds = await DispatchEligibleChangesAsync(
page.Changes, cancellationToken);
// This application method returns only IDs whose meaning it durably accepted.
await feed.AcknowledgeAsync(acceptedIds, cancellationToken);
after = page.LastScannedSequence;
if (!page.HasMore)
{
break;
}
}
```

The page limit is 1 through 4096 and is independent of the legacy `BatchSize` (default 64).
Use `LastScannedSequence` directly, without adding one. ACKs between pages can leave sequence
gaps and can make the next page empty. Empty terminal pages return `ThroughSequence` immediately.
Restarting the feed or runtime invalidates old scans; create a new scan from zero against the same
durable store. `StoreScope` identifies the running feed lifetime, not a persisted database identity.
Scans and pages retain no transaction, reader, native handle, or file content.

Custom stores opt in by implementing `ICloudOperationJournalPaging` on `Operations`; unsupported
stores throw `NotSupportedException` for scans and retain the existing batch API. Invalid or foreign
journal payloads throw `CloudLocalChangeJournalException` with the original operation ID and
sequence, leaving the row pending. Consumers do not need private payload parsing or a copied queue.
Scheduling across roots and preserving dependencies between changes remain application concerns.
Keep a dependency view across pages before dispatching dependent moves/deletes. A deferred root
does not have to be ACKed to reach an independent later root. Completing this finite scan does not
accept deferred operations: begin the next cycle at zero, preserving those original IDs. Later writes
belong to the next captured boundary. Multiple consumers must coordinate dispatch and ACK ownership.

## Rescan is a normal state

A buffer overflow, watcher error, or ambiguous rename is reported as `RequiresFullRescan`. Stop
assuming that notifications are complete, enumerate the materialized local tree, reconcile it, and
acknowledge the rescan explicitly. Recursive operations never follow links or remote children.

Scans and legacy batches both withhold dispatchable changes while recovery is required, including
when a pending backlog already exists. Pending remote creation intents and their uncertain watcher
observations are checked in the same transaction as each page. Replay creation recovery before
acknowledging reconciliation. A loss received while the watcher worker waits for the store also
blocks pages immediately. Reading a page never clears recovery markers.

Legacy batch reads and scan capture retain loss already pending at entry or received during
the read, even when the worker commits the rescan marker before the reading transaction returns.
The original journal remains pending until explicitly acknowledged after reconciliation.

An overflow/error or a full-rescan acknowledgement invalidates previously captured scans even
after the durable marker is cleared. Begin a new scan after recovery. If a new loss races
`AcknowledgeFullRescanAsync`, it either rejects acknowledgement with `InvalidOperationException`
or remains observable as a new rescan condition; finish reconciliation before retrying.

## Directory rename projection

Directory renames use retained library-captured native bindings and historical membership to
validate the target. Paths of the complete known subtree, its immutable move receipt, the new
formal Move operation, and the feed checkpoint commit together. Current revisions, tombstones,
and pending operations keep their original values. An old operation's paths and observation time
describe its historical event; projecting today's paths does not rewrite or acknowledge it.

Missing evidence, an unrelated target, unknown membership, or a delayed rename whose object has
already moved again retains an unresolved formal Move with a nullable item association and requires
full reconciliation. The feed never deletes an unrelated destination row to make a rename fit.
Explicit full-rescan acknowledgement clears the reconciliation fence while leaving those original
operations pending. Dispatch and acknowledge them only after their meaning has been accepted.
Native changes racing the transaction can require reconciliation even after its projection committed;
the committed operation retains its original ID and sequence. Native notifications remain advisory.

Renaming a directory, undoing that rename, and repeating the same path pair uses a new preparation
from the latest retained pre-rename provenance, including after runtime/store restart and for
case-only renames. The mutable path-pair index points to that new preparation; earlier proofs,
membership manifests, receipts, and journal rows remain unchanged. Duplicate completed observations
still replay the original preparation. An explicitly supplied historical proof cannot project a
later namespace transition using its old receipt.

For an existing partial projection, use the original source directory's `MoveToAsync` or the
prepared-proof recovery described in [placeholders](placeholders.md). A matching path or remote ID
alone cannot establish historical native ownership.

## Provider echoes

Provider-originated writes can be wrapped by `SuppressProviderEchoAsync` so they do not become
Expand Down
Loading
Loading