fix(remote-changes): recover file moves across outcome commit gaps

docs/remote-change-application.md

@@ -55,5 +55,103 @@ The local feed retains observations made during an uncommitted creation separate
 
 The official SQLite store upgrades to schema 5 to prevent older versions from ignoring this recovery protocol. Do not downgrade an upgraded database. Custom transactional stores remain supported using existing repositories, but all accessing runtimes must use the new protocol version together. Creation transactions contain a synchronous placeholder creation, not hydration or application callbacks.
 
+## Recovering remote file moves (CF-007)
+
+File moves and case-only file renames submitted in an immutable batch now retain a versioned
+preparation before namespace mutation. Replay the **same batch**, including its ID, entry order,
+fingerprint, IDs, paths, opaque revisions, and cursor envelope. Keep the original
+`PreserveUnsynchronizedLocalContent`, `SuppressLocalEcho`, echo lifetime, and observation count.
+`MaximumEntries` may change; a resolver instance is not part of the persisted policy.
+
+The preparation belongs to the original store and root. It captures the original official item
+state, native placeholder identity separately from official revision, complete volume/root/file
+binding, creation/write times and length, and source/destination ancestor bindings and identities.
+The native rename uses the inspected object handle with replacement disabled. It requests no data
+access and performs no content reads, identity rewrite, hydration, or in-sync transition. Revisions
+are opaque: recovery also works when previous and incoming revisions are equal.
+Preparation and recovery hold source/destination ancestors with directory-list/read-attribute
+access and delete sharing disabled. These handles prevent replacement during native and durable
+work without enumerating children. Insufficient access or file-ID/USN support is an explicit error.
+
+Replay authenticates the preparation **before** applying ordinary stale-revision checks:
+
+| Durable/native boundary | Replay behavior |
+| --- | --- |
+| Preparation committed, original object still at source | Revalidate original conditions and issue the original rename. |
+| Original object at exact target, item projection not committed | Verify original binding, identity, metadata, ancestry, and local journal; commit path/revision together without another rename. |
+| Item projection committed, original batch outcome missing | Revalidate the same object and projected state; commit the missing original outcome without another rename. |
+| Outcome committed, reply lost | Read the original committed prefix and return `AlreadyApplied`. |
+| Object, ancestry, state, or local work changed | Return `Failed` with `Error`, `RequiresRetry=true`, and the unchanged safe prefix. Retain evidence for reconciliation; do not manufacture a stale conflict. |
+
+Mismatched policy, store/root, or corrupted preparation is rejected with `InvalidDataException`
+before normalizing progress. A changed batch fingerprint is rejected with `ArgumentException`.
+
+Recovered entries return `Applied` only after their outcome commits. `AppliedEntryCount` counts
+committed outcomes, **including legitimate conflicts**. A fully processed batch has `Status=Applied`
+and `RequiresRetry=false`; callers still inspect `Entries` and `ConflictIds`. An unchanged completed
+prefix returns `AlreadyApplied` on replay, using the existing batch semantics; this status certifies
+the recorded outcome, not that every historical entry was conflict-free. `SafeCursor` advances only
+with that durable prefix. CfSharp never publishes an application's separate Adapter cursor.
+
+The projected preparation also captures the post-rename native USN after closing rename-overlapping
+handles, so NTFS's accumulated close record is included. Replay rejects subsequent native changes.
+`TrackAll` can clear in-sync because of the rename itself; recovery preserves that state rather than
+marking the object clean. Before a projected USN exists, recovery requires the original in-sync
+condition under the conservative policy. Thus an early interrupted rename that clears in-sync
+remains explicitly unresolved; roots using `PreserveForSyncEngine` can preserve that proof boundary.
+The conservative policy rejects nonzero modified content ranges; an explicit force policy retains
+its original dirty content and pending journal without acknowledging or marking them clean.
+
+The path/revision projection and echo suppression commit together. Preparation cleanup and original
+outcome/progress commit together. Any pre-commit fault leaves evidence; a late exception after commit
+is resolved by rereading durable state on the next call. Entry storage failures retain a failed,
+retryable prefix and their original exception in `Error`. Outcome storage failures propagate, so a
+caller must replay after reopening rather than assume either rollback or success. Storage errors
+do not acquire native HRESULTs. Native rename errors preserve their actual HRESULT.
+
+Cancellation before native mutation leaves an unchanged object and any committed preparation.
+After native success, item projection finishes without cancellation under the admitted owner
+lifetime; cancellation may leave batch outcome publication for replay. One owner's calls serialize,
+and disposal waits for admitted work, including recovery transactions. Use one active SDK owner
+per root/store. Reopening that pair is supported; simultaneous owners are not a distributed lock.
+No conflict resolver or content-provider callback runs inside the rename/projection transaction.
+
+Local feed observations at either pending path are durably retained and require a full rescan.
+Replay first, reconcile the root, then acknowledge the rescan. Acknowledgement cannot discard a
+pending move. Existing journal rows are never acknowledged by recovery; a changed journal or dirty
+object leaves the operation unresolved. An explicit force policy may retain its original pending
+operations while moving, but recovery rejects later operations. This fix conservatively refuses
+to settle a move after later edits instead of clearing or overwriting them.
+
+### Compatibility and limits
+
+The official SQLite provider upgrades schemas 0–6 in place to **schema 7**, retaining existing
+repositories and enforcing foreign keys. Older providers reject that version before ignoring
+pending file-move evidence. Use matching core/provider packages; never downgrade the schema marker.
+Custom stores use the existing public repositories and must atomically isolate transactions across
+all repositories, preserve opaque checkpoints and their ordinal prefix listing, and fence older
+runtimes themselves. Stores lacking those contracts cannot support this protocol. Only metadata
+queries and synchronous rename occur inside the native/projection transaction.
+
+This recovery protocol covers **files** submitted through `ApplyRemoteChangesAsync`. Directory
+movement keeps its existing subtree/proof APIs; this fix does not claim automatic recovery of a
+directory's missing remote-batch outcome. It does not add a public preparation token or infer an
+outcome from matching present-day paths/revisions. Direct lower-level file moves and explicit
+standalone conflict resolutions keep their existing contracts.
+
+Preview.5 file gaps without original preparations remain conservative conflicts. Existing durable
+conflicts are preserved for review, not silently reclassified during upgrade. Preserve the original
+batch, native object, local work, and application evidence for operator reconciliation. Supported
+conflict decisions remain explicit policy, not proof that a historical move succeeded. If proof
+cannot be restored, retire that store only after application reconciliation and rebuild state in a
+new caller-selected store through supported APIs; do not edit private checkpoints or mark a batch
+complete manually. This can require reprocessing earlier work and does not complete product cursors.
+
+Native regression tests cover ordinary/case-only moves with SQLite reopen, equal revisions,
+preparation/projection/outcome interruptions, ambiguous commits, repeated recovery faults,
+replacement/parent changes, dirty state, and pending edits. In-process exceptions followed by
+normal disposal are distinct from power loss or hard process termination. Architecture/ordinary-user
+results and remaining acceptance-matrix cases must be reported with their actual executed evidence.
+
 
 

docs/state-store-sqlite.md

@@ -49,13 +49,15 @@ Pages own no reader or transaction and do not acknowledge operations or update i
 The legacy `ICloudOperationJournal` contract is unchanged: custom stores may opt into paging on
 their repository, preserving monotonic enqueue sequences and transactional read semantics.
 
-### Schema 6 recovery fence
+### Schema 7 recovery fence
 
 Schema 6 preserves the existing table layout and data while fencing directory object provenance,
 move preparation, and completion-receipt recovery. It includes schema 5's remote placeholder
 creation and observation-reconciliation fence. Versions 0 through 5 upgrade in place, retaining
 item IDs, revisions, tombstones, journal sequences and payloads, suppressions, and checkpoints.
-Older SQLite packages reject version 6 before accessing its recovery metadata. Do not downgrade
+Schema 7 additionally fences original remote-file-move preparations and feed observation recovery.
+Versions 0 through 6 upgrade in place without fabricating proof for legacy interrupted file moves.
+Older SQLite packages reject version 7 before accessing its recovery metadata. Do not downgrade
 the schema number manually. Back up state before upgrading and use matching core and SQLite packages.
 
 Custom stores must enforce the same protocol boundary when opening persistent state with older

src/CfSharp.Native/Internal/WindowsFileMetadata.cs

@@ -25,6 +25,59 @@ internal static SafeFileHandle Open(string path, bool preventDelete = false)
         return handle;
     }
 
+    internal static SafeFileHandle OpenForRename(string path)
+    {
+        // DELETE addresses rename permission; no data access is requested, so an online-only
+        // placeholder stays unhydrated. Denying delete sharing excludes competing rename/delete
+        // owners while this handle itself retains DELETE access for the inspected-object rename.
+        SafeFileHandle handle = CreateFile(path, 0x10080, 3, 0, 3, 0x02200000, 0);
+        if (handle.IsInvalid)
+        {
+            int error = Marshal.GetLastPInvokeError();
+            handle.Dispose();
+            throw new Win32Exception(error);
+        }
+
+        return handle;
+    }
+
+    internal static SafeFileHandle OpenDirectoryGuard(string path)
+    {
+        // FILE_LIST_DIRECTORY makes no-delete sharing effective for a directory. An
+        // attribute-only directory opener does not prevent all ancestor rename races.
+        // Opening this access performs no enumeration and never requests file content.
+        SafeFileHandle handle = CreateFile(path, 0x81, 3, 0, 3, 0x02200000, 0);
+        if (handle.IsInvalid)
+        {
+            int error = Marshal.GetLastPInvokeError();
+            handle.Dispose();
+            throw new Win32Exception(error);
+        }
+
+        return handle;
+    }
+
+    internal static void Rename(nint handle, string destination)
+    {
+        // FILE_RENAME_INFO has a DWORD-sized union, a naturally aligned HANDLE, a byte
+        // length, and trailing UTF-16 characters. RootDirectory and replacement stay zero.
+        // Use the ABI field offset, not sizeof(header), which includes trailing padding.
+        // https://learn.microsoft.com/windows/win32/api/winbase/ns-winbase-file_rename_info
+        FileRenameInformation layout = default;
+        int offset = checked((int)((byte*)&layout.FileName - (byte*)&layout));
+        int nameBytes = checked(destination.Length * sizeof(char));
+        byte[] buffer = new byte[checked(offset + nameBytes + sizeof(char))];
+        fixed (byte* pointer = buffer)
+        {
+            ((FileRenameInformation*)pointer)->FileNameLength = checked((uint)nameBytes);
+            destination.AsSpan().CopyTo(new Span<char>(pointer + offset, destination.Length));
+            if (!SetFileInformationByHandle(handle, 3, pointer, checked((uint)buffer.Length)))
+            {
+                throw new Win32Exception(Marshal.GetLastPInvokeError());
+            }
+        }
+    }
+
     internal static SafeFileHandle OpenLocalAccess(string path, bool directory)
     {
         // A single file object owns both data exclusion and security access. A separate
@@ -123,6 +176,7 @@ internal static NativeFileMetadata Read(nint handle, nint protectedHandle = 0)
             attributes.Attributes, attributes.ReparseTag);
         byte[] identity = [];
         long onDisk = 0;
+        long modified = 0;
         bool inSync = false;
         if (state != CfPlaceholderState.Invalid && state.HasFlag(CfPlaceholderState.Placeholder))
         {
@@ -150,13 +204,14 @@ internal static NativeFileMetadata Read(nint handle, nint protectedHandle = 0)
 
                 identity = new ReadOnlySpan<byte>(info->FileIdentity, (int)info->FileIdentityLength).ToArray();
                 onDisk = info->OnDiskDataSize;
+                modified = info->ModifiedDataSize;
                 inSync = info->InSyncState == CfInSyncState.InSync;
             }
         }
 
         return new NativeFileMetadata(ReadIdentity(handle), standard.EndOfFile,
             standard.NumberOfLinks, standard.DeletePending != 0, standard.Directory != 0,
-            attributes.Attributes, attributes.ReparseTag, state, onDisk, inSync, identity);
+            attributes.Attributes, attributes.ReparseTag, state, onDisk, inSync, identity, modified);
     }
 
     private static void Query(nint handle, int informationClass, void* value, uint size)
@@ -203,6 +258,15 @@ internal struct FileAttributeTagInfo
         internal uint ReparseTag;
     }
 
+    [StructLayout(LayoutKind.Sequential)]
+    internal struct FileRenameInformation
+    {
+        internal uint Flags;
+        internal nint RootDirectory;
+        internal uint FileNameLength;
+        internal char FileName;
+    }
+
     [LibraryImport("kernel32.dll", EntryPoint = "CreateFileW", SetLastError = true, StringMarshalling = StringMarshalling.Utf16)]
     [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvStdcall) })]
     private static partial SafeFileHandle CreateFile(string path, uint access, uint share,
@@ -213,14 +277,19 @@ private static partial SafeFileHandle CreateFile(string path, uint access, uint
     [return: MarshalAs(UnmanagedType.Bool)]
     private static partial bool GetFileInformationByHandleEx(nint handle, int informationClass, void* value, uint size);
 
+    [LibraryImport("kernel32.dll", SetLastError = true)]
+    [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvStdcall) })]
+    [return: MarshalAs(UnmanagedType.Bool)]
+    private static partial bool SetFileInformationByHandle(nint handle, int informationClass, void* value, uint size);
+
     [LibraryImport("kernel32.dll", EntryPoint = "GetFinalPathNameByHandleW", SetLastError = true)]
     [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvStdcall) })]
     private static partial uint GetFinalPathNameByHandle(nint handle, char* path, uint capacity, uint flags);
 }
 
 internal sealed record NativeFileMetadata(WindowsFileMetadata.FileIdentity Identity, long Length,
     uint Links, bool DeletePending, bool Directory, uint Attributes, uint ReparseTag,
-    CfPlaceholderState PlaceholderState, long OnDiskDataSize, bool InSync, byte[] PlaceholderIdentity);
+    CfPlaceholderState PlaceholderState, long OnDiskDataSize, bool InSync, byte[] PlaceholderIdentity, long ModifiedDataSize);
 
 internal sealed class NativeFileException : IOException
 {

src/CfSharp.Storage.Sqlite/SqliteCloudStateStoreFactory.cs

@@ -777,7 +777,7 @@ private struct FileAttributeTagInfo
 
 internal static class SqliteSchema
 {
-    internal const int CurrentVersion = 6;
+    internal const int CurrentVersion = 7;
     internal const string PathCollationName = "CFSHARP_UNICODE_NOCASE";
 
     internal static async Task InitializeAsync(
@@ -879,6 +879,16 @@ await MigrateVersionFourAsync(connection, cancellationToken)
                 await upgrade.ExecuteNonQueryAsync(cancellationToken).ConfigureAwait(false);
             }
 
+            if (version < 7)
+            {
+                // File-move preparations bridge native rename, item projection, and original
+                // batch outcome commits using checkpoints. Fence older runtimes before they
+                // can ignore pending evidence or deliver its observations as local uploads.
+                await using SqliteCommand upgrade = connection.CreateCommand();
+                upgrade.CommandText = "UPDATE cfsharp_schema SET version = 7 WHERE singleton = 1;";
+                await upgrade.ExecuteNonQueryAsync(cancellationToken).ConfigureAwait(false);
+            }
+
             await ValidateCurrentVersionAsync(
                 connection,
                 databasePath,