diff --git a/.github/workflows/native-benchmark.yml b/.github/workflows/native-benchmark.yml index aee8556..f9fd1af 100644 --- a/.github/workflows/native-benchmark.yml +++ b/.github/workflows/native-benchmark.yml @@ -4,6 +4,12 @@ on: schedule: - cron: '23 5 * * 3' workflow_dispatch: + inputs: + sizes_mib: + description: Comma-separated input sizes in MiB + required: true + default: '1,10,50' + type: string permissions: contents: read @@ -33,6 +39,7 @@ jobs: - name: Run deterministic benchmark env: BENCHMARK_OUTPUT: native-core-${{ runner.os }}.json + BENCHMARK_SIZES_MIB: ${{ inputs.sizes_mib || '1,10,50' }} run: node scripts/benchmark-native.mjs - name: Upload benchmark report diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml index e6f7a83..4d98ea7 100644 --- a/.github/workflows/pages.yml +++ b/.github/workflows/pages.yml @@ -58,7 +58,7 @@ jobs: uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6.0.0 - name: Upload GitHub Pages artifact - uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4 + uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0 with: path: site-dist @@ -71,4 +71,4 @@ jobs: steps: - name: Deploy to GitHub Pages id: deployment - uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4 + uses: actions/deploy-pages@cd2ce8fcbc39b97be8ca5fce6e763baed58fa128 # v5.0.0 diff --git a/README.md b/README.md index 78f84a9..d7e500c 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,7 @@

Documentation · Live Playground · + Binary Patch Toolkit · 中文说明 · npm

@@ -50,15 +51,18 @@ replaces live data. cap input/output sizes, and avoid exposing partial output files. - **No patch service required:** Web diffing and patching happen locally in the browser. +- **Inspect and prove compatibility:** read patch metadata and verify restored + bytes through the same API shape on native and Web. ## Platform overview -| | Android / iOS | React Native Web | -| -------------- | ------------------------------ | -------------------------------------------------- | -| Input | Absolute file paths | `ArrayBuffer`, typed arrays, `DataView`, or `Blob` | -| Basic API | `diff()` / `patch()` | `diffBytes()` / `patchBytes()` | -| Controlled API | `startDiff()` / `startPatch()` | `AbortSignal` and binary limits | -| Engine | Native C via JNI / ObjC++ | Same C core via WASM Worker | +| | Android / iOS | React Native Web | +| -------------- | -------------------------------------------- | -------------------------------------------------- | +| Input | Absolute file paths | `ArrayBuffer`, typed arrays, `DataView`, or `Blob` | +| Basic API | `diff()` / `patch()` | `diffBytes()` / `patchBytes()` | +| Controlled API | `startDiff()` / `startPatch()` | `AbortSignal` and binary limits | +| Verification | Paths via `inspectPatch()` / `verifyPatch()` | Binary values via the same APIs | +| Engine | Native C via JNI / ObjC++ | Same C core via WASM Worker | ## Install @@ -138,17 +142,44 @@ Web calls return a new `Uint8Array` and leave caller-owned buffers usable. Aborted operations reject with `EABORTED`; configured binary limits reject with `ERESOURCE`. +## Inspect and verify a patch + +Use `inspectPatch()` for a cheap structural check, then `verifyPatch()` to apply +into a temporary result and compare it with the expected target byte-for-byte: + +```ts +import { inspectPatch, verifyPatch } from 'react-native-bs-diff-patch'; + +// Android / iOS use paths. Web uses File, Blob, ArrayBuffer, or typed arrays. +const metadata = await inspectPatch(patchPath); +const result = await verifyPatch(oldPath, patchPath, expectedPath, { + maxInputBytes: 64 * 1024 * 1024, + maxOutputBytes: 128 * 1024 * 1024, +}); + +if (!metadata.valid || !result.verified) { + throw new Error('Patch compatibility check failed'); +} +``` + +The native verification output is temporary and always cleaned up. The Web +form accepts `oldFile`, `patchFile`, and `expectedFile` in the same argument +order. Structural validity is diagnostic; authenticate trusted hashes in your +update manifest before replacing live data. + ## API matrix -| API | Android | iOS | Web | -| ------------------------------------------ | ------- | --- | --- | -| `diff(oldPath, newPath, patchPath)` | Yes | Yes | No | -| `patch(oldPath, outputPath, patchPath)` | Yes | Yes | No | -| `startDiff(...)` / `startPatch(...)` | Yes | Yes | No | -| `diffBytes(oldData, newData, options?)` | No | No | Yes | -| `patchBytes(oldData, patchData, options?)` | No | No | Yes | -| Legacy architecture, while supplied by RN | Yes | Yes | N/A | -| New Architecture / TurboModule | Yes | Yes | N/A | +| API | Android | iOS | Web | +| --------------------------------------------- | ------- | --- | --- | +| `diff(oldPath, newPath, patchPath)` | Yes | Yes | No | +| `patch(oldPath, outputPath, patchPath)` | Yes | Yes | No | +| `startDiff(...)` / `startPatch(...)` | Yes | Yes | No | +| `diffBytes(oldData, newData, options?)` | No | No | Yes | +| `patchBytes(oldData, patchData, options?)` | No | No | Yes | +| `inspectPatch(path or binary, options?)` | Yes | Yes | Yes | +| `verifyPatch(old, patch, expected, options?)` | Yes | Yes | Yes | +| Legacy architecture, while supplied by RN | Yes | Yes | N/A | +| New Architecture / TurboModule | Yes | Yes | N/A | Unavailable platform APIs reject with `EUNSUPPORTED`; the package never silently switches to a different input model. diff --git a/README.zh-CN.md b/README.zh-CN.md index adcecec..3993231 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -21,6 +21,7 @@

中文文档 · 在线 Playground · + 二进制补丁工具箱 · English · npm

@@ -47,15 +48,17 @@ - **可控制高成本任务：** 原生 job 支持进度、协作式取消、输入/输出限制，并避免 暴露未完成的输出文件。 - **Web 无需补丁服务：** 差分和还原完全在浏览器本地执行。 +- **检查并证明兼容性：** 原生与 Web 使用相同 API 读取补丁元数据，并验证还原字节。 ## 平台概览 -| | Android / iOS | React Native Web | -| -------- | ------------------------------ | ----------------------------------------------- | -| 输入 | 绝对文件路径 | `ArrayBuffer`、TypedArray、`DataView` 或 `Blob` | -| 基础 API | `diff()` / `patch()` | `diffBytes()` / `patchBytes()` | -| 可控 API | `startDiff()` / `startPatch()` | `AbortSignal` 与二进制大小限制 | -| 执行核心 | JNI / ObjC++ 调用原生 C | WASM Worker 运行同一 C 核心 | +| | Android / iOS | React Native Web | +| -------- | ----------------------------------------- | ----------------------------------------------- | +| 输入 | 绝对文件路径 | `ArrayBuffer`、TypedArray、`DataView` 或 `Blob` | +| 基础 API | `diff()` / `patch()` | `diffBytes()` / `patchBytes()` | +| 可控 API | `startDiff()` / `startPatch()` | `AbortSignal` 与二进制大小限制 | +| 验证能力 | 路径版 `inspectPatch()` / `verifyPatch()` | 相同 API 的二进制输入 | +| 执行核心 | JNI / ObjC++ 调用原生 C | WASM Worker 运行同一 C 核心 | ## 安装 @@ -134,17 +137,43 @@ const restoredBytes = await patchBytes(oldBytes, patchBytesValue, { Web API 返回新的 `Uint8Array`，不会转移或失效调用方的缓冲区。主动取消以 `EABORTED` 拒绝；命中二进制大小限制时以 `ERESOURCE` 拒绝。 +## 检查并验证补丁 + +先用 `inspectPatch()` 完成低成本结构检查，再用 `verifyPatch()` 将补丁应用到临时 +结果，并与预期目标逐字节比较： + +```ts +import { inspectPatch, verifyPatch } from 'react-native-bs-diff-patch'; + +// Android / iOS 使用路径；Web 使用 File、Blob、ArrayBuffer 或 TypedArray。 +const metadata = await inspectPatch(patchPath); +const result = await verifyPatch(oldPath, patchPath, expectedPath, { + maxInputBytes: 64 * 1024 * 1024, + maxOutputBytes: 128 * 1024 * 1024, +}); + +if (!metadata.valid || !result.verified) { + throw new Error('补丁兼容性验证失败'); +} +``` + +原生验证产生的临时输出始终会被清理。Web 版本按相同顺序传入 `oldFile`、 +`patchFile` 与 `expectedFile`。结构有效只用于诊断；替换业务数据前仍应认证更新清单 +中的可信哈希。 + ## API 矩阵 -| API | Android | iOS | Web | -| ------------------------------------------ | ------- | ------ | ------ | -| `diff(oldPath, newPath, patchPath)` | 支持 | 支持 | 不支持 | -| `patch(oldPath, outputPath, patchPath)` | 支持 | 支持 | 不支持 | -| `startDiff(...)` / `startPatch(...)` | 支持 | 支持 | 不支持 | -| `diffBytes(oldData, newData, options?)` | 不支持 | 不支持 | 支持 | -| `patchBytes(oldData, patchData, options?)` | 不支持 | 不支持 | 支持 | -| 旧架构（限 RN 仍提供时） | 支持 | 支持 | 不适用 | -| 新架构 / TurboModule | 支持 | 支持 | 不适用 | +| API | Android | iOS | Web | +| --------------------------------------------- | ------- | ------ | ------ | +| `diff(oldPath, newPath, patchPath)` | 支持 | 支持 | 不支持 | +| `patch(oldPath, outputPath, patchPath)` | 支持 | 支持 | 不支持 | +| `startDiff(...)` / `startPatch(...)` | 支持 | 支持 | 不支持 | +| `diffBytes(oldData, newData, options?)` | 不支持 | 不支持 | 支持 | +| `patchBytes(oldData, patchData, options?)` | 不支持 | 不支持 | 支持 | +| `inspectPatch(path 或 binary, options?)` | 支持 | 支持 | 支持 | +| `verifyPatch(old, patch, expected, options?)` | 支持 | 支持 | 支持 | +| 旧架构（限 RN 仍提供时） | 支持 | 支持 | 不适用 | +| 新架构 / TurboModule | 支持 | 支持 | 不适用 | 调用当前平台不可用的 API 会以 `EUNSUPPORTED` 拒绝，不会静默切换成其他输入 模型。 diff --git a/android/newarch/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchModule.kt b/android/newarch/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchModule.kt index 44076b9..67a9808 100644 --- a/android/newarch/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchModule.kt +++ b/android/newarch/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchModule.kt @@ -17,6 +17,25 @@ class BsDiffPatchModule(reactContext: ReactApplicationContext) : override fun diff(oldFile: String, newFile: String, patchFile: String, promise: Promise) = support.diff(oldFile, newFile, patchFile, promise) + override fun inspectPatch(patchFile: String, maxInputBytes: Double, promise: Promise) = + support.inspectPatch(patchFile, maxInputBytes, promise) + + override fun verifyPatch( + oldFile: String, + patchFile: String, + expectedFile: String, + maxInputBytes: Double, + maxOutputBytes: Double, + promise: Promise + ) = support.verifyPatch( + oldFile, + patchFile, + expectedFile, + maxInputBytes, + maxOutputBytes, + promise + ) + override fun startPatch( jobId: String, oldFile: String, diff --git a/android/oldarch/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchModule.kt b/android/oldarch/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchModule.kt index 38813c4..c25fb9b 100644 --- a/android/oldarch/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchModule.kt +++ b/android/oldarch/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchModule.kt @@ -21,6 +21,27 @@ class BsDiffPatchModule(reactContext: ReactApplicationContext) : fun diff(oldFile: String, newFile: String, patchFile: String, promise: Promise) = support.diff(oldFile, newFile, patchFile, promise) + @ReactMethod + fun inspectPatch(patchFile: String, maxInputBytes: Double, promise: Promise) = + support.inspectPatch(patchFile, maxInputBytes, promise) + + @ReactMethod + fun verifyPatch( + oldFile: String, + patchFile: String, + expectedFile: String, + maxInputBytes: Double, + maxOutputBytes: Double, + promise: Promise + ) = support.verifyPatch( + oldFile, + patchFile, + expectedFile, + maxInputBytes, + maxOutputBytes, + promise + ) + @ReactMethod fun startPatch( jobId: String, diff --git a/android/src/main/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchModuleSupport.kt b/android/src/main/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchModuleSupport.kt index 0659107..7ff19d0 100644 --- a/android/src/main/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchModuleSupport.kt +++ b/android/src/main/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchModuleSupport.kt @@ -4,6 +4,7 @@ import com.facebook.react.bridge.Arguments import com.facebook.react.bridge.Promise import com.facebook.react.bridge.ReactApplicationContext import com.facebook.react.modules.core.DeviceEventManagerModule +import java.io.File import java.util.concurrent.atomic.AtomicInteger internal class BsDiffPatchModuleSupport( @@ -28,6 +29,47 @@ internal class BsDiffPatchModuleSupport( } } + fun inspectPatch(patchFile: String, maxInputBytes: Double, promise: Promise) { + taskRunner.execute(promise) { + BsDiffPatchNative.inspectPatch(patchFile, maxInputBytes) + } + } + + fun verifyPatch( + oldFile: String, + patchFile: String, + expectedFile: String, + maxInputBytes: Double, + maxOutputBytes: Double, + promise: Promise + ) { + taskRunner.execute(promise) { + val temporaryOutput = File.createTempFile( + "bsdiffpatch-verify-", + ".tmp", + reactContext.cacheDir + ) + if (!temporaryOutput.delete()) { + throw BsDiffPatchException( + "EUNSPECIFIED", + "could not prepare temporary verification output" + ) + } + try { + BsDiffPatchNative.verifyPatch( + oldFile, + patchFile, + expectedFile, + temporaryOutput.absolutePath, + maxInputBytes, + maxOutputBytes + ) + } finally { + temporaryOutput.delete() + } + } + } + fun startPatch( jobId: String, oldFile: String, diff --git a/android/src/main/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchNative.kt b/android/src/main/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchNative.kt index 8d0a730..c77172a 100644 --- a/android/src/main/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchNative.kt +++ b/android/src/main/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchNative.kt @@ -1,7 +1,9 @@ package com.jimmydaddy.bsdiffpatch import java.io.File +import java.io.FileInputStream import java.io.RandomAccessFile +import org.json.JSONObject import kotlin.math.floor internal object BsDiffPatchNative { @@ -34,6 +36,76 @@ internal object BsDiffPatchNative { ) } + fun inspectPatch(patchFile: String, maxInputBytes: Double): String { + val inputLimit = validateLimit(maxInputBytes, "maxInputBytes") + val patch = File(normalizePath(patchFile)) + validateNonEmpty(patchFile, "patchFile") + requireInput(patch, "patchFile", patchFile) + enforceInputLimit(inputLimit, patch) + return inspectPatchFile(patch).toJson().toString() + } + + fun verifyPatch( + oldFile: String, + patchFile: String, + expectedFile: String, + outputFile: String, + maxInputBytes: Double, + maxOutputBytes: Double + ): String { + validateNonEmpty(oldFile, "oldFile") + validateNonEmpty(patchFile, "patchFile") + validateNonEmpty(expectedFile, "expectedFile") + validateNonEmpty(outputFile, "outputFile") + val normalized = listOf(oldFile, patchFile, expectedFile, outputFile).map(::normalizePath) + if (normalized.toSet().size != normalized.size) { + throw BsDiffPatchException( + "EINVAL", + "oldFile, patchFile, expectedFile, and internal output can not be the same" + ) + } + + val old = File(normalized[0]) + val patch = File(normalized[1]) + val expected = File(normalized[2]) + val output = File(normalized[3]) + requireInput(old, "oldFile", oldFile) + requireInput(patch, "patchFile", patchFile) + requireInput(expected, "expectedFile", expectedFile) + requireOutput(output, "outputFile", outputFile) + + val inputLimit = validateLimit(maxInputBytes, "maxInputBytes") + val outputLimit = validateLimit(maxOutputBytes, "maxOutputBytes") + enforceInputLimit(inputLimit, old, patch, expected) + enforcePatchOutputLimit(outputLimit, patch) + val metadata = inspectPatchFile(patch) + if (!metadata.valid) { + throw BsDiffPatchException( + "EPATCH", + "patch structure is invalid: ${metadata.issue ?: "UNKNOWN"}" + ) + } + + requireSuccess( + "EPATCH", + "patch verification", + bsPatchFile(old.absolutePath, output.absolutePath, patch.absolutePath) + ) + if (outputLimit > 0 && output.length() > outputLimit) { + throw BsDiffPatchException( + "EOUTPUT_TOO_LARGE", + "output is ${output.length()} bytes and exceeds the configured $outputLimit byte limit" + ) + } + val verified = filesEqual(output, expected) + return JSONObject() + .put("verified", verified) + .put("restoredBytes", output.length()) + .put("expectedBytes", expected.length()) + .put("patch", metadata.toJson()) + .toString() + } + fun patchJob( jobId: String, oldFile: String, @@ -189,6 +261,83 @@ internal object BsDiffPatchNative { } } + private fun inspectPatchFile(patchFile: File): PatchMetadataValue { + val patchBytes = patchFile.length() + val headerBytes = minOf(patchBytes, 24L).toInt() + val header = ByteArray(headerBytes) + FileInputStream(patchFile).use { input -> + var offset = 0 + while (offset < header.size) { + val count = input.read(header, offset, header.size - offset) + if (count < 0) break + offset += count + } + } + val legacyMagic = header.copyOfRange(0, minOf(8, header.size)).toString(Charsets.US_ASCII) + val currentMagic = header.copyOfRange(0, minOf(16, header.size)).toString(Charsets.US_ASCII) + if (header.size < 24) { + return PatchMetadataValue( + format = if (legacyMagic == "BSDIFF40") "BSDIFF40" else "UNKNOWN", + patchBytes = patchBytes, + headerBytes = header.size, + declaredTargetBytes = null, + valid = false, + issue = if (legacyMagic == "BSDIFF40") "LEGACY_FORMAT" else "TRUNCATED_HEADER" + ) + } + if (currentMagic != "ENDSLEY/BSDIFF43") { + return PatchMetadataValue( + format = if (legacyMagic == "BSDIFF40") "BSDIFF40" else "UNKNOWN", + patchBytes = patchBytes, + headerBytes = 24, + declaredTargetBytes = null, + valid = false, + issue = if (legacyMagic == "BSDIFF40") "LEGACY_FORMAT" else "INVALID_MAGIC" + ) + } + if (header[23].toInt() and 0x80 != 0) { + return PatchMetadataValue( + format = "ENDSLEY/BSDIFF43", + patchBytes = patchBytes, + headerBytes = 24, + declaredTargetBytes = null, + valid = false, + issue = "INVALID_TARGET_SIZE" + ) + } + var targetBytes = 0L + for (index in 23 downTo 16) { + targetBytes = targetBytes * 256 + (header[index].toInt() and 0xff) + } + return PatchMetadataValue( + format = "ENDSLEY/BSDIFF43", + patchBytes = patchBytes, + headerBytes = 24, + declaredTargetBytes = targetBytes.toString(), + valid = true, + issue = null + ) + } + + private fun filesEqual(first: File, second: File): Boolean { + if (first.length() != second.length()) return false + FileInputStream(first).buffered().use { firstInput -> + FileInputStream(second).buffered().use { secondInput -> + val firstBuffer = ByteArray(DEFAULT_BUFFER_SIZE) + val secondBuffer = ByteArray(DEFAULT_BUFFER_SIZE) + while (true) { + val firstCount = firstInput.read(firstBuffer) + val secondCount = secondInput.read(secondBuffer) + if (firstCount != secondCount) return false + if (firstCount < 0) return true + for (index in 0 until firstCount) { + if (firstBuffer[index] != secondBuffer[index]) return false + } + } + } + } + } + private fun requireInput(file: File, fieldName: String, originalPath: String) { if (!file.exists()) { throw BsDiffPatchException("ENOENT", "$fieldName: $originalPath does not exist") @@ -273,6 +422,26 @@ internal object BsDiffPatchNative { private external fun bsCancelOperation(jobId: String): Boolean private data class Paths(val old: File, val input: File, val output: File) + + private data class PatchMetadataValue( + val format: String, + val patchBytes: Long, + val headerBytes: Int, + val declaredTargetBytes: String?, + val valid: Boolean, + val issue: String? + ) { + fun toJson(): JSONObject = JSONObject() + .put("format", format) + .put("patchBytes", patchBytes) + .put("headerBytes", headerBytes) + .put("payloadBytes", (patchBytes - 24L).coerceAtLeast(0L)) + .put("declaredTargetBytes", declaredTargetBytes ?: JSONObject.NULL) + .put("valid", valid) + .apply { + if (issue != null) put("issue", issue) + } + } } internal class BsDiffPatchException( diff --git a/android/src/main/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchTaskRunner.kt b/android/src/main/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchTaskRunner.kt index 9b7a6f6..9ae44e8 100644 --- a/android/src/main/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchTaskRunner.kt +++ b/android/src/main/java/com/jimmydaddy/bsdiffpatch/BsDiffPatchTaskRunner.kt @@ -14,7 +14,7 @@ internal class BsDiffPatchTaskRunner { Thread(runnable, "BsDiffPatchWorker") } - fun execute(promise: Promise, block: () -> Int) { + fun execute(promise: Promise, block: () -> Any?) { submit(promise) { block() } } @@ -52,7 +52,7 @@ internal class BsDiffPatchTaskRunner { return activeJobIds } - private fun submit(promise: Promise, block: () -> Int): Boolean { + private fun submit(promise: Promise, block: () -> Any?): Boolean { try { executor.execute { try { diff --git a/benchmarks/README.md b/benchmarks/README.md index aa2cd48..bc6829f 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -1,15 +1,29 @@ # Benchmarks -`web-wasm.json` is a reproducible reference measurement for the checked-in -WebAssembly implementation. It is not a performance guarantee: file contents, -hardware, browser scheduling, and available memory can change results. +`web-wasm.json` and `native-core.json` are the inexpensive reference trend +lines. `web-wasm-large.json` and `native-core-large.json` record the explicit +16/64/128 MiB feasibility profile. They are not performance guarantees: file +contents, hardware, browser scheduling, and available memory can change +results. Regenerate the 1, 10, and 50 MiB workload with: ```sh BENCHMARK_OUTPUT=benchmarks/web-wasm.json yarn benchmark:web +BENCHMARK_OUTPUT=benchmarks/native-core.json yarn benchmark:native +BENCHMARK_OUTPUT=benchmarks/web-wasm-large.json yarn benchmark:large:web +BENCHMARK_OUTPUT=benchmarks/native-core-large.json yarn benchmark:large:native ``` The workload changes one byte per 4 KiB and verifies every restored byte. The -benchmark measures the WebAssembly core after a small initialization warm-up; -the public browser API additionally transfers data through a module Worker. +Web benchmark measures the WebAssembly core after a small initialization +warm-up; the public browser API additionally transfers data through a module +Worker. Every size runs in a fresh process so peak resident memory is +comparable. + +On the recorded Apple M3 Pro baseline, native completed all three large sizes. +Web completed 16 and 64 MiB, but its 128 MiB diff returned `EWEBASSEMBLY` after +reaching the current WebAssembly memory boundary. The failed sample is retained +intentionally: it is a measured limitation, not a flaky result. See the +[large-file roadmap](../docs/large-files-v04.md) before interpreting or changing +these limits. diff --git a/benchmarks/native-core-large.json b/benchmarks/native-core-large.json new file mode 100644 index 0000000..ec638e9 --- /dev/null +++ b/benchmarks/native-core-large.json @@ -0,0 +1,42 @@ +{ + "generatedAt": "2026-07-20T06:10:29.044Z", + "runtime": { + "cpu": "Apple M3 Pro", + "compiler": "Apple clang version 17.0.0 (clang-1700.6.3.2)", + "platform": "darwin-arm64" + }, + "workload": { + "description": "Deterministic files with one changed byte per 4 KiB", + "memoryMetric": "Peak resident set size for one diff and patch process" + }, + "summary": { + "passed": 3, + "failed": 0 + }, + "results": [ + { + "status": "passed", + "sizeMiB": 16, + "diffMs": 10012.4, + "patchMs": 70.7, + "patchBytes": 113, + "peakRssKiB": 316368 + }, + { + "status": "passed", + "sizeMiB": 64, + "diffMs": 41943.7, + "patchMs": 249, + "patchBytes": 189, + "peakRssKiB": 1256080 + }, + { + "status": "passed", + "sizeMiB": 128, + "diffMs": 101698.9, + "patchMs": 586.4, + "patchBytes": 323, + "peakRssKiB": 2486416 + } + ] +} diff --git a/benchmarks/web-wasm-large.json b/benchmarks/web-wasm-large.json new file mode 100644 index 0000000..0a49f0f --- /dev/null +++ b/benchmarks/web-wasm-large.json @@ -0,0 +1,50 @@ +{ + "generatedAt": "2026-07-20T06:07:44.301Z", + "runtime": { + "cpu": "Apple M3 Pro", + "node": "v22.22.0", + "platform": "darwin-arm64" + }, + "workload": { + "description": "Deterministic buffers with one changed byte per 4 KiB", + "memoryMetric": "Peak resident set size for one initialized WebAssembly diff and patch process", + "processIsolation": "Each input size runs in a fresh Node.js process" + }, + "summary": { + "passed": 2, + "failed": 1 + }, + "results": [ + { + "sizeMiB": 16, + "status": "passed", + "initializationMs": 15.6, + "diffMs": 8769.2, + "patchMs": 125.4, + "patchBytes": 113, + "peakRssMiB": 639.1, + "residentAfterMiB": 639.1, + "externalAfterMiB": 442.2, + "arrayBuffersAfterMiB": 129.1 + }, + { + "sizeMiB": 64, + "status": "passed", + "initializationMs": 13.2, + "diffMs": 49136.5, + "patchMs": 784.7, + "patchBytes": 189, + "peakRssMiB": 2094.3, + "residentAfterMiB": 2094.3, + "externalAfterMiB": 1614.7, + "arrayBuffersAfterMiB": 389.6 + }, + { + "sizeMiB": 128, + "status": "failed", + "errorCode": "EWEBASSEMBLY", + "errorMessage": "diff failed: native function returned -1", + "peakRssMiB": 851.4 + } + ] +} diff --git a/docs/README.md b/docs/README.md index eff93af..9a535e2 100644 --- a/docs/README.md +++ b/docs/README.md @@ -25,6 +25,8 @@ The [Chinese documentation](./zh-CN/README.md) mirrors the same public guides. - [Architecture](./architecture.md) — execution paths and patch compatibility. - [Controllable native operations](./native-operations-v03.md) — resource limits, cancellation, progress, and atomic output contract. +- [Large-file roadmap](./large-files-v04.md) — memory baselines, honest progress, + and streaming feasibility for the next architecture iteration. - [Troubleshooting](./troubleshooting.md) — common integration failures. - [Development](./development.md) — local builds, tests, WebAssembly, and release checks. diff --git a/docs/api-reference.md b/docs/api-reference.md index c9853f2..19b66d4 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -11,8 +11,12 @@ import { startPatch, diffBytes, patchBytes, + inspectPatch, + verifyPatch, type BinaryInput, type BinaryOperationOptions, + type PatchMetadata, + type PatchVerificationResult, } from 'react-native-bs-diff-patch'; ``` @@ -147,6 +151,85 @@ bytes. Available on Web. - Rejects before allocating the declared output when the patch header exceeds `maxOutputBytes`. +## `inspectPatch` + +```ts +interface PatchInspectionOptions { + maxInputBytes?: number; +} + +interface PatchMetadata { + format: 'ENDSLEY/BSDIFF43' | 'BSDIFF40' | 'UNKNOWN'; + patchBytes: number; + headerBytes: number; + payloadBytes: number; + declaredTargetBytes: string | null; + valid: boolean; + issue?: + | 'TRUNCATED_HEADER' + | 'LEGACY_FORMAT' + | 'INVALID_MAGIC' + | 'INVALID_TARGET_SIZE'; +} + +function inspectPatch( + patchInput: string | BinaryInput, + options?: PatchInspectionOptions +): Promise; +``` + +Reads the 24-byte patch header without applying the patch. Pass a native patch +path on Android/iOS or a `BinaryInput` on Web. + +- `declaredTargetBytes` is a decimal string so values above + `Number.MAX_SAFE_INTEGER` remain exact. +- `valid` only establishes structural compatibility. It does not authenticate + the patch or prove that its compressed payload is intact. +- `BSDIFF40` is reported as `LEGACY_FORMAT`, not accepted as + `ENDSLEY/BSDIFF43`. +- `maxInputBytes` bounds the patch file or binary input before its header is + inspected. + +## `verifyPatch` + +```ts +interface PatchVerificationResult { + verified: boolean; + restoredBytes: number; + expectedBytes: number; + patch: PatchMetadata; +} + +// Android / iOS paths +function verifyPatch( + oldFile: string, + patchFile: string, + expectedFile: string, + options?: NativeOperationOptions +): Promise; + +// Web binary values +function verifyPatch( + oldData: BinaryInput, + patchData: BinaryInput, + expectedData: BinaryInput, + options?: BinaryOperationOptions +): Promise; +``` + +Applies the patch and compares the restored result with the expected target +byte-for-byte. + +- A valid match resolves with `verified: true`; a well-formed patch that + restores different bytes resolves with `verified: false`. +- Malformed or incompatible structure rejects with `EPATCH`. +- Native implementations use a library-owned temporary output and remove it on + success, mismatch, or failure. The method never replaces application data. +- Web verification uses the same Worker/Wasm path as `patchBytes` and honors + `AbortSignal` and byte limits. +- Resource-limit failures from these portable APIs use `ERESOURCE` on every + platform. + ## Web operation options - `signal` cancels the current Web operation. A call with a signal receives a @@ -165,7 +248,8 @@ path operations use `startDiff` or `startPatch` for equivalent controls. All functions remain exported so shared code has one stable import shape. Calling `diffBytes` or `patchBytes` on native rejects with `EUNSUPPORTED`. Calling `diff`, `patch`, `startDiff`, or `startPatch` on Web behaves the same -way. +way. `inspectPatch` and `verifyPatch` are available on every platform, but they +require native paths on Android/iOS and binary values on Web. Importing the Web entry during server-side rendering does not start a Worker. Calling a binary API without browser Worker support rejects with @@ -180,22 +264,22 @@ platform can classify the failure. type PatchError = Error & { code?: string }; ``` -| Code | Meaning | -| -------------- | ----------------------------------------------------------- | -| `EINVAL` | Empty, duplicate, or invalid input. | -| `ENOENT` | A required native file does not exist. | -| `EEXIST` | A native output path already exists. | -| `EUNSUPPORTED` | The selected API is not available on the current platform. | -| `EUNAVAILABLE` | The native module worker has already shut down. | -| `ECANCELLED` | A native job was cooperatively cancelled. | -| `EINPUT_TOO_LARGE` | A native input exceeded `maxInputBytes`. | -| `EOUTPUT_TOO_LARGE` | Native generated/restored output exceeded its limit. | -| `EABORTED` | The Web operation was cancelled through its signal. | -| `ERESOURCE` | A configured Web input or output byte limit was exceeded. | -| `EDIFF` | The native diff core rejected or could not write the input. | -| `EPATCH` | The native patch core rejected a malformed patch or output. | -| `EWEBASSEMBLY` | WebAssembly loading, patch validation, or execution failed. | -| `EUNSPECIFIED` | An unclassified native exception occurred. | +| Code | Meaning | +| ------------------- | ----------------------------------------------------------- | +| `EINVAL` | Empty, duplicate, or invalid input. | +| `ENOENT` | A required native file does not exist. | +| `EEXIST` | A native output path already exists. | +| `EUNSUPPORTED` | The selected API is not available on the current platform. | +| `EUNAVAILABLE` | The native module worker has already shut down. | +| `ECANCELLED` | A native job was cooperatively cancelled. | +| `EINPUT_TOO_LARGE` | A native input exceeded `maxInputBytes`. | +| `EOUTPUT_TOO_LARGE` | Native generated/restored output exceeded its limit. | +| `EABORTED` | The Web operation was cancelled through its signal. | +| `ERESOURCE` | A portable or Web input/output byte limit was exceeded. | +| `EDIFF` | The native diff core rejected or could not write the input. | +| `EPATCH` | The native patch core rejected a malformed patch or output. | +| `EWEBASSEMBLY` | WebAssembly loading, patch validation, or execution failed. | +| `EUNSPECIFIED` | An unclassified native exception occurred. | Treat error messages as diagnostic text rather than a stable machine-readable contract. Branch on `code` when recovery behavior differs. diff --git a/docs/development.md b/docs/development.md index ae0bc29..ae2bd8a 100644 --- a/docs/development.md +++ b/docs/development.md @@ -65,7 +65,7 @@ source-pattern assertions. `test:native-operations` deterministically covers job progress, cancellation, limits, malformed patches, atomic destination behavior, and temporary cleanup. -Run the repeatable Web performance baseline with: +Run the repeatable Web and native performance baselines with: ```sh yarn benchmark:web @@ -74,6 +74,22 @@ yarn benchmark:native BENCHMARK_OUTPUT=/tmp/native-core.json yarn benchmark:native ``` +Each input size runs in a fresh process. Both reports include peak resident +memory; the Web report also records the resident, external, and ArrayBuffer +memory still live after the round trip. Use the non-blocking large-file profile +before changing buffer ownership or the patch format: + +```sh +BENCHMARK_OUTPUT=/tmp/web-large.json yarn benchmark:large:web +BENCHMARK_OUTPUT=/tmp/native-large.json yarn benchmark:large:native +``` + +The large profile uses 16, 64, and 128 MiB fixtures and can consume several +gigabytes of memory. It is intentionally not a pull-request gate. A manual +`Native Core Benchmark` run accepts a comma-separated size list when a shared +runner baseline is useful. Interpret the numbers with the scope and acceptance +criteria in the [large-file roadmap](./large-files-v04.md). + The published-package canaries install directly from npm and intentionally use current Vite and Expo toolchains. They are scheduled CI checks, not release gates. Run them on demand with `yarn test:registry:vite` and diff --git a/docs/large-files-v04.md b/docs/large-files-v04.md new file mode 100644 index 0000000..223789d --- /dev/null +++ b/docs/large-files-v04.md @@ -0,0 +1,102 @@ +# Large-file roadmap (v0.4) + +This document defines how the project will evaluate larger inputs, expose +honest progress, and investigate streaming without weakening patch +compatibility. It is a feasibility and measurement plan, not a promise that +every browser or mobile device can process a particular file size. + +## Current constraints + +The current diff algorithm needs random access to the complete old and new +inputs while building and traversing its suffix array. Native calls therefore +operate on file paths but still allocate memory proportional to the input. The +Web implementation additionally moves complete buffers between JavaScript, a +Worker, and WebAssembly linear memory. + +Patch application is less demanding than diff generation, but the current C +and Web boundaries still materialize the complete operation state. Resource +limits prevent unbounded work; they do not make the algorithm streaming. + +## Measurement matrix + +The default 1, 10, and 50 MiB benchmarks remain the inexpensive trend line. +The explicit large-file profile uses deterministic 16, 64, and 128 MiB inputs +with one changed byte per 4 KiB: + +```sh +BENCHMARK_OUTPUT=/tmp/web-large.json yarn benchmark:large:web +BENCHMARK_OUTPUT=/tmp/native-large.json yarn benchmark:large:native +``` + +Each size runs in a fresh process so peak resident memory is comparable. Record +the runtime, diff and patch duration, patch size, round-trip result, and peak +resident memory. The Web report also records live external and ArrayBuffer +memory after the round trip. Compare only runs from similar hardware and +toolchains; shared-runner numbers are diagnostic artifacts, not PR thresholds. +The large profiling commands preserve per-size failures in the JSON report and +complete the remaining samples; a recorded report is not itself evidence that +all sizes passed. Default benchmarks still exit unsuccessfully on any failure. + +The profiling run passes when every requested size either completes a verified +round trip within the host's documented memory budget or fails with a defined +resource error. A crash, leaked temporary output, corrupted result, or silent +fallback is a failure. Results do not imply support above the measured size or +on lower-memory devices. + +The initial Apple M3 Pro / Node 22 record is checked in under `benchmarks/`. +Native completed 128 MiB with approximately 2.37 GiB peak RSS. Web completed +64 MiB with approximately 2.09 GiB peak RSS, while 128 MiB returned the generic +`EWEBASSEMBLY` error. That generic failure remains an error-taxonomy gap and +means the project does not currently claim 128 MiB Web diff support. + +## Progress semantics + +Progress must be produced by real algorithm checkpoints, never a timer or an +animation that guesses completion. A future cross-platform operation can use +the existing stages: + +- `reading`: validating inputs and loading the data required by the core. +- `processing`: suffix-array/diff work or patch reconstruction. +- `writing`: persisting and atomically committing native output; Web completes + this stage when the result buffer is ready to transfer. + +Native jobs already expose these stages. Web parity requires Worker messages +emitted from instrumented C/WebAssembly boundaries. Until those checkpoints +exist, Web should report only start, cancellation, and completion rather than +synthetic percentages. The public callback remains optional and must not change +the result or error behavior when it is absent. + +## Streaming feasibility + +True streaming diff is not a compatible optimization of the current BSDiff +algorithm: suffix-array construction and matching require global random access +to both inputs. Supporting it would mean selecting a different algorithm or a +new patch format, with an explicit compatibility and migration decision. + +Patch application is a better candidate for incremental work. A prototype can +read the old file and compressed control/diff/extra streams in bounded chunks, +write a temporary destination, and retain the current `ENDSLEY/BSDIFF43` +contract. Browser support should start with `Blob`/`File` and an internal +bounded reader; writable file handles can remain a progressive enhancement. + +## Delivery sequence + +1. Keep 16/64/128 MiB time and peak-memory baselines for native and Web. +2. Instrument core checkpoints and add truthful Web progress events without + changing the existing `diff`, `patch`, or `startPatch` contracts. +3. Prototype file-backed, incremental patch application and prove cancellation, + resource limits, temporary cleanup, and byte-for-byte compatibility. +4. Decide whether the measured benefit justifies a new public API. Treat a + streaming diff algorithm or new patch format as a separate proposal. + +Any production API must preserve deterministic output validation, reject sizes +outside configured limits before large allocations where possible, clean up on +cancellation and failure, and pass Android API 24, iOS Simulator, browser, and +cross-platform golden-patch tests. + +## Non-goals + +- Claiming that a single measured desktop result is a mobile support guarantee. +- Advertising inputs larger than 128 MiB without a new measurement record. +- Uploading user files to a hosted service; the site toolkit remains local-only. +- Introducing simulated progress merely to make a long operation look active. diff --git a/docs/platform-support.md b/docs/platform-support.md index 7b22b14..68968f6 100644 --- a/docs/platform-support.md +++ b/docs/platform-support.md @@ -2,16 +2,18 @@ ## Capability matrix -| Capability | Android | iOS | React Native Web | -| ------------------------------ | ------------------ | --------------------- | ------------------ | -| File-path APIs | Yes | Yes | No | -| Binary-data APIs | No | No | Yes | -| Progress / cooperative cancel | Native jobs | Native jobs | `AbortSignal` | -| Input / output limits | Native jobs | Native jobs | Binary options | -| Legacy bridge | Yes | Yes | N/A | -| TurboModule / New Architecture | Yes | Yes | N/A | -| Background execution | Serial executor | Serialized worker | Module Web Worker | -| Patch format | `ENDSLEY/BSDIFF43` | `ENDSLEY/BSDIFF43` | `ENDSLEY/BSDIFF43` | +| Capability | Android | iOS | React Native Web | +| ------------------------------ | ------------------ | ------------------ | ------------------ | +| File-path APIs | Yes | Yes | No | +| Binary-data APIs | No | No | Yes | +| Progress / cooperative cancel | Native jobs | Native jobs | `AbortSignal` | +| Input / output limits | Native jobs | Native jobs | Binary options | +| Patch metadata | File path | File path | Binary input | +| Byte-for-byte verification | Temporary file | Temporary file | In-memory result | +| Legacy bridge | Yes | Yes | N/A | +| TurboModule / New Architecture | Yes | Yes | N/A | +| Background execution | Serial executor | Serialized worker | Module Web Worker | +| Patch format | `ENDSLEY/BSDIFF43` | `ENDSLEY/BSDIFF43` | `ENDSLEY/BSDIFF43` | The example application uses React Native 0.86.0 and exercises the New Architecture on Android API 24/31 and iOS Simulator. Direct compatibility @@ -33,6 +35,9 @@ Native minor version: Native operations run on a module-owned single-thread executor. Jobs add a registry for queued/active cancellation, rate-limited progress events, limits, and cleanup. The packaged C code is built with CMake and invoked through JNI. +`inspectPatch` reads only the patch header. `verifyPatch` applies into a cache +file, compares it with the expected path in bounded chunks, and removes the +temporary output before resolving. ## iOS @@ -45,6 +50,8 @@ following cancellation request is handled. Patch work runs asynchronously on a separate serial worker queue, keeping the bridge responsive while preserving the shared C library's single-operation boundary. The job registry is cancelled and cleaned when the module is invalidated. +Metadata inspection reads only 24 header bytes. Verification writes to a unique +file under the system temporary directory and removes it on every exit path. ## React Native Web @@ -75,6 +82,9 @@ Calls without an `AbortSignal` share a module Worker and initialized WebAssembly module. Calls with a signal receive a dedicated Worker so cancellation is isolated to that operation. Both paths serialize work inside their Worker; callers should still enforce an application memory budget. +`inspectPatch` does not start a Worker. `verifyPatch` first validates metadata, +then uses the same Worker path as `patchBytes` and discards the restored buffer +after comparing it with the expected input. ## Server-side rendering diff --git a/docs/zh-CN/README.md b/docs/zh-CN/README.md index 4a9d7d7..50233fd 100644 --- a/docs/zh-CN/README.md +++ b/docs/zh-CN/README.md @@ -25,5 +25,7 @@ - [架构](/docs/zh-CN/architecture/) — 执行路径与补丁兼容性。 - [可控制的原生操作](/docs/zh-CN/native-operations-v03/) — 资源限制、取消、进度与 原子输出约定。 +- [大文件演进路线](/docs/zh-CN/large-files-v04/) — 下一阶段的内存基线、真实进度和 + 流式能力可行性。 - [常见问题与排障](/docs/zh-CN/troubleshooting/) — 常见集成失败。 - [开发与验证](/docs/zh-CN/development/) — 本地构建、测试、WASM 与发布检查。 diff --git a/docs/zh-CN/api-reference.md b/docs/zh-CN/api-reference.md index 0ae7c03..d3dbd2f 100644 --- a/docs/zh-CN/api-reference.md +++ b/docs/zh-CN/api-reference.md @@ -11,8 +11,12 @@ import { startPatch, diffBytes, patchBytes, + inspectPatch, + verifyPatch, type BinaryInput, type BinaryOperationOptions, + type PatchMetadata, + type PatchVerificationResult, } from 'react-native-bs-diff-patch'; ``` @@ -139,6 +143,79 @@ function patchBytes( - 不会修改 `oldData` 或 `patchData`。 - 当补丁头声明的输出超过 `maxOutputBytes` 时，会在分配输出前拒绝。 +## `inspectPatch` + +```ts +interface PatchInspectionOptions { + maxInputBytes?: number; +} + +interface PatchMetadata { + format: 'ENDSLEY/BSDIFF43' | 'BSDIFF40' | 'UNKNOWN'; + patchBytes: number; + headerBytes: number; + payloadBytes: number; + declaredTargetBytes: string | null; + valid: boolean; + issue?: + | 'TRUNCATED_HEADER' + | 'LEGACY_FORMAT' + | 'INVALID_MAGIC' + | 'INVALID_TARGET_SIZE'; +} + +function inspectPatch( + patchInput: string | BinaryInput, + options?: PatchInspectionOptions +): Promise; +``` + +不应用补丁，只读取 24 字节补丁头。Android/iOS 传入补丁路径，Web 传入 +`BinaryInput`。 + +- `declaredTargetBytes` 使用十进制字符串，避免超过 `Number.MAX_SAFE_INTEGER` + 后丢失精度。 +- `valid` 只表示结构兼容，不能认证补丁，也不能证明压缩负载完整。 +- `BSDIFF40` 会报告为 `LEGACY_FORMAT`，不会被当作 + `ENDSLEY/BSDIFF43` 接受。 +- `maxInputBytes` 在读取补丁头前限制原生文件或 Web 二进制输入。 + +## `verifyPatch` + +```ts +interface PatchVerificationResult { + verified: boolean; + restoredBytes: number; + expectedBytes: number; + patch: PatchMetadata; +} + +// Android / iOS 文件路径 +function verifyPatch( + oldFile: string, + patchFile: string, + expectedFile: string, + options?: NativeOperationOptions +): Promise; + +// Web 二进制值 +function verifyPatch( + oldData: BinaryInput, + patchData: BinaryInput, + expectedData: BinaryInput, + options?: BinaryOperationOptions +): Promise; +``` + +应用补丁，并将还原结果与预期目标逐字节比较。 + +- 完全一致时返回 `verified: true`；结构有效但还原内容不同时返回 + `verified: false`。 +- 补丁结构损坏或不兼容时以 `EPATCH` 拒绝。 +- 原生实现使用库管理的临时输出，并在成功、不匹配或失败后清理；不会替换业务文件。 +- Web 使用与 `patchBytes` 相同的 Worker/Wasm 路径，并支持 `AbortSignal` 和字节限制。 +- 这些跨平台 API 的资源限制错误在所有平台统一为 `ERESOURCE`。 + ## Web 操作选项 - `signal` 取消当前 Web 操作。带 signal 的调用使用专用 Worker，因此取消不会中断 @@ -155,7 +232,8 @@ function patchBytes( 所有函数始终导出，以便共享代码保持稳定导入形式。在原生端调用 `diffBytes` 或 `patchBytes`，以及在 Web 调用 `diff`、`patch`、`startDiff` 或 `startPatch`， -都会以 `EUNSUPPORTED` 拒绝。 +都会以 `EUNSUPPORTED` 拒绝。`inspectPatch` 与 `verifyPatch` 在所有平台可用，但 +Android/iOS 必须传文件路径，Web 必须传二进制值。 SSR 阶段导入 Web 入口不会启动 Worker；在没有浏览器 Worker 的环境调用二进制 API 会以 `EUNSUPPORTED` 拒绝。 @@ -168,22 +246,22 @@ API 会以 `EUNSUPPORTED` 拒绝。 type PatchError = Error & { code?: string }; ``` -| 错误码 | 含义 | -| -------------- | ----------------------------------------- | -| `EINVAL` | 输入为空、重复或类型无效。 | -| `ENOENT` | 原生端所需文件不存在。 | -| `EEXIST` | 原生端输出路径已经存在。 | -| `EUNSUPPORTED` | 当前平台不支持所选 API。 | -| `EUNAVAILABLE` | 原生模块工作队列已经关闭。 | -| `ECANCELLED` | 原生 job 被协作式取消。 | -| `EINPUT_TOO_LARGE` | 原生输入超过 `maxInputBytes`。 | -| `EOUTPUT_TOO_LARGE` | 原生生成或还原输出超过配置上限。 | -| `EABORTED` | Web 操作被传入的 signal 取消。 | -| `ERESOURCE` | 超过配置的 Web 输入或输出字节上限。 | -| `EDIFF` | 原生 diff 核心拒绝输入或无法写入补丁。 | -| `EPATCH` | 原生 patch 核心拒绝损坏补丁或输出失败。 | -| `EWEBASSEMBLY` | Worker、补丁校验或 WebAssembly 执行失败。 | -| `EUNSPECIFIED` | 未分类的原生异常。 | +| 错误码 | 含义 | +| ------------------- | ------------------------------------------ | +| `EINVAL` | 输入为空、重复或类型无效。 | +| `ENOENT` | 原生端所需文件不存在。 | +| `EEXIST` | 原生端输出路径已经存在。 | +| `EUNSUPPORTED` | 当前平台不支持所选 API。 | +| `EUNAVAILABLE` | 原生模块工作队列已经关闭。 | +| `ECANCELLED` | 原生 job 被协作式取消。 | +| `EINPUT_TOO_LARGE` | 原生输入超过 `maxInputBytes`。 | +| `EOUTPUT_TOO_LARGE` | 原生生成或还原输出超过配置上限。 | +| `EABORTED` | Web 操作被传入的 signal 取消。 | +| `ERESOURCE` | 超过跨平台或 Web API 的输入/输出字节上限。 | +| `EDIFF` | 原生 diff 核心拒绝输入或无法写入补丁。 | +| `EPATCH` | 原生 patch 核心拒绝损坏补丁或输出失败。 | +| `EWEBASSEMBLY` | Worker、补丁校验或 WebAssembly 执行失败。 | +| `EUNSPECIFIED` | 未分类的原生异常。 | 错误消息仅用于诊断，不是稳定的机器可读约定。恢复策略不同时应根据 `code` 分支。 diff --git a/docs/zh-CN/development.md b/docs/zh-CN/development.md index 3dd3894..fa8fef7 100644 --- a/docs/zh-CN/development.md +++ b/docs/zh-CN/development.md @@ -62,7 +62,7 @@ React Native artifact 直接编译真实 Android 模块源码，不依赖源码 `test:native-operations` 会确定性覆盖 job 进度、取消、限制、畸形补丁、原子目标行为 和临时文件清理。 -可复现的 Web 性能基准命令： +可复现的 Web 与原生性能基准命令： ```sh yarn benchmark:web @@ -71,6 +71,20 @@ yarn benchmark:native BENCHMARK_OUTPUT=/tmp/native-core.json yarn benchmark:native ``` +每个输入尺寸都在独立进程中运行。两份报告都会记录峰值常驻内存；Web 报告还会记录 +往返结束后仍存活的常驻、external 和 ArrayBuffer 内存。修改 buffer 所有权或补丁 +格式前，可运行不阻塞 PR 的大文件 profiling： + +```sh +BENCHMARK_OUTPUT=/tmp/web-large.json yarn benchmark:large:web +BENCHMARK_OUTPUT=/tmp/native-large.json yarn benchmark:large:native +``` + +大文件 profiling 使用 16、64、128 MiB fixture，可能消耗数 GiB 内存，因此不会作为 +pull request 门禁。手动运行 `Native Core Benchmark` 时可以传入逗号分隔的尺寸列表， +以获得共享 Runner 基线。解读结果时应遵循[大文件演进路线](/docs/zh-CN/large-files-v04/) +中的范围和验收标准。 + 发布包 canary 会直接从 npm 安装，并有意使用当前 Vite 与 Expo 工具链；它们是定时 CI，不是发版门禁。可用 `yarn test:registry:vite` 和 `yarn test:registry:expo` 手动执行，并通过 `PACKAGE_SPEC` 验证指定 tag 或 tarball。 diff --git a/docs/zh-CN/large-files-v04.md b/docs/zh-CN/large-files-v04.md new file mode 100644 index 0000000..d4e85bb --- /dev/null +++ b/docs/zh-CN/large-files-v04.md @@ -0,0 +1,80 @@ +# 大文件演进路线（v0.4） + +本文定义项目如何评估更大输入、提供可信进度并研究流式处理，同时不削弱补丁兼容性。 +它是一份可行性与测量计划，不承诺所有浏览器或移动设备都能处理某个固定尺寸。 + +## 当前约束 + +当前 diff 算法在构建和遍历后缀数组时，需要随机访问完整的旧、新输入。原生调用虽然 +接收文件路径，内存使用仍然与输入规模成比例。Web 实现还需要在 JavaScript、Worker +与 WebAssembly 线性内存之间传递完整 buffer。 + +应用补丁比生成 diff 的要求低，但当前 C 和 Web 边界仍会把完整操作状态放入内存。 +资源限制可以阻止无界工作，但不会让算法自动变成流式处理。 + +## 测量矩阵 + +默认 1、10、50 MiB 基准继续作为成本较低的趋势线。显式的大文件 profiling 使用 +16、64、128 MiB 确定性输入，每 4 KiB 修改一个字节： + +```sh +BENCHMARK_OUTPUT=/tmp/web-large.json yarn benchmark:large:web +BENCHMARK_OUTPUT=/tmp/native-large.json yarn benchmark:large:native +``` + +每个尺寸在独立进程运行，以便比较峰值常驻内存。需要记录 runtime、diff 和 patch +耗时、补丁尺寸、往返结果与峰值常驻内存。Web 报告还记录往返结束后的 external 与 +ArrayBuffer 存活内存。只比较相近硬件和工具链上的结果；共享 Runner 数据只作为诊断 +artifact，不作为 PR 阈值。 +大文件 profiling 命令会把单个尺寸的失败保留在 JSON 报告中并继续其他样本；生成了 +报告不代表全部尺寸通过。默认基准遇到任何失败时仍以失败状态退出。 + +当每个请求尺寸都在宿主机记录的内存预算内完成并验证往返，或以明确的资源错误失败 +时，profiling 才算通过。崩溃、临时输出泄漏、结果损坏或静默降级都属于失败。测量 +结果不代表更大尺寸或更低内存设备也获得支持。 + +首份 Apple M3 Pro / Node 22 记录已检入 `benchmarks/`。原生端以约 2.37 GiB 峰值 RSS +完成 128 MiB；Web 端以约 2.09 GiB 峰值 RSS 完成 64 MiB，但 128 MiB 返回通用 +`EWEBASSEMBLY`。这一通用失败仍是错误分类缺口，因此项目目前不宣称 Web diff 支持 +128 MiB。 + +## 进度语义 + +进度必须来自真实算法检查点，不能使用定时器或动画猜测完成比例。未来的跨平台操作 +可以沿用现有阶段： + +- `reading`：验证输入并加载核心需要的数据。 +- `processing`：执行后缀数组/diff 工作或重建补丁。 +- `writing`：原生端持久化并原子提交输出；Web 端在结果 buffer 可以传输时完成此阶段。 + +原生 job 已暴露这些阶段。Web 对齐需要从插桩后的 C/WebAssembly 边界发出 Worker +消息。在这些检查点出现之前，Web 只应报告开始、取消和完成，不能提供虚假百分比。 +公开 callback 保持可选，未传入时不得改变结果或错误行为。 + +## 流式处理可行性 + +真正的流式 diff 不是当前 BSDiff 算法的兼容优化：后缀数组构建和匹配需要全局随机 +访问两份输入。要实现它，必须选择另一种算法或新补丁格式，并明确兼容与迁移策略。 + +补丁应用更适合增量处理。原型可以按有界 chunk 读取旧文件和压缩后的 +control/diff/extra 流，写入临时目标，同时保留 `ENDSLEY/BSDIFF43` 约定。浏览器应先 +支持 `Blob`/`File` 和内部有界 reader；可写文件句柄继续作为渐进增强能力。 + +## 实施顺序 + +1. 保持原生与 Web 的 16/64/128 MiB 耗时和峰值内存基线。 +2. 为核心检查点插桩，增加真实的 Web 进度事件，不改变现有 `diff`、`patch` 或 + `startPatch` 约定。 +3. 验证文件后备、增量应用补丁的原型，并证明取消、资源限制、临时清理和逐字节兼容。 +4. 根据实测收益决定是否值得新增公开 API；流式 diff 算法或新补丁格式另立提案。 + +所有生产 API 都必须保持确定性输出验证，在大额分配前尽可能拒绝超限尺寸，在取消和 +失败时清理资源，并通过 Android API 24、iOS Simulator、浏览器与跨平台 golden patch +测试。 + +## 非目标 + +- 把单次桌面测量结果宣传为移动端支持保证。 +- 没有新测量记录就宣称支持超过 128 MiB 的输入。 +- 将用户文件上传到托管服务；站点工具继续只在本地处理。 +- 为了让耗时操作看起来有响应而引入模拟进度。 diff --git a/docs/zh-CN/platform-support.md b/docs/zh-CN/platform-support.md index 0ecb210..f29cef0 100644 --- a/docs/zh-CN/platform-support.md +++ b/docs/zh-CN/platform-support.md @@ -2,16 +2,18 @@ ## 能力矩阵 -| 能力 | Android | iOS | React Native Web | -| -------------------- | ------------------ | -------- | ---------------- | -| 文件路径 API | 支持 | 支持 | 不支持 | -| 二进制数据 API | 不支持 | 不支持 | 支持 | -| 进度 / 协作式取消 | 原生 job | 原生 job | `AbortSignal` | -| 输入 / 输出限制 | 原生 job | 原生 job | 二进制 options | -| 旧桥接架构 | 支持 | 支持 | 不适用 | -| TurboModule / 新架构 | 支持 | 支持 | 不适用 | +| 能力 | Android | iOS | React Native Web | +| -------------------- | ------------------ | ----------- | ---------------- | +| 文件路径 API | 支持 | 支持 | 不支持 | +| 二进制数据 API | 不支持 | 不支持 | 支持 | +| 进度 / 协作式取消 | 原生 job | 原生 job | `AbortSignal` | +| 输入 / 输出限制 | 原生 job | 原生 job | 二进制 options | +| 补丁元数据 | 文件路径 | 文件路径 | 二进制输入 | +| 逐字节验证 | 临时文件 | 临时文件 | 内存结果 | +| 旧桥接架构 | 支持 | 支持 | 不适用 | +| TurboModule / 新架构 | 支持 | 支持 | 不适用 | | 后台执行 | 串行 executor | 串行 worker | 模块 Web Worker | -| 补丁格式 | `ENDSLEY/BSDIFF43` | 同左 | 同左 | +| 补丁格式 | `ENDSLEY/BSDIFF43` | 同左 | 同左 | 示例应用使用 React Native 0.86.0，并在 Android API 24/31 与 iOS Simulator 验证新架构运行时。直接兼容 fixture 会针对 React Native 0.73.11、0.74.7 编译 @@ -29,6 +31,8 @@ Android 根据 React Native 次版本选择新架构包实现： 原生操作运行在模块自有的单线程 executor 上。job registry 负责排队/运行中取消、 限频进度、资源限制和清理。项目内置 C 代码通过 CMake 构建，并由 JNI 调用。 +`inspectPatch` 只读取补丁头；`verifyPatch` 将结果写入缓存临时文件，分块比较预期 +路径，并在返回前删除临时输出。 ## iOS @@ -39,6 +43,8 @@ TurboModule 实例。 模块方法使用串行 dispatch queue，确保后续取消请求到达时 job 已经登记。耗时补丁 操作会异步进入独立的串行工作队列，在保持桥接响应的同时维护共享 C 核心的单操作 边界。模块失效时会取消并清理 job registry。 +元数据检查只读取 24 字节补丁头。验证使用系统临时目录中的唯一文件，并在所有退出 +路径上清理。 ## React Native Web @@ -67,6 +73,8 @@ Web 入口面向浏览器，不是 Node.js 文件系统适配器；它不会在 未传 `AbortSignal` 的调用共用模块 Worker 与已初始化的 WebAssembly 模块；带 signal 的调用使用专用 Worker，保证取消只影响当前任务。两种路径都会在各自 Worker 内串行执行，但调用方仍应设置应用级内存预算。 +`inspectPatch` 不会启动 Worker。`verifyPatch` 先验证元数据，再复用 `patchBytes` +的 Worker 路径，并在与预期输入比较后丢弃还原缓冲区。 ## 服务端渲染 diff --git a/example/src/App.tsx b/example/src/App.tsx index a0316af..6068be3 100644 --- a/example/src/App.tsx +++ b/example/src/App.tsx @@ -3,10 +3,12 @@ import * as React from 'react'; import { StyleSheet, View, Text } from 'react-native'; import { diff, + inspectPatch, patch, startDiff, startPatch, type NativeOperationProgress, + verifyPatch, } from 'react-native-bs-diff-patch'; import * as FS from 'react-native-fs'; @@ -112,6 +114,43 @@ export default function App() { crossPlatformFixture.patchBase64, 'base64' ); + const goldenNewFileInfo = await FS.stat(goldenNewFile); + const goldenMetadata = await inspectPatch(goldenPatchFile, { + maxInputBytes: 1024 * 1024, + }); + if ( + !goldenMetadata.valid || + goldenMetadata.format !== 'ENDSLEY/BSDIFF43' || + goldenMetadata.declaredTargetBytes !== String(goldenNewFileInfo.size) + ) { + throw new Error('native patch metadata did not match the fixture'); + } + const goldenVerification = await verifyPatch( + goldenOldFile, + goldenPatchFile, + goldenNewFile, + { + maxInputBytes: 1024 * 1024, + maxOutputBytes: 1024 * 1024, + } + ); + if ( + !goldenVerification.verified || + goldenVerification.patch.declaredTargetBytes !== + String(goldenNewFileInfo.size) + ) { + throw new Error( + 'native patch verification did not match the fixture' + ); + } + const mismatchVerification = await verifyPatch( + goldenOldFile, + goldenPatchFile, + oldFile + ); + if (mismatchVerification.verified) { + throw new Error('native patch verification accepted a wrong target'); + } await diff(goldenOldFile, goldenNewFile, generatedGoldenPatchFile); const generatedGoldenPatch = await FS.readFile( generatedGoldenPatchFile, @@ -137,6 +176,44 @@ export default function App() { 'bm90IGEgYnNkaWZmIHBhdGNo', 'base64' ); + const corruptMetadata = await inspectPatch(corruptPatchFile); + if ( + corruptMetadata.valid || + corruptMetadata.issue !== 'TRUNCATED_HEADER' + ) { + throw new Error('native patch inspection accepted a corrupt patch'); + } + const corruptVerificationCode = await verifyPatch( + goldenOldFile, + corruptPatchFile, + goldenNewFile + ).then( + () => undefined, + (error: { code?: string }) => error.code + ); + if (corruptVerificationCode !== 'EPATCH') { + throw new Error( + `corrupt verification should reject with EPATCH, got ${String( + corruptVerificationCode + )}` + ); + } + const verificationLimitCode = await verifyPatch( + goldenOldFile, + goldenPatchFile, + goldenNewFile, + { maxOutputBytes: 1 } + ).then( + () => undefined, + (error: { code?: string }) => error.code + ); + if (verificationLimitCode !== 'ERESOURCE') { + throw new Error( + `verification limit should reject with ERESOURCE, got ${String( + verificationLimitCode + )}` + ); + } let corruptPatchErrorCode: string | undefined; try { await patch(goldenOldFile, corruptOutputFile, corruptPatchFile); diff --git a/ios/BsDiffPatch.mm b/ios/BsDiffPatch.mm index 88567e1..8982bd0 100644 --- a/ios/BsDiffPatch.mm +++ b/ios/BsDiffPatch.mm @@ -170,6 +170,123 @@ - (NSString *)normalizedPath:(NSString *)path return [path hasPrefix:@"file://"] ? [path substringFromIndex:7] : path; } +- (NSDictionary *)patchMetadataAtPath:(NSString *)patchFile + error:(NSError **)error +{ + NSFileManager *fileManager = [NSFileManager defaultManager]; + NSDictionary *attributes = [fileManager attributesOfItemAtPath:patchFile error:error]; + if (attributes == nil) return nil; + + unsigned long long patchBytes = [attributes fileSize]; + NSFileHandle *handle = [NSFileHandle fileHandleForReadingAtPath:patchFile]; + if (handle == nil) { + if (error != nullptr) { + *error = [NSError errorWithDomain:@"BsDiffPatch" + code:1 + userInfo:@{NSLocalizedDescriptionKey : @"could not read patchFile"}]; + } + return nil; + } + NSData *header = [handle readDataOfLength:24]; + [handle closeFile]; + const uint8_t *bytes = static_cast(header.bytes); + NSUInteger headerBytes = header.length; + NSString *legacyMagic = [[NSString alloc] + initWithBytes:bytes + length:MIN((NSUInteger)8, headerBytes) + encoding:NSASCIIStringEncoding] ?: @""; + NSString *currentMagic = [[NSString alloc] + initWithBytes:bytes + length:MIN((NSUInteger)16, headerBytes) + encoding:NSASCIIStringEncoding] ?: @""; + + NSString *format = @"UNKNOWN"; + NSString *issue = nil; + NSString *declaredTargetBytes = nil; + BOOL valid = NO; + if (headerBytes < 24) { + if ([legacyMagic isEqualToString:@"BSDIFF40"]) { + format = @"BSDIFF40"; + issue = @"LEGACY_FORMAT"; + } else { + issue = @"TRUNCATED_HEADER"; + } + } else if (![currentMagic isEqualToString:@"ENDSLEY/BSDIFF43"]) { + if ([legacyMagic isEqualToString:@"BSDIFF40"]) { + format = @"BSDIFF40"; + issue = @"LEGACY_FORMAT"; + } else { + issue = @"INVALID_MAGIC"; + } + } else if ((bytes[23] & 0x80) != 0) { + format = @"ENDSLEY/BSDIFF43"; + issue = @"INVALID_TARGET_SIZE"; + } else { + unsigned long long targetBytes = 0; + for (NSInteger index = 23; index >= 16; index--) { + targetBytes = targetBytes * 256 + bytes[index]; + } + format = @"ENDSLEY/BSDIFF43"; + declaredTargetBytes = [NSString stringWithFormat:@"%llu", targetBytes]; + valid = YES; + } + + NSMutableDictionary *metadata = [@{ + @"format" : format, + @"patchBytes" : @(patchBytes), + @"headerBytes" : @(headerBytes), + @"payloadBytes" : @(patchBytes > 24 ? patchBytes - 24 : 0), + @"declaredTargetBytes" : declaredTargetBytes ?: [NSNull null], + @"valid" : @(valid) + } mutableCopy]; + if (issue != nil) metadata[@"issue"] = issue; + return metadata; +} + +- (NSString *)JSONStringForObject:(id)object error:(NSError **)error +{ + NSData *data = [NSJSONSerialization dataWithJSONObject:object options:0 error:error]; + if (data == nil) return nil; + return [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding]; +} + +- (BOOL)fileAtPath:(NSString *)firstPath equalsFileAtPath:(NSString *)secondPath +{ + NSDictionary *firstAttributes = [[NSFileManager defaultManager] + attributesOfItemAtPath:firstPath + error:nil]; + NSDictionary *secondAttributes = [[NSFileManager defaultManager] + attributesOfItemAtPath:secondPath + error:nil]; + if (firstAttributes == nil || secondAttributes == nil || + [firstAttributes fileSize] != [secondAttributes fileSize]) { + return NO; + } + + NSFileHandle *first = [NSFileHandle fileHandleForReadingAtPath:firstPath]; + NSFileHandle *second = [NSFileHandle fileHandleForReadingAtPath:secondPath]; + if (first == nil || second == nil) { + [first closeFile]; + [second closeFile]; + return NO; + } + BOOL equal = YES; + while (YES) { + @autoreleasepool { + NSData *firstChunk = [first readDataOfLength:64 * 1024]; + NSData *secondChunk = [second readDataOfLength:64 * 1024]; + if (![firstChunk isEqualToData:secondChunk]) { + equal = NO; + break; + } + if (firstChunk.length == 0) break; + } + } + [first closeFile]; + [second closeFile]; + return equal; +} + - (BOOL)validateOldFile:(NSString *)oldFile newFile:(NSString *)newFile patchFile:(NSString *)patchFile @@ -399,6 +516,166 @@ - (void)runJob:(NSString *)operation }); } +RCT_EXPORT_METHOD(inspectPatch:(NSString *)patchFile + maxInputBytes:(double)maxInputBytes + resolve:(RCTPromiseResolveBlock)resolve + reject:(RCTPromiseRejectBlock)reject) +{ + if (patchFile == nil || patchFile.length == 0) { + reject(@"EINVAL", @"patchFile should not be nil or empty", nil); + return; + } + if (![self validateLimit:maxInputBytes name:@"maxInputBytes" reject:reject]) return; + patchFile = [self normalizedPath:patchFile]; + if (![[NSFileManager defaultManager] fileExistsAtPath:patchFile]) { + reject(@"ENOENT", [NSString stringWithFormat:@"patchFile: %@ does not exist", patchFile], nil); + return; + } + + dispatch_async(operationQueue(), ^{ + NSError *error = nil; + NSDictionary *attributes = [[NSFileManager defaultManager] + attributesOfItemAtPath:patchFile + error:&error]; + if (attributes == nil) { + reject(@"ENOENT", error.localizedDescription ?: @"could not inspect patchFile", error); + return; + } + if (maxInputBytes > 0 && [attributes fileSize] > (unsigned long long)maxInputBytes) { + reject( + @"ERESOURCE", + [NSString stringWithFormat: + @"patchData is %llu bytes and exceeds the configured %.0f byte limit", + [attributes fileSize], + maxInputBytes], + nil); + return; + } + NSDictionary *metadata = [self patchMetadataAtPath:patchFile error:&error]; + NSString *json = metadata == nil + ? nil + : [self JSONStringForObject:metadata error:&error]; + if (json == nil) { + reject(@"EUNSPECIFIED", error.localizedDescription ?: @"could not encode patch metadata", error); + return; + } + resolve(json); + }); +} + +RCT_EXPORT_METHOD(verifyPatch:(NSString *)oldFile + patchFile:(NSString *)patchFile + expectedFile:(NSString *)expectedFile + maxInputBytes:(double)maxInputBytes + maxOutputBytes:(double)maxOutputBytes + resolve:(RCTPromiseResolveBlock)resolve + reject:(RCTPromiseRejectBlock)reject) +{ + if (oldFile == nil || oldFile.length == 0 || + patchFile == nil || patchFile.length == 0 || + expectedFile == nil || expectedFile.length == 0) { + reject(@"EINVAL", @"oldFile, patchFile, and expectedFile should not be nil or empty", nil); + return; + } + if (![self validateLimit:maxInputBytes name:@"maxInputBytes" reject:reject] || + ![self validateLimit:maxOutputBytes name:@"maxOutputBytes" reject:reject]) { + return; + } + oldFile = [self normalizedPath:oldFile]; + patchFile = [self normalizedPath:patchFile]; + expectedFile = [self normalizedPath:expectedFile]; + if ([oldFile isEqualToString:patchFile] || + [oldFile isEqualToString:expectedFile] || + [patchFile isEqualToString:expectedFile]) { + reject(@"EINVAL", @"oldFile, patchFile, and expectedFile should not be the same", nil); + return; + } + NSFileManager *fileManager = [NSFileManager defaultManager]; + for (NSString *path in @[ oldFile, patchFile, expectedFile ]) { + if (![fileManager fileExistsAtPath:path]) { + reject(@"ENOENT", [NSString stringWithFormat:@"input: %@ does not exist", path], nil); + return; + } + } + + dispatch_async(operationQueue(), ^{ + NSError *error = nil; + NSDictionary *oldAttributes = [fileManager attributesOfItemAtPath:oldFile error:&error]; + NSDictionary *patchAttributes = [fileManager attributesOfItemAtPath:patchFile error:&error]; + NSDictionary *expectedAttributes = [fileManager attributesOfItemAtPath:expectedFile error:&error]; + if (oldAttributes == nil || patchAttributes == nil || expectedAttributes == nil) { + reject(@"ENOENT", error.localizedDescription ?: @"could not read input metadata", error); + return; + } + if (maxInputBytes > 0 && + ([oldAttributes fileSize] > (unsigned long long)maxInputBytes || + [patchAttributes fileSize] > (unsigned long long)maxInputBytes || + [expectedAttributes fileSize] > (unsigned long long)maxInputBytes)) { + reject(@"ERESOURCE", @"verification input exceeds maxInputBytes", nil); + return; + } + + NSDictionary *metadata = [self patchMetadataAtPath:patchFile error:&error]; + if (metadata == nil) { + reject(@"EPATCH", error.localizedDescription ?: @"could not inspect patchFile", error); + return; + } + if (![metadata[@"valid"] boolValue]) { + reject( + @"EPATCH", + [NSString stringWithFormat: + @"patch structure is invalid: %@", + metadata[@"issue"] ?: @"UNKNOWN"], + nil); + return; + } + unsigned long long declaredTargetBytes = + [metadata[@"declaredTargetBytes"] longLongValue]; + if (maxOutputBytes > 0 && declaredTargetBytes > (unsigned long long)maxOutputBytes) { + reject(@"ERESOURCE", @"declared output exceeds maxOutputBytes", nil); + return; + } + + NSString *outputFile = [NSTemporaryDirectory() + stringByAppendingPathComponent:[NSString stringWithFormat: + @"bsdiffpatch-verify-%@.tmp", + [NSUUID UUID].UUIDString]]; + int result = bsdiffpatch::patchFile( + [oldFile UTF8String], [outputFile UTF8String], [patchFile UTF8String]); + if (result != 0) { + [fileManager removeItemAtPath:outputFile error:nil]; + reject(@"EPATCH", [NSString stringWithFormat:@"patch verification failed with native result %d", result], nil); + return; + } + + NSDictionary *outputAttributes = [fileManager attributesOfItemAtPath:outputFile error:&error]; + if (outputAttributes == nil) { + [fileManager removeItemAtPath:outputFile error:nil]; + reject(@"EPATCH", error.localizedDescription ?: @"could not read restored output", error); + return; + } + if (maxOutputBytes > 0 && [outputAttributes fileSize] > (unsigned long long)maxOutputBytes) { + [fileManager removeItemAtPath:outputFile error:nil]; + reject(@"ERESOURCE", @"restored output exceeds maxOutputBytes", nil); + return; + } + BOOL verified = [self fileAtPath:outputFile equalsFileAtPath:expectedFile]; + NSDictionary *verification = @{ + @"verified" : @(verified), + @"restoredBytes" : @([outputAttributes fileSize]), + @"expectedBytes" : @([expectedAttributes fileSize]), + @"patch" : metadata + }; + NSString *json = [self JSONStringForObject:verification error:&error]; + [fileManager removeItemAtPath:outputFile error:nil]; + if (json == nil) { + reject(@"EUNSPECIFIED", error.localizedDescription ?: @"could not encode verification result", error); + return; + } + resolve(json); + }); +} + RCT_EXPORT_METHOD(startPatch:(NSString *)jobId oldFile:(NSString *)oldFile newFile:(NSString *)newFile diff --git a/package.json b/package.json index b3e02fb..d9748c1 100644 --- a/package.json +++ b/package.json @@ -57,6 +57,8 @@ "test:native-operations": "sh scripts/test-native-operations.sh", "benchmark:web": "node scripts/benchmark-web.mjs", "benchmark:native": "node scripts/benchmark-native.mjs", + "benchmark:large:web": "BENCHMARK_SIZES_MIB=16,64,128 BENCHMARK_ALLOW_FAILURES=1 node scripts/benchmark-web.mjs", + "benchmark:large:native": "BENCHMARK_SIZES_MIB=16,64,128 BENCHMARK_ALLOW_FAILURES=1 node scripts/benchmark-native.mjs", "site:build": "node scripts/build-site.mjs", "site:test": "node scripts/test-site.mjs", "site:test:browser": "node scripts/test-site-browser.mjs", diff --git a/scripts/benchmark-native.mjs b/scripts/benchmark-native.mjs index a61a579..50afea0 100644 --- a/scripts/benchmark-native.mjs +++ b/scripts/benchmark-native.mjs @@ -62,9 +62,22 @@ try { executable, ]); - const results = sizes.map((size) => - JSON.parse(run(executable, [String(size)])) - ); + const results = sizes.map((sizeMiB) => { + try { + return { + status: 'passed', + ...JSON.parse(run(executable, [String(sizeMiB)])), + }; + } catch (error) { + return { + sizeMiB, + status: 'failed', + errorCode: 'ENATIVE', + errorMessage: error instanceof Error ? error.message : String(error), + }; + } + }); + const failed = results.filter((result) => result.status === 'failed').length; const report = { generatedAt: new Date().toISOString(), runtime: { @@ -76,6 +89,10 @@ try { description: 'Deterministic files with one changed byte per 4 KiB', memoryMetric: 'Peak resident set size for one diff and patch process', }, + summary: { + passed: results.length - failed, + failed, + }, results, }; const serialized = `${JSON.stringify(report, null, 2)}\n`; @@ -83,6 +100,9 @@ try { await writeFile(process.env.BENCHMARK_OUTPUT, serialized); } process.stdout.write(serialized); + if (failed > 0 && process.env.BENCHMARK_ALLOW_FAILURES !== '1') { + process.exitCode = 1; + } } finally { await rm(temporaryDirectory, { recursive: true, force: true }); } diff --git a/scripts/benchmark-web.mjs b/scripts/benchmark-web.mjs index d08bdee..1d2230c 100644 --- a/scripts/benchmark-web.mjs +++ b/scripts/benchmark-web.mjs @@ -1,3 +1,5 @@ +import { spawnSync } from 'node:child_process'; +import { fileURLToPath } from 'node:url'; import { cpus } from 'node:os'; import { writeFile } from 'node:fs/promises'; import { performance } from 'node:perf_hooks'; @@ -5,6 +7,7 @@ import { performance } from 'node:perf_hooks'; import { runOperation } from '../web/operations.mjs'; const mebibyte = 1024 * 1024; +const scriptPath = fileURLToPath(import.meta.url); const sizes = (process.env.BENCHMARK_SIZES_MIB || '1,10,50') .split(',') .map((value) => Number(value.trim())); @@ -29,52 +32,116 @@ function createInputs(size) { return { oldData, newData }; } -const warmup = createInputs(64 * 1024); -const initializationStartedAt = performance.now(); -await runOperation('diff', warmup.oldData, warmup.newData); -const initializationMs = performance.now() - initializationStartedAt; -const results = []; +function toMiB(bytes) { + return Number((bytes / mebibyte).toFixed(1)); +} -for (const sizeMiB of sizes) { - const { oldData, newData } = createInputs(sizeMiB * mebibyte); - const diffStartedAt = performance.now(); - const patchData = await runOperation('diff', oldData, newData); - const diffMs = performance.now() - diffStartedAt; - const patchStartedAt = performance.now(); - const restoredData = await runOperation('patch', oldData, patchData); - const patchMs = performance.now() - patchStartedAt; +async function runSample(sizeMiB) { + try { + const warmup = createInputs(64 * 1024); + const initializationStartedAt = performance.now(); + await runOperation('diff', warmup.oldData, warmup.newData); + const initializationMs = performance.now() - initializationStartedAt; + const { oldData, newData } = createInputs(sizeMiB * mebibyte); + const diffStartedAt = performance.now(); + const patchData = await runOperation('diff', oldData, newData); + const diffMs = performance.now() - diffStartedAt; + const patchStartedAt = performance.now(); + const restoredData = await runOperation('patch', oldData, patchData); + const patchMs = performance.now() - patchStartedAt; - if ( - restoredData.byteLength !== newData.byteLength || - !restoredData.every((value, index) => value === newData[index]) - ) { - throw new Error(`Benchmark round trip failed for ${sizeMiB} MiB`); - } + if ( + restoredData.byteLength !== newData.byteLength || + !restoredData.every((value, index) => value === newData[index]) + ) { + throw new Error(`Benchmark round trip failed for ${sizeMiB} MiB`); + } - results.push({ - sizeMiB, - diffMs: Number(diffMs.toFixed(1)), - patchMs: Number(patchMs.toFixed(1)), - patchBytes: patchData.byteLength, - }); + const memory = process.memoryUsage(); + return { + sizeMiB, + status: 'passed', + initializationMs: Number(initializationMs.toFixed(1)), + diffMs: Number(diffMs.toFixed(1)), + patchMs: Number(patchMs.toFixed(1)), + patchBytes: patchData.byteLength, + peakRssMiB: toMiB(process.resourceUsage().maxRSS * 1024), + residentAfterMiB: toMiB(memory.rss), + externalAfterMiB: toMiB(memory.external), + arrayBuffersAfterMiB: toMiB(memory.arrayBuffers), + }; + } catch (error) { + return { + sizeMiB, + status: 'failed', + errorCode: error?.code || 'EBENCHMARK', + errorMessage: error instanceof Error ? error.message : String(error), + peakRssMiB: toMiB(process.resourceUsage().maxRSS * 1024), + }; + } } -const report = { - generatedAt: new Date().toISOString(), - runtime: { - cpu: cpus()[0]?.model || 'unknown', - node: process.version, - platform: `${process.platform}-${process.arch}`, - }, - workload: { - description: 'Deterministic buffers with one changed byte per 4 KiB', - initializationMs: Number(initializationMs.toFixed(1)), - }, - results, -}; -const serializedReport = `${JSON.stringify(report, null, 2)}\n`; +async function main() { + if (process.argv[2] === '--sample') { + const sizeMiB = Number(process.argv[3]); + if (!Number.isSafeInteger(sizeMiB) || sizeMiB <= 0 || sizeMiB > 512) { + throw new Error('Sample size must be an integer from 1 to 512 MiB'); + } + process.stdout.write(`${JSON.stringify(await runSample(sizeMiB))}\n`); + return; + } + + const results = sizes.map((sizeMiB) => { + const sample = spawnSync( + process.execPath, + [scriptPath, '--sample', String(sizeMiB)], + { + encoding: 'utf8', + env: { ...process.env, BENCHMARK_OUTPUT: '' }, + maxBuffer: 10 * mebibyte, + } + ); + if (sample.status !== 0 || !sample.stdout.trim()) { + return { + sizeMiB, + status: 'failed', + errorCode: 'EPROCESS', + errorMessage: (sample.stderr || sample.stdout || 'process failed') + .trim() + .slice(0, 2000), + }; + } + return JSON.parse(sample.stdout); + }); + const failed = results.filter((result) => result.status === 'failed').length; + const report = { + generatedAt: new Date().toISOString(), + runtime: { + cpu: cpus()[0]?.model || 'unknown', + node: process.version, + platform: `${process.platform}-${process.arch}`, + }, + workload: { + description: 'Deterministic buffers with one changed byte per 4 KiB', + memoryMetric: + 'Peak resident set size for one initialized WebAssembly diff and patch process', + processIsolation: 'Each input size runs in a fresh Node.js process', + }, + summary: { + passed: results.length - failed, + failed, + }, + results, + }; + const serializedReport = `${JSON.stringify(report, null, 2)}\n`; -if (process.env.BENCHMARK_OUTPUT) { - await writeFile(process.env.BENCHMARK_OUTPUT, serializedReport); + if (process.env.BENCHMARK_OUTPUT) { + await writeFile(process.env.BENCHMARK_OUTPUT, serializedReport); + } + process.stdout.write(serializedReport); + if (failed > 0 && process.env.BENCHMARK_ALLOW_FAILURES !== '1') { + process.exitCode = 1; + } } -process.stdout.write(serializedReport); + +await main(); diff --git a/scripts/build-site.mjs b/scripts/build-site.mjs index 3158ff1..40571d6 100644 --- a/scripts/build-site.mjs +++ b/scripts/build-site.mjs @@ -51,6 +51,13 @@ const pages = [ 'Native resource limits, cancellation, progress, and atomic output behavior.', file: 'native-operations-v03.md', }, + { + slug: 'large-files-v04', + title: 'Large-file roadmap', + description: + 'Memory baselines, honest progress semantics, and streaming feasibility for larger inputs.', + file: 'large-files-v04.md', + }, { slug: 'troubleshooting', title: 'Troubleshooting', @@ -104,6 +111,12 @@ const chinesePages = [ description: '原生资源限制、取消、进度与原子输出行为。', file: 'native-operations-v03.md', }, + { + slug: 'large-files-v04', + title: '大文件演进路线', + description: '面向更大输入的内存基线、真实进度语义与流式处理可行性。', + file: 'large-files-v04.md', + }, { slug: 'troubleshooting', title: '常见问题与排障', @@ -670,8 +683,8 @@ function localizeHomepage(source) { ['New Architecture', '新架构'], ['Current matrix anchor', '当前矩阵基准'], ['npm tarball', 'npm 安装包'], - ['125 KiB packed', '125 KiB 压缩包'], - ['459 KiB unpacked · 58 files', '459 KiB 解压后 · 58 个文件'], + ['133 KiB packed', '133 KiB 压缩包'], + ['500 KiB unpacked · 58 files', '500 KiB 解压后 · 58 个文件'], ['WebAssembly reference', 'WebAssembly 参考数据'], ['Repeatable, not theoretical.', '可复现，而非纸面数据。'], [ diff --git a/scripts/test-package-consumers.mjs b/scripts/test-package-consumers.mjs index e993855..cc3c6bb 100644 --- a/scripts/test-package-consumers.mjs +++ b/scripts/test-package-consumers.mjs @@ -232,8 +232,9 @@ try { await writeFile( path.join(consumerDirectory, 'browser.mjs'), [ - "import { diffBytes, patch } from 'react-native-bs-diff-patch';", + "import { diffBytes, inspectPatch, patch } from 'react-native-bs-diff-patch';", "if (typeof diffBytes !== 'function') throw new Error('Missing Web binary API');", + "if (typeof inspectPatch !== 'function') throw new Error('Missing patch metadata API');", "const error = await patch('old', 'new', 'patch').catch((value) => value);", "if (!error || error.code !== 'EUNSUPPORTED') throw new Error('Wrong Web path API');", ].join('\n') @@ -245,9 +246,11 @@ try { await writeFile( path.join(consumerDirectory, 'consumer.ts'), [ - "import { diffBytes, type BinaryInput } from 'react-native-bs-diff-patch';", + "import { diffBytes, inspectPatch, verifyPatch, type BinaryInput, type PatchMetadata } from 'react-native-bs-diff-patch';", 'const input: BinaryInput = new Uint8Array([1, 2, 3]);', 'void diffBytes(input, input);', + 'void inspectPatch(input).then((value: PatchMetadata) => value.valid);', + 'void verifyPatch(input, input, input).then((value) => value.verified);', ].join('\n') ); run( diff --git a/scripts/test-site-browser.mjs b/scripts/test-site-browser.mjs index ab4ac06..15004b7 100644 --- a/scripts/test-site-browser.mjs +++ b/scripts/test-site-browser.mjs @@ -220,6 +220,57 @@ try { 'ENDSLEY/BSDIFF43' ); + await selectFile('#manifest-old-file', fixtures.oldBytes, 'release-v1.bin'); + await selectFile('#manifest-new-file', fixtures.newBytes, 'release-v2.bin'); + await selectFile('#manifest-patch-file', fixtures.patch, 'release-v2.patch'); + await page.click('#manifest-run'); + await page.waitForSelector('#manifest-status[data-state="success"]'); + const generatedManifest = JSON.parse( + await page.$eval('#manifest-output', (element) => element.textContent || '') + ); + assert.equal(generatedManifest.manifestVersion, 1); + assert.equal(generatedManifest.patchFormat, 'ENDSLEY/BSDIFF43'); + assert.equal(generatedManifest.target.bytes, fixtures.newBytes.length); + assert.equal(generatedManifest.target.sha256.length, 64); + assert.equal(generatedManifest.patch.sha256.length, 64); + + await page.$eval('#savings-target-mib', (element) => { + element.value = '10'; + element.dispatchEvent(new Event('input', { bubbles: true })); + }); + await page.$eval('#savings-patch-mib', (element) => { + element.value = '2'; + element.dispatchEvent(new Event('input', { bubbles: true })); + }); + await page.$eval('#savings-downloads', (element) => { + element.value = '5'; + element.dispatchEvent(new Event('input', { bubbles: true })); + }); + assert.equal( + await page.$eval('#savings-rate', (element) => element.textContent), + '80.0%' + ); + assert.equal( + await page.$eval('#savings-equivalent', (element) => element.textContent), + '4' + ); + + await page.select('#diagnostic-code', 'ERESOURCE'); + assert.match( + await page.$eval( + '#diagnostic-action', + (element) => element.textContent || '' + ), + /maxInputBytes/ + ); + assert.match( + await page.$eval( + '#diagnostic-example', + (element) => element.textContent || '' + ), + /maxOutputBytes/ + ); + await page.setViewport({ width: 390, height: 844, deviceScaleFactor: 1 }); await page.reload({ waitUntil: 'networkidle0' }); const toolsMobile = await page.evaluate(() => ({ diff --git a/scripts/test-site.mjs b/scripts/test-site.mjs index d7fcf6b..6364ef8 100644 --- a/scripts/test-site.mjs +++ b/scripts/test-site.mjs @@ -37,6 +37,8 @@ const requiredFiles = [ 'docs/recipes/index.html', 'docs/platform-support/index.html', 'docs/architecture/index.html', + 'docs/native-operations-v03/index.html', + 'docs/large-files-v04/index.html', 'docs/troubleshooting/index.html', 'docs/development/index.html', 'docs/zh-CN/index.html', @@ -45,6 +47,8 @@ const requiredFiles = [ 'docs/zh-CN/recipes/index.html', 'docs/zh-CN/platform-support/index.html', 'docs/zh-CN/architecture/index.html', + 'docs/zh-CN/native-operations-v03/index.html', + 'docs/zh-CN/large-files-v04/index.html', 'docs/zh-CN/troubleshooting/index.html', 'docs/zh-CN/development/index.html', ]; @@ -200,8 +204,8 @@ assert.match(homepage, /EINPUT_TOO_LARGE/); assert.match(homepage, /id="evidence"/); assert.match(homepage, /RN 0\.73\.11/); assert.match(homepage, /RN 0\.86\.0/); -assert.match(homepage, /125 KiB packed/); -assert.match(homepage, /459 KiB unpacked · 58 files/); +assert.match(homepage, /133 KiB packed/); +assert.match(homepage, /500 KiB unpacked · 58 files/); assert.match(homepage, /30,697\.5 ms/); assert.match(homepage, /assets\/playground\.js/); @@ -232,6 +236,9 @@ assert.match( assert.match(toolsPage, /id="create-panel"/); assert.match(toolsPage, /id="apply-panel"/); assert.match(toolsPage, /id="inspect-panel"/); +assert.match(toolsPage, /id="manifest-run"/); +assert.match(toolsPage, /id="savings-target-mib"/); +assert.match(toolsPage, /id="diagnostic-code"/); assert.match(toolsPage, /id="apply-expected-sha256"/); assert.match(toolsPage, /id="tool-code"/); assert.match(toolsPage, /assets\/tools\.js/); @@ -247,6 +254,9 @@ assert.match( / 24); console.log('Browser Web Worker diff/patch round trip passed'); diff --git a/scripts/test-web.mjs b/scripts/test-web.mjs index 9def537..7d967cf 100644 --- a/scripts/test-web.mjs +++ b/scripts/test-web.mjs @@ -4,6 +4,7 @@ import path from 'node:path'; import { fileURLToPath } from 'node:url'; import { runOperation } from '../web/operations.mjs'; +import { inspectPatch } from '../web/index.mjs'; const scriptDirectory = path.dirname(fileURLToPath(import.meta.url)); const fixture = JSON.parse( @@ -33,6 +34,24 @@ assert.deepEqual( newData, 'patch should reconstruct the new bytes' ); +const metadata = await inspectPatch(patchData); +assert.deepEqual(metadata, { + declaredTargetBytes: String(newData.byteLength), + format: 'ENDSLEY/BSDIFF43', + headerBytes: 24, + patchBytes: patchData.byteLength, + payloadBytes: patchData.byteLength - 24, + valid: true, +}); +assert.deepEqual(await inspectPatch(new Uint8Array([1, 2, 3])), { + declaredTargetBytes: null, + format: 'UNKNOWN', + headerBytes: 3, + issue: 'TRUNCATED_HEADER', + patchBytes: 3, + payloadBytes: 0, + valid: false, +}); const goldenOldData = new Uint8Array(Buffer.from(fixture.oldBase64, 'base64')); const goldenNewData = new Uint8Array(Buffer.from(fixture.newBase64, 'base64')); diff --git a/scripts/web-test.html b/scripts/web-test.html index e07ba88..e7c0bad 100644 --- a/scripts/web-test.html +++ b/scripts/web-test.html @@ -11,7 +11,13 @@

BsDiffPatch Web Test

Running…