Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,27 @@ Development contracts:

Portable archive trust is scope-aware. Unfiltered exports set `scope.represents_entire_source_database` and preserve completeness, indexed-HEAD, run, and unknown-extension metadata. Filtered exports normalize only the archive snapshot to `index_complete=false` with `partial_archive`, remove source-wide HEAD/run provenance, and omit unavailable unknown-extension summaries. Import repeats that normalization for legacy manifests without scope metadata, while current full-snapshot manifests preserve trust. Scoped indexing of a partial archive falls back to a full workspace scan before it may clear `partial_archive`.

Portable archive path privacy is opt-in for compatibility. Default exports retain
`manifest.project_root`, the snapshot's `indexed_project_root`, requested scope
values, and resolved success paths. `--redact-paths` must operate only on the
private copied snapshot: resolve and apply scope first, delete the copied project
root, replace absolute POSIX/Windows/file-URI scope values and persisted
flat and grouped path-sample values with `[redacted]`, and fail closed by deleting
or emptying malformed or over-budget path metadata from the copy. Losing a
workspace-verification pending-path identity must also stamp its coverage marker
incomplete. Run exactly one final
`VACUUM` after scope and redaction before computing
`database_sha256`. Successful manifest and export JSON must keep
`path_redaction_requested`, `path_redaction_complete`, and bounded stable
`path_redaction_omitted_categories` synchronized. Redacted success output must
not repeat the resolved archive, database, or source-root paths. Repository-relative
indexed paths, source content, hashes, readiness, and commit provenance are not
redaction targets. Import derives its destination project root from the destination
DB path/current directory and must tolerate an absent source root in both execution
and dry-run modes. Imports accepting `path_redaction_complete=true` must verify
the manifest root/scope values and known embedded path metadata before reporting
that claim as complete.

Checkpoint plan drift detection covers DB/WAL/SHM content changes and sidecar appearance or disappearance through the final pre-publication validation. The plan's `uncertainty` value records the remaining post-validation race; copied outputs are independently hash-verified against the plan before atomic publication. DB/WAL/SHM candidates must pass native regular-file type validation before hashing so Unix FIFOs cannot block planning. `metadata_policy` reports `owner_only_files_and_directories` on POSIX and the actually inherited `inherited_windows_acls` policy on Windows. A database payload whose output name collides with `manifest.txt`, including filesystem-equivalent casing, makes the plan not ready and is rejected before mutation.

## Filesystem Permissions
Expand Down Expand Up @@ -4245,6 +4266,24 @@ net9 CI lane に合わせる場合は `FRAMEWORK=net9.0 make test` を使いま

portable archive の trust は scope を考慮します。filter なし export は `scope.represents_entire_source_database` を設定し、completeness、indexed-HEAD、run、unknown-extension metadata を維持します。filter 済み export は archive snapshot だけを `index_complete=false` / `partial_archive` に正規化し、source 全体に対する HEAD / run provenance を削除して、未計測の unknown-extension summary を省略します。scope metadata がない legacy manifest は import 時に同じ正規化を行い、現行の full-snapshot manifest は trust を維持します。partial archive に対する scoped index は `partial_archive` を解除する前に full workspace scan へ fallback します。

portable archive の path privacy は互換性のため opt-in です。既定 export は
`manifest.project_root`、snapshot の `indexed_project_root`、指定 scope value、解決済み
success path を保持します。`--redact-paths` は private な copy 済み snapshot だけを変更します。
最初に scope を解決・適用し、copy 側の project root を削除して、POSIX / Windows /
file URI 形式の絶対 scope value と永続化済み path sample を `[redacted]` に置換します。
flat / group 別 sample の両方を対象とし、不正または上限超過の path metadata は fail-closed
として copy から削除するか空にします。workspace verification の pending-path identity を失う場合は、
coverage marker も incomplete に stamp します。scope と redaction の後に最終 `VACUUM` を一度だけ実行してから
`database_sha256` を計算します。成功時の manifest と
export JSON では `path_redaction_requested`、`path_redaction_complete`、上限付きで安定した
`path_redaction_omitted_categories` を同期させます。redacted success output は解決済み
archive / database / source-root path を再表示してはいけません。repository-relative な
indexed path、source content、hash、readiness、commit provenance は redaction 対象外です。
import は destination DB path / current directory から destination project root を導出し、
source root がない archive を execution / dry-run の双方で許容する必要があります。
`path_redaction_complete=true` を受け入れる import は、その claim を complete と報告する前に
manifest の root / scope value と既知の embedded path metadata を検証する必要があります。

checkpoint plan の drift 検出は、publish 前の最終検証までに起きる DB/WAL/SHM content の変更と sidecar の出現・消失を対象にします。plan の `uncertainty` 値は最終検証後に残る race を記録し、copy 済み output は atomic publish 前に plan の hash と個別に照合されます。DB/WAL/SHM 候補は hash 読み取り前に native regular-file type validation を通すため、Unix FIFO が plan 作成を停止させることはありません。`metadata_policy` は POSIX では `owner_only_files_and_directories`、Windows では実際に継承される `inherited_windows_acls` policy を報告します。database payload の output 名が filesystem 上で同一の大小文字を含め `manifest.txt` と衝突する場合、plan は not ready となり、変更前に拒否されます。

## ファイルシステム権限
Expand Down
4 changes: 4 additions & 0 deletions TESTING_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -445,6 +445,8 @@ Candidate-ordered parallel-index recovery tests must prove that the fatal result
keeps the overwrite boundary and artifact attestation deterministic. Preserve default refusal of regular and dangling-symlink destinations, explicit replacement with verifiable byte-size/SHA-256/manifest metadata and POSIX owner-only mode, failed-write temp cleanup, and a publish-boundary injection that creates a concurrent winner without using sleeps.
- `ExportImportCommandRunnerIssue5185Tests`
owns the import-staging confidentiality regression. Keep the non-dry-run parent at mode `0755`, set process umask `022` only inside the non-parallel SQLite-sensitive collection, observe the empty staging DB before extraction can write its first byte, and verify owner-only DB/WAL/SHM modes through validation, trust rewriting, `--prune-paths`, rollback-backup creation, cancellation cleanup, collision rejection, and final replacement. POSIX mode assertions must remain platform-conditional while success and `CreateNew` collision behavior continue to run on Windows.
- `ExportImportCommandRunnerIssue5195Tests`
owns portable-archive path-redaction coverage. Keep default path compatibility separate from the redacted full-archive fixture; the latter jointly verifies manifest and SQLite root removal, flat and grouped POSIX/Windows path samples, whitespace-wrapped absolute scope selectors, pending-path coverage demotion, hash-after-redaction ordering, source-byte immutability, owner-only atomic replacement, import/dry-run tolerance, and a post-import query. Preserve the separate fail-closed fixture for valid path-list metadata above 64 KiB plus malformed or over-budget path metadata, the forged completed-claim import rejection, and exact-case absolute project selection.
- `IndexCommandRunnerTests.Run_CancelDuringFreshIndex_ReturnsInterruptedJson`, `Run_CancelDuringDryRunScan_ReturnsInterruptedJson`, and `Run_CancelBeforeFreshScan_ReturnsInterruptedJson`
exercise the same in-process cancellation paths used after Ctrl-C/SIGINT wiring, including scan-time cancellation, so interrupted index runs keep returning the canonical JSON error contract.
- `IndexCommandRunnerTests.RunOptimizeFts_DryRunPreviewsWithoutWritingThenOptimizeMutates_Issue4577`, `RunOptimizeFts_DryRunPathDisclosureRequiresShowPaths_Issue5193`, `RunOptimizeFts_LockHeld_ReportsDbLocked`, and `RunOptimizeFts_ReadOnlyUri_ReturnsDbNotWritable`
Expand Down Expand Up @@ -1592,6 +1594,8 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests"
overwrite 境界と artifact attestation を deterministic に固定します。通常 file と dangling symlink destination の既定拒否、明示置換で検証可能な byte size / SHA-256 / manifest metadata と POSIX owner-only mode、書き込み失敗時の temp cleanup、sleep を使わず publish 境界へ concurrent winner を生成する injection を維持してください。
- `ExportImportCommandRunnerIssue5185Tests`
import staging の confidentiality 回帰を担当します。non-dry-run の parent を mode `0755` にし、process umask `022` は非並列の SQLite-sensitive collection 内だけで設定し、extraction が最初の byte を書く前の空 staging DB を観測してください。validation、trust rewrite、`--prune-paths`、rollback backup 作成、cancellation cleanup、衝突拒否、最終 replacement を通して DB/WAL/SHM が owner-only であることを検証します。POSIX mode assertion は platform 条件付きに保ちつつ、success と `CreateNew` の衝突挙動は Windows でも実行してください。
- `ExportImportCommandRunnerIssue5195Tests`
portable archive の path-redaction coverage を担当します。既定 path の互換性は redacted full-archive fixture と分離し、後者では manifest / SQLite の root 除去、flat / group 別の POSIX / Windows path sample、前後に空白を含む絶対 scope selector、pending-path coverage の demotion、redaction 後の hash 計算順、source byte 不変性、owner-only atomic replacement、import / dry-run の許容、import 後 query を一体で検証してください。64 KiB を超える正当な path-list metadata と不正または上限超過の path metadata を扱う fail-closed fixture、偽造された完了 claim の import 拒否、絶対 project 指定の正確な大小文字選択も維持してください。
- `IndexCommandRunnerTests.Run_CancelDuringFreshIndex_ReturnsInterruptedJson`、`Run_CancelDuringDryRunScan_ReturnsInterruptedJson`、`Run_CancelBeforeFreshScan_ReturnsInterruptedJson`
Ctrl-C/SIGINT 配線後に使われる in-process cancellation 経路を、scan 中のキャンセルも含めて検証し、interrupted index run が標準の JSON error contract を返し続けることを固定する。
- `IndexCommandRunnerTests.RunOptimizeFts_DryRunPreviewsWithoutWritingThenOptimizeMutates_Issue4577`、`RunOptimizeFts_DryRunPathDisclosureRequiresShowPaths_Issue5193`、`RunOptimizeFts_LockHeld_ReportsDbLocked`、`RunOptimizeFts_ReadOnlyUri_ReturnsDbNotWritable`
Expand Down
57 changes: 51 additions & 6 deletions USER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -542,6 +542,7 @@ job:
```bash
cdidx export codeindex.cdidx.zip
cdidx export codeindex.cdidx.zip --overwrite --json
cdidx export support.cdidx.zip --redact-paths --json
cdidx export app.cdidx.zip --project App --lang csharp --exclude-tests
cdidx export shared.cdidx.zip --path 'src/shared/*' --exclude-path 'src/shared/generated/*'
cdidx import codeindex.cdidx.zip
Expand Down Expand Up @@ -576,15 +577,37 @@ complete immutable `manifest` object. That manifest carries the database hash,
row counts, schema contract stamps, readiness state, unknown-extension summary,
and export scope needed to evaluate the artifact before import.

By default, compatibility archives retain `manifest.project_root` and the
embedded database's `indexed_project_root`, and successful human/JSON output
reports the resolved archive and database paths. Use `--redact-paths` before
sharing an archive outside the source machine. This opt-in mode removes the
project root from both manifest and snapshot, replaces absolute POSIX, Windows,
or file-URI scope values and unknown-extension path samples with `[redacted]`,
including the grouped samples exposed by `status`. Malformed or over-budget
path-sample metadata is removed or replaced with an empty fail-closed value instead
of being retained under a completed-redaction claim. If a workspace-verification
pending-path identity is removed or redacted, its coverage marker is set incomplete
so a later scoped refresh cannot trust the placeholder. The exporter then vacuums the copied snapshot
once before computing `database_sha256`. Indexed
repository-relative paths, source text, hashes, readiness, and commit provenance
are retained. Export JSON and the manifest expose `path_redaction_requested`,
`path_redaction_complete`, and `path_redaction_omitted_categories`; top-level
`archive_path` / `db_path` are `[redacted]`, and human success output does not
repeat a local path. The source database is never modified.

The archive path is intended for trusted CodeIndex databases. Import validates
that the embedded SQLite file is a CodeIndex DB before replacing the destination
database. `--prune-paths` rewrites the imported `indexed_project_root` metadata
database. A completed path-redaction claim is also checked against the manifest
root/scope fields and the corresponding embedded path metadata; an inconsistent
claim is rejected rather than echoed as verified. `--prune-paths` rewrites the imported `indexed_project_root` metadata
to the import target project root. Imports targeting `.../.cdidx/codeindex.db`
use the sibling project directory; other database paths fall back to the process
current directory. `--dry-run` and its `--check` alias also compare an existing
current directory. A path-redacted archive may omit the source root entirely;
import and dry-run validation do not trust it and remain usable without
`--prune-paths`. `--dry-run` and its `--check` alias also compare an existing
destination DB with the validated archive without replacing it. JSON results
expose the normalized `index_complete`, `index_incomplete_reasons`, and `scope`
values. Archives with no scope metadata are treated conservatively as partial
expose the normalized `index_complete`, `index_incomplete_reasons`, `scope`, and
path-redaction state. Archives with no scope metadata are treated conservatively as partial
during import; current unfiltered archives explicitly preserve full-snapshot
trust. JSON `destination_delta.comparison` reports schema and count deltas plus bounded
file, symbol, reference-edge, chunk, and metadata records. Text fields in those
Expand Down Expand Up @@ -4250,6 +4273,7 @@ legacy database も query でき、generated-code policy は `unavailable` と
```bash
cdidx export codeindex.cdidx.zip
cdidx export codeindex.cdidx.zip --overwrite --json
cdidx export support.cdidx.zip --redact-paths --json
cdidx export app.cdidx.zip --project App --lang csharp --exclude-tests
cdidx export shared.cdidx.zip --path 'src/shared/*' --exclude-path 'src/shared/generated/*'
cdidx import codeindex.cdidx.zip
Expand Down Expand Up @@ -4280,14 +4304,35 @@ portable export は既存 destination を既定で拒否します。意図して
には import 前に artifact を評価するための database hash、row count、schema contract
stamp、readiness state、unknown-extension summary、export scope が含まれます。

互換性を維持する既定 archive は `manifest.project_root` と embedded database の
`indexed_project_root` を保持し、成功時の human / JSON output も解決済み archive path と
database path を報告します。source machine の外へ共有する前に `--redact-paths` を指定してください。
この opt-in mode は manifest と snapshot の両方から project root を除去し、scope と
unknown-extension path sample に含まれる POSIX / Windows / file URI 形式の絶対 path を
`[redacted]` に置換し、`status` が公開する group 別 sample も同様に処理します。不正または
上限超過の path-sample metadata は redaction 完了と報告したまま保持せず、private copy から
削除するか、空の fail-closed 値へ置換します。workspace verification の pending-path identity を
削除または秘匿した場合は coverage marker を incomplete にし、後続の scoped refresh が placeholder を
信頼しないようにします。その後 copy 済み snapshot を一度だけ vacuum し、
`database_sha256` を計算します。repository-relative な indexed path、source text、hash、
readiness、commit provenance は維持します。export JSON と manifest は
`path_redaction_requested`、`path_redaction_complete`、
`path_redaction_omitted_categories` を公開します。top-level の `archive_path` /
`db_path` は `[redacted]` となり、human success output も local path を再表示しません。
source database は変更しません。

archive は信頼できる CodeIndex database の共有向けです。Import は埋め込まれた
SQLite file が CodeIndex DB であることを検証してから destination database を置き換えます。
path redaction 完了の claim は manifest の root / scope field と対応する embedded path metadata に
照合し、不整合な claim は verified として再表示せず拒否します。
`--prune-paths` は import した `indexed_project_root` metadata を import 先 project root に書き換えます。
`.../.cdidx/codeindex.db` を import 先にした場合は sibling の project directory を使い、
それ以外の database path では process current directory に fallback します。
それ以外の database path では process current directory に fallback します。path-redacted
archive は source root を完全に省略できますが、import と dry-run validation はこの値を
信頼しないため、`--prune-paths` なしでも利用できます。
`--dry-run` と alias の `--check` は置換せず、既存 destination DB と検証済み archive を
比較します。JSON result は正規化後の `index_complete`、`index_incomplete_reasons`、
`scope` を公開します。scope metadata がない archive は import 時に保守的に partial と
`scope`、path-redaction state を公開します。scope metadata がない archive は import 時に保守的に partial と
扱い、現行の filter なし archive だけが full snapshot の trust を明示的に維持します。
JSON の `destination_delta.comparison` には schema / count delta と、
file、symbol、reference edge、chunk、metadata の bounded record が含まれます。
Expand Down
Loading
Loading