fix(package): scan semantic content for internal references

[mldangelo-oai]

Summary

The package disclosure check decoded the complete npm tar stream as UTF-8. Valid compressed binary metadata could therefore resemble an internal reference even when every publishable path, text file, and decompressed payload was clean.

Scan the semantic contents that would actually be published. The checker now validates the package's canonical plain-ustar contract, extracts it with the local tar implementation into a private temporary directory, and scans and hashes the resulting regular files. Raw compressed bytes and tar padding are no longer treated as text.

Changes

• Move internal-reference detection into a focused helper that scans archive paths, regular text, and decompressed single-file or partitioned Brotli payloads.
• Keep PNG validation on the extracted file bytes without interpreting arbitrary binary bodies as UTF-8.
• Validate the emitted npm package as canonical plain ustar before extraction: scan each complete header, require the exact regular-file type, ustar signature, octal size encoding, path order, zero padding, and terminal zero blocks. Reject extension, sparse, link, alternate-size, and concatenated archive forms that are outside this package's contract.
• Stream the already-read gzip bytes to every tar operation through stdin, without changing the child working directory or passing the archive path to tar. Drain only zero padding that the strict parser has already validated.
• Extract once into a private directory, verify the exact expected regular-file tree and sizes without following links, enforce the existing expanded-byte bound, and always remove the temporary directory.
• Add focused regression coverage for the binary false positive, real markers in text and decompressed Brotli, canonical prefix paths, complete headers, unsupported tar forms, alternate size/signature encodings, padding, archive termination, Windows archive paths, command lookup, and large stdin writes.

Testing

• Failing-first: a valid Brotli stream whose raw bytes decode to a marker-like �Go/1 sequence failed the old whole-tar scan even though its decompressed payload was empty; the semantic scan accepts it.
• Failing-first header coverage: two full 265-entry packages carrying a marker only in ignored regular-file header bytes passed the prior semantic checker; complete-header scanning rejects both while the canonical package remains valid.
• Failing-first Windows coverage: on initial head 6d0bc657, both hosted Windows verification jobs failed because GNU tar interpreted the drive-letter archive path as remote-host syntax. Tar now receives only stdin.
• Working-directory regression guard: a discarded repair selected an archive-adjacent synthetic tar shim when PATH began with a relative entry. Keeping the caller working directory prevents the archive directory from participating in executable resolution.
• Streaming regression guard: an intermediate repair treated semantically identical default-gzip and 31 MiB level-0 archives differently because tar completed before Node finished writing stdin, producing EPIPE. Central --ignore-zeros now drains only padding that the strict parser has already proved is all zero; short listing, verbose listing, and extraction all complete.
• Focused package regressions: 25 passed, 0 failed, with 72 expectations across five files using seed 618.
• Prettier, generated-model check, TypeScript --noEmit, package build, and Node syntax checks passed.
• Current 0.1.18 package artifact: full package check and installed-package smoke passed for all 265 entries, including public import, NodeNext types, CLI execution, 113 bundled plugin files, bundled CLI version, and nested worker execution.
• Historical fix: preserve accepted evidence during Deep scan reduction #442 compatibility artifact: the semantic checker passed all 263 entries with its exact historical contract, and its matching installed-package smoke passed with 111 bundled plugin files. This is not a current-main package; fix: preserve accepted evidence during Deep scan reduction #442 must regenerate and check its artifact after reconciliation.
• Adversarial tar corpus: the full checker rejected all 35 unsupported extension, sparse, dump-directory, alternate-size, header-marker, path-alias, duplicate-path, padding, and concatenated-archive artifacts while accepting the current and historical canonical controls.
• Packaging is byte-neutral: clean main and the repaired worktree both produced npm tarball SHA-256 da76cfe405b5ff3c45ac3e356461c09b742367503d739fd18d471683ae96e531.
• Full TypeScript test suite: 1,682 passed, 28 skipped, 0 failed, with 22,121 expectations across 1,710 tests and 94 files using seed 618.
• Fresh post-merge exact-head correctness and public-disclosure reviews, plus three independent native reviews, found no actionable P0–P3 defects.

Risk and rollout

The publishable package contents do not change. The checker intentionally accepts only the plain-ustar shape emitted by the current package build; unsupported tar metadata fails closed instead of being interpreted by a second custom tar implementation. The current artifact uses 265 regular type-0 entries with a maximum path length of 87 bytes and requires no compatibility exception.

The semantic scan preserves internal-reference coverage for paths, complete ustar headers, normal text files, and decompressed Brotli payloads. PNG files remain digest-allowlisted, and all extracted regular-file bytes remain subject to the existing cumulative bound. The temporary extraction tree is private and removed on success or failure. The validation-stage tar subprocesses never receive the archive path: Node reads the archive bytes, and tar receives those bytes on stdin plus an absolute extraction destination.

Public disclosure review

• No customer, partner, prospect, or user identities, data, or identifying details are included.
• No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
• I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

[mldangelo-oai]

@codex review

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-08-23T22:58:07.494169Z	35cc277	Manual request
🔒 Security Review	✅ Completed 2026-08-23T19:52:42.362463Z	6d0bc65	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: 6d0bc657c1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[mldangelo-oai]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: fd44228f5b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[mldangelo-oai]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: 35cc277e85

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[kmbroai]

Critical review

Reviewed head 35cc277e85843e61a6cd55a051f4425bc9fdd6d7.

Recommendation: keep semantic-content scanning; no new blocking correctness defect found in the reviewed changes. Decoding compressed archive bytes as text can reject a clean package because of coincidental byte sequences. Scanning filenames, actual text, and decompressed bundled code fixes the right problem without weakening the intended disclosure check.

Correctness

check-package.mjs validates the closed archive format before extraction, compares the native tar listing with parsed paths, verifies the extracted tree and sizes, and cleans up in finally. Scanning complete headers retains metadata coverage, while PNGs remain restricted to the known digest instead of arbitrary text interpretation. The Brotli helper checks decompression consumption, so trailing data is not silently ignored.

Passing the already-read archive on stdin avoids Windows drive-letter interpretation and a separate path reread. Keeping the caller's working directory also avoids accidentally selecting a relative-PATH executable from beside the archive. The --ignore-zeros use is acceptable here specifically because the preceding parser proves the remainder is zero padding; it should not be copied to an unconstrained archive reader.

Simplification and necessity

The semantic scanner is the essential fix. The extra parser/extractor agreement is defensible only as validation of this package's controlled plain-ustar contract—not as a general tar library. Preserve that narrow scope. An npm pack-format change should cause one explicit compatibility update, not accumulate ad hoc support for every extension.

There are three native tar passes plus the in-memory parser and filesystem walk. I would avoid adding further independent interpretations. If this becomes costly to maintain, evaluate a single maintained archive reader with equivalent path/type/metadata guarantees; do not simply remove checks while retaining native extraction. The existing expanded-size bound is unchanged by this PR and is not a newly introduced scan-input cap.

Verification

Ran the five focused package helper suites: 25 passed, 0 failed, 72 assertions, including binary-marker false positives, decompressed content, tar forms/listing, provenance, and timeout helpers. Linux with Bun 1.3.14 / Node 22.13.1 and cached dependencies. I did not independently rebuild the full npm archive, run installed-package smoke, or execute native Windows/macOS packaging, so the author's broader artifact claims are not my verification results.