Fix empty-project index readiness, shell-search nudge, context() budget, and global rules workspace stamp

[escott-]

Summary

Four agent-facing fixes found while investigating why agents mostly skip ContextStream search, plus a metric to measure the effect. No tool names, descriptions, input schemas or HTTP routes change.

1. Empty projects no longer report "index ready". The hosted index status labels a project with no index row project_index_state: "ready" / status: "completed" with 0 files. project_index_status_reports_canonical_ready accepted that label (or a stale generation number), so init returned canonical_index_ready: true for a project that was never indexed and sent agents to an empty search. A reported file count of 0 is now authoritative; state/generation still stand in when the response omits the count, and an explicit indexed: true still wins.
2. Shell code search is nudged in a checkout with no recorded index. Bash rg / grep -r / find -name only got a nudge when the checkout was in indexed-projects.json. Otherwise the hook took the initial-index-wait path, which has no Bash arm, and fell through to a silent allow. It now emits a non-blocking nudge at most once per 10 minutes per checkout. It does not tell the agent to search (that would hit an empty index); it says no index is recorded and to run project(action="index"). Piped filters, log greps and single-file greps are still left alone.
3. context() keeps its actionable fields under the wire budget. The budget stripped every structured field, including instructions, matched_skills, coordination_inbox and grounding_hits, before touching the text that was actually over budget, then removed whole text blocks, leaving a 4k budget under a third used. The large duplicates (items, summary, context) now go first, coordination_inbox is listed explicitly (it used to fall into the lexical pass where it sorts first), low-priority text shrinks before the small fields are dropped when that fits, and a truncated head of the largest block is kept rather than dropping it whole. Under a budget too tight to hold both copies, the old order is kept so the text's instructions outlive their structured duplicates.
4. Global rules no longer carry a workspace. setup and rules refresh stamped the workspace of whichever directory they ran in into the global rules file, which applies to every project, so the file named a different workspace after each run and disagreed with the project rules beside it. write_editor_rules now writes the neutral block, so a repair also clears an old stamp while keeping the user's own text around the managed block. A global-only refresh skips the API workspace lookup.
5. testing/adoption/search_adoption.py (+ tests, wired into CI like testing/grounding) counts per agent how often code search goes through ContextStream versus shell rg/grep/find or the native Grep/Glob tools, using a port of the hook's shell classifier. It reads local Claude Code / Codex transcripts and prints counts only; --json writes a report for before/after comparison. The Claude Code reader is checked against a real transcript; the Codex reader is best-effort.

Public-boundary impact: none. No private implementation, operator material or deployment detail is added.

Validation

• cargo fmt --all --check
• cargo clippy --locked --workspace --all-targets -- -D warnings
• cargo test --locked --workspace --all-targets
• npm test
• python3 .github/scripts/public_boundary.py .

Not ticked because the exact command was not run. What was run instead:

• cargo clippy --locked -p mcp-client -p mcp-tools -p mcp-server --all-targets -- -D warnings: clean.
• cargo test --locked -p mcp-client -p mcp-tools -p mcp-server --lib: mcp-server 961, mcp-tools 1469 and mcp-client 283 pass. 3 mcp-client tests fail (ingest_guard::tests::rejects_home_directory, rejects_sensitive_directory, rejects_path_inside_sensitive_directory); they fail identically on unmodified main when run as root with HOME=/root, so they are environmental. 2 mcp-server watch:: tests also fail when CONTEXTSTREAM_WATCH=0 is set in the environment and pass with it unset.
• New tests fail on the old code and pass now: the two context() budget tests (oversized_context_pack_does_not_cost_the_small_actionable_fields, budget_is_filled_by_truncating_a_block_rather_than_dropping_it_whole), plus tests for the empty-project readiness, the shell nudge and its throttle, and the global rules rewrite.
• testing/adoption unit tests (7) and the .github/scripts workflow, public-boundary, release-contract and DCO policy tests pass.

The context() fixture is a hand-built, production-sized payload, not captured production output, so it demonstrates the mechanism rather than reproducing exact production numbers.

Data handling and compatibility

• The change does not add credentials, customer data, raw local paths, or private deployment topology.
• Any new collection or transmission of user data is documented in docs/data-handling.md. Nothing new is collected or transmitted: the hook records a local timestamp in ~/.contextstream/prompt-state.json, and the adoption script reads local transcripts and prints counts.
• Existing npm executable aliases and MCP wire compatibility are preserved. No tool schema changes; the context() change only affects which fields survive when a response exceeds its requested token budget.

Contribution certification

• Every commit includes a Signed-off-by: trailer (git commit -s).

The commits do not carry Signed-off-by: yet; the author needs to add it (for example git rebase --signoff main) before the DCO check can pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_018gMJ8CEW7U8zTm2GifkwEH

---

Generated by Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}