Keep quoted CLI arguments exact, share one CLI parser, and fix vault gym path scoring

[ProfSynapse]

Summary

Stacked on #166.

Root cause: multi-line CLI writes were broken

shared/environments/tool_executor.py had three problems:

• It normalized all whitespace before tokenizing, which flattened newlines inside quoted values. Curly quotes in content were also rewritten as straight ones.
• It never decoded \n, even though the CLI schema tells models to use it for multi-line content.
• It treated any value starting with -- as a flag. YAML front matter (---) was therefore looked up as an unknown option and dropped.

As a result, content write couldn't produce a multi-line file, and vault gym cases like vault_create_daily_note couldn't pass.

Fix

• One shared parser: shared/validation/parsing/cli_commands.py holds the tokenizer, option detection, command matching and argument binding.
  • Quoting follows POSIX shlex, the same tokenizer the repo already used everywhere.
  • Commands split on commas outside quotes, braces and brackets.
  • A token is a flag only if it is a declared flag or has the form --name.
  • -- keeps its catalog meaning: prompt sub declares it as a literal flag.
• Escapes come from config: tool_call_formats.yaml gains command_escapes: {n, t, r}, decoded only inside double quotes. A format without the key gets plain POSIX quoting.
• The three copied parsers are deleted, with no re-exports: Evaluator/config_validator.py (_expand_cli_wrapper), Tools/migrations/cli_schema_utils.py and tools/analyze_tool_coverage.py. Each copy had the same --- bug.
• The multistep tests write directly again. The copy-then-replace workaround only existed because of this bug.

Vault gym path scoring

Evaluator/runner.py::_run_path_scoring compared the CLI command names in vault_gym.yaml against wrapper call names. Those paths could never match, even when the environment passed.

Observed calls are now matched at the level each path is written in:

• Catalog commands (content write) are matched against the CLI commands a wrapper call expands to.
• Tool names (contentManager_write) are matched against the tools those commands run.
• Anything else, including the wrapper paths in the existing scoring tests, is matched against calls as written.

vault_create_daily_note now scores 1.0 on its preferred path.

Effect on existing output

• All 60 catalog examples and 16 probe commands expand identically before and after.
• Tool-coverage output is unchanged across all 379k checked-in dataset rows.
• Migration output changes in 934 rows. 920 of them are content write calls whose --- content the old parser dropped; the rest gain decoded \n or inline --flag=value handling.

No hash-locked file is touched.

Test plan

• New tests:
  • executor CLI tests: multi-line front matter, values starting with ---, embedded quotes, \n decoding, single quotes not decoded;
  • config validator CLI tests;
  • dataset-tool tests;
  • path scoring at each level;
  • vault gym end to end (environment passes, template-driven-daily-note, score 1.0).
• Targeted suites: 1169 passed. All 14 failures are unrelated to this change and come from the sandbox: torch and pytest_asyncio aren't installed, and the launcher-lock test rejects the sandbox's ambient environment.
• Ruff clean on changed files.

🤖 Generated with Claude Code

https://claude.ai/code/session_013etFNjLuo22CCoQmUGYqCT

---

Generated by Claude Code