Drive dataset and structure validators from tool-call format config

[ProfSynapse]

Summary

Stacked on #168.

This removes useTools-specific hardcoding from the validators, as CLAUDE.md requires: tool-call formats come from config.

Dataset validator (shared/validation/dataset_validator.py)

• Wrapper checks come from config. The wrapper name, required argument fields and per-field constraints come from SynthChat/config/tool_call_formats.yaml, read through the existing loader and configured_formats.build_wrapper_specs. Wrapper-less formats now validate. --tool-call-formats PATH lets a host validate against its own config.
• Schema validation now actually runs. SCHEMAS_FILE pointed at tools/tool_schemas.json, but the file is at Tools/, so schema validation was always skipped on Linux. It now resolves from the engine root, and --tool-schemas PATH lets a host override it.
• ID checks come from config. They use a new format key, prompt_bound_fields ({pattern, in_tag} sources per field). This replaces the hardcoded sessionId/workspaceId regexes.
• Removed, with no config basis:
  • the memoryManager_/agentManager_ ID warnings;
  • the get_tools special case;
  • the argument allow-list (the catalog's parameters already declares every accepted argument);
  • the old flat-context check;
  • the dead validate_tool_call_structure.
• The validate_syngen.py star-import shim is deleted. Every caller now runs python3 -m shared.validation.dataset_validator, and a test keeps the shim from coming back.

Wrapper command field

• A new format key, command_field, names the CLI command argument. The default format sets it to tool.
• It replaces the literal tool everywhere it was hardcoded: the sanitizers in configured_formats, response_parser, the CLI expansion in tool_executor, and Keep quoted CLI arguments exact, share one CLI parser, and fix vault gym path scoring #168's Evaluator/config_validator.py.
• The duplicate, unread wrapper.cli_field is removed from Tools/tool_schemas.json.

Structure validator (structure_validator.py)

• The tools rule's error template is now applied, with {tool_name} and {details}. Every active tool rubric sets it, but it was read and then ignored.
• Schema-level _item_schema is now enforced. This also fixes a crash on scalar item schemas, and arrays of arrays that were silently skipped.
• Subtool keys come from config. They are declared through a new required _subtool_keys: {group, tool, params} instead of hardcoded agent/tool/params. All configs and docs that use _subtools are updated.

Other changes

• args_match.py: removed call_fields. It has been unused since it was replaced by mappings (98098e4), and no config sets it.
• SynthChat workspace prompt: "Required wrapper fields" now renders from argument_required. It used to render an empty list because the default format never sets context_fields.

What schema validation now finds in the checked-in datasets

These are reported, not suppressed. No datasets were changed.

• Legacy argument shape in about 49 files. They use the old useTools {context, calls} arguments, which gives about 178k "missing required parameter" errors. Examples: nonthinking_tools_sft_01.02.26.jsonl, sft_train_67pct_12.25.jsonl, and the older manager tool sets.
• Field-constraint violations in 10 newer CLI-format files: 56 empty required fields and 18 values outside their allowed list. Includes nonthinking_tools_sft_04.22.26.jsonl and promptManager/tools_v2.6–v2.8.
• 44 IDs that don't match their system prompt, across 16 files.

No hash-locked file is touched.

Test plan

• New tests:
  • test_dataset_validator.py (20)
  • test_configured_formats.py (6)
  • structure validator: error template, item schemas, subtool keys
  • verifier and workspace tests
  • fixtures use a format whose names have nothing to do with useTools
• Keep quoted CLI arguments exact, share one CLI parser, and fix vault gym path scoring #168's suites (tool-executor CLI, vault gym, multistep, evaluator scoring, config-validator CLI, dataset tools): 202 passed.
• Broader sweep: 3800 passed. It has the same 39 failures and 8 errors as the untouched base, all from missing packages in the sandbox.
• ruff check . and sync_skill_trees.py --check are clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_013etFNjLuo22CCoQmUGYqCT

---

Generated by Claude Code