Repository navigation
Clarify model rationales with prompt and model selectors - #41
Conversation
|
Review: no merge-blocking findings in the prompt provenance, condition/model selection, or per-call probability handling. Verified in an isolated checkout at 1a5b011: 463 Python tests passed (16 data-dependent skips plus 10 initially missing npm harness tests); after npm ci all 10 harness integration tests passed as well. The two Node tooltip tests passed. redlines build + assemble reproduced the tracked tree exactly. Tests used a separate empty forecasting agent directory and disk-backed temp directory; no provider calls or credentials copied. Approved the waiting fork workflow so the normal CI matrix can run before merge. Deployment remains separate: as noted in the PR, the preserved published snapshot requires a rationale metadata migration before this interface is deployed. |
6bb2ff4
into
forecastingresearch:main
Model rationales currently appear as consecutive blocks without identifying the prompt behind each explanation. This change lets readers select a prompt condition and model using the site's existing button groups, then see one forecast row and its rationale immediately.
The row uses archived probabilities for the selected model, prompt condition, question and date. The prompt's added instruction is visible, while full prompts, sources and provenance remain expandable. Original explanations and source records are preserved, and the main charts retain their existing averages. No new dependencies or forecast runs are introduced.
Files changed:
web/demo/65-rationales.jsx: adds prompt/model selection, a condition-specific forecast row, visible rationale and expandable archived prompts.redlines/views/rationales.py: supplies archived prompt metadata and per-call probabilities, sharing identical prompt text across models.web/demo/60-atoms.jsx: adds accessible group labels and selected states to the existing Toggle component; its styling is unchanged.tests/test_rationale_prompts.py: adds eight tests for prompt accuracy, missing metadata and preservation of forecasts and explanations.index.htmlandresults/rationales_data.json: regenerated outputs reflecting the source changes.Deployment note: the new interface needs additional rationale metadata. Normal frontend deployment preserves the old published data, so deployment must include a validated data publication from existing production logs or an explicit migration of the rationale data. No new model calls are needed. The public checkout lacks current-instrument calls; the preview was verified separately against the captured public archive without adding that archive to the repository.