Skip to content

docs(traces): imported traces stay local; fix the docstrings that said otherwise #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
babaliauskas merged 1 commit into from
Oct 1, 2026
Merged

docs(traces): imported traces stay local; fix the docstrings that said otherwise #28

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion src/evalshift_cli/hosted/trace_events.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@

Both map here onto one wire shape so the server, the database and the client
speak a single language. Neither internal model changes.

Only :func:`from_tool_trace` has a production caller (``hosted/bundle.py``): a run
bundle's ``examples[].traces`` come from the replay's own tool-call traces.
:func:`from_agent_trace` converts imported agent traces to the same wire shape but is
not wired into bundling today — those traces stay local.
"""

from __future__ import annotations
Expand Down Expand Up @@ -170,7 +175,14 @@ def from_tool_trace(trace: ToolTrace | None, *, side: str) -> dict[str, Any] | N


def from_agent_trace(trace: AgentTrace) -> dict[str, Any] | None:
"""Wire stream for one imported bring-your-own-agent trace.
"""Convert an imported agent trace to bundle wire events.

Not called by ``bundle.py`` today: imported traces (``evalshift traces import``)
stay local in ``.evalshift/runs/<id>/traces.jsonl`` and are read only by
``evaluate``, ``report.json`` and the ``diff``/``inspect``/``replay case``
commands. A run bundle's ``examples[].traces`` come solely from the replay's own
tool-call traces via :func:`from_tool_trace`. This function exists for a future
caller that would include imported traces in the bundle.

A round begins at each ``model_call``. The counter increments on every
``model_call`` after the first, so events preceding any model call stay in
Expand Down
11 changes: 7 additions & 4 deletions src/evalshift_cli/traces/models.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,13 @@ def _to_utc(value: datetime) -> datetime:
UtcTimestamp = Annotated[AwareDatetime, AfterValidator(_to_utc)]
"""An event timestamp, offset-aware and stored in UTC.

The bundle contract requires a zero offset (`BUNDLE_SPEC.md` §Validation, enforced by
`app/runs/bundle.py`), and these events are copied into the bundle verbatim by
`evalshift_cli.hosted.trace_events.from_agent_trace`. So the two cases are settled here, where
the error can still name the capture file and line:
Imported agent traces stay local: `evalshift traces import` writes them to
`.evalshift/runs/<id>/traces.jsonl`, and only `evaluate`, `report.json` and the
`diff`/`inspect`/`replay case` commands read them back. They are never copied into a
hosted run bundle — `evalshift_cli.hosted.trace_events.from_agent_trace` exists for that
purpose but has no production caller today; `bundle.py` only calls `from_tool_trace` for
the replay's own tool-call traces. So the two cases below are settled here, for internal
consistency and so the error can still name the capture file and line:

* an offset timestamp is unambiguous and is converted;
* a naive one is refused, because assuming UTC would silently relabel a trace recorded
Expand Down
Loading