Repository navigation
chore(agents): verify-ftw skill that drives Core with the simulators - #1678
Conversation
…ators Agents claim behaviour from reading code or from tests that pass. This skill gives them one scripted way to start Core with the Ferroamp and Sungrow simulators on isolated ports and an empty state directory, drive a feature through the public API, and keep the evidence. launch.sh waits for live driver telemetry, doctor.sh checks the instance is ours, drive-mode.sh proves the idle journey down to the power sent to each battery, and cleanup.sh stops only the pids it started. The feature map seeds live status, mode and stale-driver fallback. Proved end to end on this branch: launch, doctor, drive-mode idle, cleanup, evidence kept. .cursor/ joins the changeset allowlist because nothing in it ships to users.
Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
…1679) * docs: propose a stepwise plan to make FTW safe for agents to change Agents write most of FTW, and review bots raised 1,150 findings on 573 of their pull requests. This plan groups those findings into recurring mistake classes, proposes a structural fix for each at the highest level that works, backtests each fix against the history, and orders the fixes into independently mergeable steps. Steps 1, 2 and 4 are draft PRs #1676, #1677 and #1678. tools/agent-pitfalls holds the script and the labels so a reviewer can rerun or spot-check every number. * docs: keep the pitfall lever under docs/ so the PR stays docs-only * docs: distinguish model estimates from executable evidence Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com> --------- Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f47aa0c1ab
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ("path: ../dev-data/state.db", f"path: {self.run}/data/state.db"), | ||
| ("lua: drivers/", f"lua: {self.repo}/drivers/")]: | ||
| require(old in config, f"simulator config is missing {old!r}") | ||
| config = config.replace(old, new) |
There was a problem hiding this comment.
When an override equals a placeholder processed later—for example, FTW_VERIFY_MQTT_PORT=8887 with a distinct custom OCPP port—this sequential global replacement first writes port: 8887 into the MQTT block and then replaces it again while processing the OCPP placeholder. Core consequently connects to the OCPP port instead of the simulator's MQTT port, so an otherwise valid documented port configuration never becomes ready. Replace only the intended YAML fields or derive the generated config structurally.
Useful? React with 👍 / 👎.
Add a verification skill for Core with the Ferroamp and Sungrow simulators. The helpers share a Python runner that records process identity, checks API-port ownership before each request, refuses occupied ports and existing scratch state, and stops its children when startup fails. Cleanup checks process identity before signalling and keeps evidence.
The supported automated journey is
idle. It requires both named batteries exactly once in dispatch and feedback, accepted 0 W commands, fresh readings, and command and observation times after this action. Empty feedback, one missing battery, stale evidence or another process owning the API port must fail. Other modes need their own assertions; stale-driver fallback remains a manual journey.Validation:
ps/lsoflistener check. CI runs these tests on macOS and Linux.bash -nfor all four entry points.make verifypassed on the shared Go base in ci: run the Go race detector on the control-loop packages #1676; this PR changes no Go or product code.The helper tests run in CI. The complete simulator journey above is a local run, and no hardware behaviour follows from it.
.verify-evidence/stays gitignored. Nothing in this skill ships to users, so no changeset is needed.Part of #1679. Remains draft for the owner's merge decision.