EchoAgent

Official repository for EchoAgent: guideline-centric reasoning agent for echocardiography measurement and interpretation.

Echocardiographic interpretation requires video-level reasoning combined with guideline-based measurement analysis. EchoAgent coordinates an LLM with specialized vision tools including a phase detection model, measurement segmentation models, and a measurement-feasibility model to answer clinical questions about an echo clip with results grounded in visual evidence and clinical guidelines.

Install

Model weights are stored with Git LFS.

git lfs install
git clone <this-repo-url>
cd EchoAgent
git lfs pull   # only needed if the clone above didn't already fetch LFS content
pip install -e .

Requirements: Python >= 3.10, PyTorch with CUDA recommended for the vision tools. MP4 loading uses TorchCodec, which needs FFmpeg's shared libraries (any of versions 4-8) available on your system, e.g. conda install -c conda-forge ffmpeg if import torchcodec fails to find them. DICOM loading has no such requirement.

Set up an LLM (Ollama)

EchoAgent talks to any OpenAI-compatible chat-completions endpoint, but it's only been tested against Ollama. Start there:

# Install Ollama: https://ollama.com/download
ollama serve &
ollama pull gpt-oss:20b

http://127.0.0.1:11434/v1 (Ollama's default) is what the examples below point at.

Quick start (Python API)

from echoagent import EchoAgent, load_video
from echoagent.llm import ReasoningModel

# DICOM: pixel-spacing calibration is read automatically.
video = load_video("study.dcm")
# MP4: no calibration embedded, so pass it explicitly if you want take_measurement.
# video = load_video("study.mp4", physical_delta=(0.029, 0.029))

agent = EchoAgent(
    reasoning_model=ReasoningModel(model="gpt-oss:20b", api_base="http://127.0.0.1:11434/v1"),
)
print(agent.ask(video, "What is the LV ejection fraction?"))

Quick start (CLI)

echoagent ask \
  --video study.mp4 \
  --question "Is the left ventricle dilated?" \
  --physical-delta 0.029,0.029 \
  --api-base http://127.0.0.1:11434/v1 \
  --model gpt-oss-20b

See also examples/quickstart.py.

Verbose tracing

Pass verbose=True to .ask()/.run() (or --verbose on the CLI) to pretty-print each reasoning step as it runs: the assistant's reasoning text, every tool call with its arguments, every tool result, and the final answer:

print(agent.ask(video, "What is the LV ejection fraction?", verbose=True))

echoagent ask --video study.mp4 --question "..." --physical-delta 0.029,0.029 --verbose

Tools

Tool	Enabled when
search_guidelines	always
calculate_ef	always
detect_phase	always
predict_measurement_feasibility	always
take_measurement	ECHOAGENT_MEASUREMENT_WEIGHTS_DIR set

Only take_measurement needs setup; see below.

Model weights

The phase-detector backbone and head, and feasibility model exist in this repo (via Git LFS, under echoagent/assets/weights/) and just work once you've run git lfs pull.

take_measurement's per-structure segmentation checkpoints are not bundled here and you need to get them from echonet/measurements and point at the directory containing the *_weights.ckpt files:

export ECHOAGENT_MEASUREMENT_WEIGHTS_DIR=/path/to/echonet_measurements_ckpts

Environment variables (all optional, all overridable by CLI flags / Config(...)):

• ECHOAGENT_DEVICE: cuda / cpu (default cuda)
• ECHOAGENT_MAX_ITERATIONS: reasoning-loop cap (default 15)
• ECHOAGENT_SYNCNET_CHECKPOINT, ECHOAGENT_PHASE_CHECKPOINT, ECHOAGENT_FEASIBILITY_CHECKPOINT: override the bundled weights with your own
• ECHOAGENT_MEASUREMENT_WEIGHTS_DIR: see above
• ECHOAGENT_GUIDELINES_DIR: defaults to the bundled corpus; ECHOAGENT_GUIDELINES_TOP_K: default 5

MIMIC-IV-EchoQA (example)

EchoAgent can run inference over clips from the public MIMIC-IV-EchoQA dataset. See examples/mimic_echoqa/README.md for the example and scope details. You must download the credentialed PhysioNet data yourself; this repo does not redistribute it.

LLM backends

from echoagent.llm import ReasoningModel
model = ReasoningModel(model="gpt-oss:20b", api_base="http://127.0.0.1:11434/v1")

For an endpoint that requires an API key, pass it explicitly:

model = ReasoningModel(model="some-model", api_base="https://your-endpoint/v1", api_key="...")

Named presets (gpt-oss-20b, qwen3-coder-30b, llama3.1-8b) target local Ollama-style model IDs; any other string is passed through as a literal model id for the endpoint.

Citation

@article{daghyani2026echoagent,
  title={EchoAgent: guideline-centric reasoning agent for echocardiography measurement and interpretation},
  author={Daghyani, Matin and Wang, Lyuyang and Hashemi, Nima and Medhat, Bassant and Abdelsamad, Baraa and Rojas Velez, Eros and Li, XiaoXiao and Tsang, Michael YC and Luong, Christina and Abolmaesumi, Purang and others},
  journal={International Journal of Computer Assisted Radiology and Surgery},
  pages={1--8},
  year={2026},
  publisher={Springer}
}

License

Released under the Software Evaluation License Agreement (UBC, non-commercial academic research and educational use only).