Conversation evaluation workspace with a React frontend, FastAPI backend, and LangGraph assessment pipeline. Submit a transcript, follow evaluation progress over SSE, and inspect a structured scorecard.
- Conversation threads and message history.
- Evaluators for clarity, actionability, grounding, and tone.
- Weighted scorecard synthesis and actionable feedback.
- Real-time pipeline transition events.
- Provider integrations and deterministic fallback scoring.
- SQLite development storage and optional PostgreSQL configuration.
Requires Python 3.11 or newer and Node.js.
git clone https://github.com/Jemade/Relay.git
cd Relay
python -m venv .venv
source .venv/bin/activate
pip install -r backend/requirements.txt
cd frontend
npm ci
npm run build
cd ..
uvicorn backend.app.main:app --reload --port 8000Open http://localhost:8000. API documentation is at http://localhost:8000/docs.
For frontend development, run npm run dev in frontend/; Vite serves port 3000 and proxies /api to port 8000.
Use backend/.env.example as a reference. Configuration loads .env from the working directory. Set the selected provider credentials, DEFAULT_PROVIDER, and a supported DEFAULT_MODEL for live model requests.
docker compose up --buildCompose persists SQLite in a named volume. Its optional PostgreSQL profile starts a database; switching the application also requires setting DATABASE_URL to a postgresql+asyncpg:// connection string.
Run the backend tests from the repository root:
pytest -q backend/testsSee GitHub Actions for configured checks.
Rubric scores depend on the selected evaluator and input. Fallback results use heuristics. Evaluation work and SSE broadcasting run in the API process; the service does not provide a durable distributed worker queue.
Read the engineering notes for implementation boundaries and verification commands, the review checklist for evidence still required, and CONTRIBUTING.md to propose changes. Report vulnerabilities through SECURITY.md.
