Memory-Chat-API

A production-style AI chat backend with persistent memory across conversations. Built with FastAPI, PostgreSQL, and Qdrant vector search — the AI remembers what you've told it, even across separate sessions.

---

What it does

Most AI chatbots forget everything the moment a conversation ends. Memory-Chat-API solves this with two memory layers:

• Short-term memory — full conversation history is stored in PostgreSQL and sent to the LLM on every message, so the AI remembers everything said within a thread.
• Long-term memory — every message is embedded into a vector and stored in Qdrant. On each new message, the system searches for semantically similar content from past conversations and injects it into the AI's context. The AI can recall things you said weeks ago, in a completely different conversation.

---

Tech stack

Layer	Technology
API framework	FastAPI (async)
LLM provider	Groq (OpenAI-compatible — swappable)
Relational DB	PostgreSQL 16
Vector DB	Qdrant
Embeddings	all-MiniLM-L6-v2 via sentence-transformers (runs locally)
DB migrations	Alembic
HTTP client	httpx (async)
Containerization	Docker + Docker Compose

---

Architecture

User Request
     │
     ▼
FastAPI (chat.py)
     │
     ├── 1. Get or create User + Conversation (PostgreSQL)
     ├── 2. Load last 20 messages from conversation (short-term memory)
     ├── 3. Search Qdrant for semantically similar past messages (long-term memory)
     ├── 4. Build prompt: system + memory context + history + new message
     ├── 5. Send to LLM API → get reply
     ├── 6. Save user message + reply to PostgreSQL
     └── 7. Embed + store both messages in Qdrant

---

Project structure

Memory-Chat-API/
├── backend/
│   ├── app/
│   │   ├── main.py                  # App entry point, startup hooks
│   │   ├── core/
│   │   │   └── config.py            # All settings loaded from .env
│   │   ├── db/
│   │   │   └── session.py           # Async SQLAlchemy engine + session factory
│   │   ├── models/
│   │   │   └── models.py            # DB models: User, Conversation, Message
│   │   ├── api/
│   │   │   ├── chat.py              # POST /api/chat and POST /api/chat/stream
│   │   │   └── conversations.py     # GET/DELETE /api/conversations
│   │   └── services/
│   │       ├── llm_service.py       # LLM API client (OpenAI-compatible)
│   │       ├── conversation_service.py  # DB operations for conversations/messages
│   │       └── memory_service.py    # Qdrant vector storage + semantic search
│   ├── alembic/                     # Database migration scripts
│   ├── .env.example                 # Template for environment variables
│   ├── requirements.txt             # Python dependencies
│   └── stream_test.py               # Manual integration test for streaming
├── docker-compose.yml               # PostgreSQL + Qdrant containers
├── Dockerfile                       # Backend container
└── README.md

---

Getting started

Prerequisites

• Python 3.11+
• Docker Desktop
• A Groq API key (free)

1. Clone the repo

git clone https://github.com/YOUR_USERNAME/Memory-Chat-API.git
cd Memory-Chat-API

2. Set up environment variables

cd backend
cp .env.example .env

Edit .env and fill in your values:

LLM_API_KEY=your-groq-api-key-here
LLM_API_BASE=https://api.groq.com/openai/v1

3. Start the databases

cd ..
docker-compose up -d

This starts PostgreSQL on port 5432 and Qdrant on port 6333.

4. Install dependencies

cd backend
pip install -r requirements.txt

First install takes a few minutes — it downloads PyTorch and the sentence-transformers embedding model (~90MB, one time only).

5. Start the server

uvicorn app.main:app --reload

On first startup the server will:

• Create all database tables automatically
• Initialize the Qdrant vector collection
• Download the embedding model if not already cached

Visit http://localhost:8000/docs for the interactive API documentation.

---

API reference

POST /api/chat

Non-streaming chat. Returns the full reply in one response.

Request body:

{
  "message": "Hello, my name is Alex",
  "username": "alex",
  "conversation_id": "optional-uuid-to-continue-existing-conversation"
}

Response:

{
  "reply": "Hi Alex! How can I help you today?",
  "conversation_id": "d653dc16-8a2e-49d1-ab05-46ec7469b3ca"
}

---

POST /api/chat/stream

Streaming chat. Returns the reply as a Server-Sent Events (SSE) stream — tokens arrive as they're generated, like a typing effect.

Same request body as /api/chat. The conversation_id is returned in the X-Conversation-Id response header.

SSE format:

data: Hello
data:  Alex
data: !
data: [DONE]

---

GET /api/conversations?username=alex

Lists all conversations for a user, newest first.

Response:

[
  {
    "id": "d653dc16-8a2e-49d1-ab05-46ec7469b3ca",
    "title": "Hello, my name is Alex",
    "created_at": "2026-09-10T12:00:00+00:00",
    "updated_at": "2026-09-10T12:05:00+00:00",
    "message_count": 0
  }
]

---

GET /api/conversations/{conversation_id}

Returns a single conversation with all its messages.

---

DELETE /api/conversations/{conversation_id}

Deletes a conversation and all its messages.

---

GET /health

Health check endpoint.

{ "status": "ok", "environment": "development" }

---

How memory works

Short-term (within a conversation)

Every message is stored in PostgreSQL. When you send a new message with a conversation_id, the last 20 messages are loaded and sent to the LLM as conversation history. The AI has full context of the current thread.

Long-term (across conversations)

Every message is embedded using all-MiniLM-L6-v2 — a local embedding model that converts text into a 384-dimensional vector representing its semantic meaning. These vectors are stored in Qdrant with metadata (user ID, conversation ID, role).

Before each LLM call, the system runs a cosine similarity search in Qdrant to find the most relevant things the user has said in other conversations. Results above a relevance threshold are formatted and injected into the system prompt.

This means: even in a brand new conversation, the AI can recall facts from weeks ago.

---

Swapping the LLM provider

The LLM client uses the OpenAI /chat/completions wire format, which is supported by most providers. To switch:

1. Update LLM_API_KEY and LLM_API_BASE in your .env
2. Update self.model in backend/app/services/llm_service.py

Provider	LLM_API_BASE	Example model
Groq	https://api.groq.com/openai/v1	openai/gpt-oss-20b
OpenAI	https://api.openai.com/v1	gpt-4o
Together AI	https://api.together.xyz/v1	meta-llama/Llama-3-70b
Ollama (local)	http://localhost:11434/v1	llama3

---

Environment variables

Variable	Required	Default	Description
LLM_API_KEY	Yes	—	API key for your LLM provider
LLM_API_BASE	No	Groq URL	Base URL for the LLM API
POSTGRES_URL	No	localhost	PostgreSQL connection string
QDRANT_URL	No	localhost:6333	Qdrant connection URL
QDRANT_API_KEY	No	—	Qdrant API key (for cloud deployments)
DEBUG	No	false	Enables SQL query logging
ENVIRONMENT	No	development	Environment name

---

Running with Docker

To run the entire stack (backend + databases) in containers:

docker-compose up --build

The backend Dockerfile is at the root of the repo.

---

Future roadmap

• Neo4j knowledge graph for relationship mapping between concepts
• User authentication (JWT)
• Frontend UI
• Conversation summarization for very long threads
• Multi-modal support (images, documents)

---

License

MIT