From 1abe17f9f6a0836bfbe1d7801d85670e71f65265 Mon Sep 17 00:00:00 2001
From: JinBa1 <72070041+JinBa1@users.noreply.github.com>
Date: Sat, 20 Jun 2026 03:36:20 +0100
Subject: [PATCH 1/2] docs: rewrite README as a gateway-first overview; split
 engine internals into engine/README
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The root README led with "an in-memory relational query engine" and listed only
engine SQL features — the REST + MCP gateway was invisible. Reframe it around what
the project now is: a self-hosted, read-only, budgeted SQL gateway that AI agents
reach over REST and MCP.

- root README: gateway-first and punchy — tagline + why, a capability matrix
  (gateway + engine), a container-first quick start, the five MCP tools, the REST
  surface with fail-closed budget semantics (429/504), a real EXPLAIN example, and a
  short "how it works" (the QueryService choke point). Links to the engine README.
- engine/README.md (new): the deep engine internals relocated here — architecture,
  scope, CLI usage + demo, total-work budgets, full EXPLAIN, join algorithms, the JMH
  benchmark table, and the sample-query runner.
- accuracy pass: EXPLAIN examples regenerated from real CLI output (the planner
  auto-selects HashJoin for equi-joins; the old plan showed a plain Join); test
  counts corrected to engine 419 + server 90; file counts refreshed.

Docs only — no code change.
---
 README.md        | 318 +++++++++++------------------------------------
 engine/README.md | 166 +++++++++++++++++++++++++
 2 files changed, 236 insertions(+), 248 deletions(-)
 create mode 100644 engine/README.md

diff --git a/README.md b/README.md
index 0cd763a..62410ef 100644
--- a/README.md
+++ b/README.md
@@ -4,102 +4,28 @@
 ![Coverage](https://codecov.io/gh/JinBa1/java-query-engine/branch/main/graph/badge.svg)
 ![Dependencies](https://img.shields.io/badge/dependencies-up%20to%20date-brightgreen)
 
-An in-memory relational query engine built on the Volcano/iterator model. Parses SQL via JSqlParser, builds an operator tree, and executes queries tuple-by-tuple against CSV data.
+**A self-hosted gateway that gives AI agents safe, read-only, budgeted SQL access to your CSV files — no database required.**
 
-## Architecture
+Everyone has CSVs — exports, dumps, logs — and AI agents increasingly need to query them. Embedding a database in every agent environment hands over raw file access; what you actually want is a *guarded window* onto the data: an endpoint that is read-only by construction, resource-budgeted, and auditable. cuckooDB is that gateway, built on a from-scratch query engine and exposed over both a **REST API** and the **Model Context Protocol (MCP)**, so an agent can discover tables, preview data, check a query's cost, and run SQL — without writing SQL blind or bypassing the guardrails.
 
-```
-SQL → JSqlParser → QueryPlanner → QueryPlanOptimizer → Operator Tree → Results
-```
-
-**Core components:**
-
-| Component | Role |
-|-----------|------|
-| `QueryPlanner` | Parses SQL and builds the operator pipeline |
-| `QueryPlanOptimizer` | Selection pushdown, trivial operator removal |
-| `DBCatalog` | Schema and table metadata (singleton) |
-| `Value` | Typed tuple values (sealed interface: `IntValue`, `StringValue`) |
-| `ExpressionEvaluator` | Evaluates WHERE/HAVING conditions per tuple |
-| `ExpressionPreprocessor` | Resolves column references to indices |
-| `ConditionSplitter` | Separates join predicates from selection predicates |
-
-**Operator hierarchy** (all extend `Operator`):
-
-`ScanOperator` → `SelectOperator` → `ProjectOperator` → `JoinOperator` / `HashJoinOperator` → `SortOperator` → `AggregateOperator` → `DuplicateEliminationOperator` → `LimitOperator`
-
-## Feature Matrix
-
-| Feature | Status |
-|---------|--------|
-| `SELECT *` / projection | ✅ Supported |
-| `WHERE` predicates | ✅ Supported |
-| Inner joins (nested-loop) | ✅ Supported |
-| Hash join (auto-selected for equi-joins) | ✅ Supported |
-| `ORDER BY` | ✅ Supported |
-| `GROUP BY` + `SUM`, `COUNT`, `AVG`, `MIN`, `MAX` | ✅ Supported |
-| `LIMIT n` | ✅ Supported |
-| `DISTINCT` | ✅ Supported |
-| Nested arithmetic/comparison expressions | ✅ Supported |
-| Query optimisation (selection pushdown) | ✅ Supported |
-| Typed columns (int, string) | ✅ Supported |
-| CSV header support | ✅ Supported |
-| Query budgets (`--max-tuples`, `--timeout-ms`) | ✅ Supported |
-| `EXPLAIN` plan inspection | ✅ Supported |
-| Indexes | ❌ Not supported |
-| Transactions | ❌ Not supported |
-| INSERT / UPDATE / DELETE | ❌ Not supported |
-| Concurrency | ❌ Not supported |
-| Persistence | ❌ Not supported |
-| Full SQL dialect | ❌ Not supported |
-
-## Scope
-
-This engine supports **SQL-over-CSV query execution**: read-only queries against tables stored as CSV files. It does not support transactions, indexes, data modification (INSERT/UPDATE/DELETE), concurrency, persistence, or a full SQL dialect. Values are typed int or string, inferred per column from the data. Tables are discovered from CSV files with header rows; no separate schema file.
-
-Supported SQL features include `SELECT`/`FROM`/`WHERE`, `GROUP BY` with `SUM`, `COUNT`, `AVG`, `MIN`, and `MAX` aggregates, `ORDER BY`, `DISTINCT`, inner joins, and `LIMIT n`.
-
-The focus is on demonstrating query planning, optimisation, and the Volcano iterator execution model.
-
-### Aggregate and LIMIT semantics
-
-| Case | Behavior |
-|---|---|
-| `AVG` of ints | truncated integer division (toward zero) |
-| Aggregate over empty input, no `GROUP BY` | zero rows (header only) — deviates from SQL's NULL row |
-| `COUNT(col)` | equals `COUNT(*)` — the engine has no NULLs |
-| `SUM`/`AVG` on a string column | error |
-| `MIN`/`MAX` on strings | lexicographic |
-| `SUM` past int range | error |
-| `LIMIT 0` | header-only output |
-| `OFFSET`, `LIMIT ALL` | not supported (error) |
-
-## Quick Start
-
-**Prerequisites:** Java 17, Maven (or use the included Maven Wrapper).
-
-```bash
-# Clone
-git clone https://github.com/JinBa1/java-query-engine.git
-cd java-query-engine
+## Features
 
-# Build fat JAR (engine module)
-./mvnw -pl engine -DskipTests clean package
-```
-
-**Run a query:**
-
-```bash
-java -cp engine/target/cuckoodb-engine-1.0.0-jar-with-dependencies.jar \
-  com.github.jinba1.cuckoodb.CuckooDB \
-  database_dir input_file output_file [--max-tuples=N] [--timeout-ms=N]
-```
-
-Both `--max-tuples` and `--timeout-ms` are optional and independent. Omit either to impose no limit on that dimension.
+| Capability | |
+|---|:--:|
+| Read-only SQL over CSV — `SELECT` / `WHERE` / `JOIN` / `GROUP BY` / `ORDER BY` / `LIMIT` / `DISTINCT` | ✅ |
+| Aggregates — `COUNT` / `SUM` / `AVG` / `MIN` / `MAX` | ✅ |
+| Hash + nested-loop joins (planner auto-selects) | ✅ |
+| Typed columns (int / string), CSV headers | ✅ |
+| `EXPLAIN` plan inspection | ✅ |
+| Tuple + time budgets, fail-closed | ✅ |
+| **REST API** + OpenAPI / Swagger | ✅ |
+| **MCP server** — five agent tools, Streamable-HTTP | ✅ |
+| Runs as a container (published to GHCR) | ✅ |
+| Writes / transactions / indexes / persistence | ❌ read-only by design |
 
-### Run the server as a container
+## Quick start
 
-The Spring Boot gateway (REST + MCP) ships as a container image, so you can run it next to your data with no Java toolchain. Put your CSV files in a folder and mount it as the catalog's data directory:
+Run the gateway next to your data — no Java toolchain needed. Put your CSVs in a folder and mount it:
 
 ```bash
 docker run --rm -p 8080:8080 \
@@ -107,198 +33,94 @@ docker run --rm -p 8080:8080 \
   ghcr.io/jinba1/cuckoodb:latest
 ```
 
-- **REST:** `POST http://localhost:8080/queries`, `GET /tables`, `GET /tables/{name}` (OpenAPI at `/swagger-ui.html`).
-- **MCP:** Streamable-HTTP endpoint at `http://localhost:8080/mcp` — point an MCP client at it to query your CSVs with `list_tables` / `describe_table` / `sample_rows` / `explain_query` / `query`.
-
-The image is published to GHCR on each merge to `main`. To build it locally instead: `docker build -t cuckoodb .`
-
-### Query budgets
-
-The engine enforces **total-work semantics**: every tuple emitted by any operator in the tree counts against the budget, including intermediate tuples that are later filtered or joined. A cross-product explosion that never produces output rows will still hit the tuple limit. The timeout clock starts lazily at the first tuple emission.
-
-When a budget is exceeded:
-- The partial output file is deleted.
-- `Error: <message>` is written to stderr.
-- The process exits with code 1.
-
-Both flags are optional and independent — you can use one, both, or neither.
-
-### EXPLAIN
-
-Prefix any query with `EXPLAIN` to inspect the query plan without executing it:
-
-```sql
-EXPLAIN SELECT Student.B, SUM(Student.C) FROM Student, Enrolled
-WHERE Student.D > 30 AND Student.A = Enrolled.A
-GROUP BY Student.B;
-```
-
-The output file receives a two-section plan:
+Query over REST:
 
-```
-=== Plan (as written) ===
-Aggregate[group by: Student.B; calls: SUM(Student.c)]
-  Project[Enrolled.A, Student.A, Student.B, Student.C, Student.D]
-    Select[Student.D > 30]
-      Join[Student.A = Enrolled.A]
-        Scan[Student]
-        Scan[Enrolled]
+```bash
+curl -s localhost:8080/tables
+# ["People"]
 
-=== Plan (optimized) ===
-Aggregate[group by: Student.B; calls: SUM(Student.c)]
-  Project[Enrolled.A, Student.A, Student.B, Student.C, Student.D]
-    Join[Student.A = Enrolled.A]
-      Select[Student.D > 30]
-        Scan[Student]
-      Project[Enrolled.A]
-        Scan[Enrolled]
+curl -s localhost:8080/queries -H 'Content-Type: application/json' \
+  -d '{"sql":"SELECT * FROM People LIMIT 5"}'
+# {"columns":[{"name":"id","type":"INT"},...],"rows":[[1,"alice"],...],"rowCount":5,"truncated":true,"hint":"..."}
 ```
 
-No query execution occurs for EXPLAIN queries.
-
-## Join algorithms
-
-The engine supports two join algorithms; the planner selects between them automatically.
+…or connect an AI agent over MCP (below). To use the engine directly from the command line instead, see the **[engine README](engine/README.md)**.
 
-### Nested-loop join
+## For agents: MCP
 
-`JoinOperator` implements a classic nested-loop join: for every outer tuple the inner child is rewound and scanned in full. It handles any join condition (equality, inequality, arbitrary expression, or cross product with no condition). `EXPLAIN` shows it as `Join[<condition>]`.
+The server exposes a Model Context Protocol endpoint at `http://localhost:8080/mcp` (Streamable-HTTP). Point an MCP client (e.g. Claude Desktop) at it and the agent gets five tools:
 
-### Hash join
-
-`HashJoinOperator` extends `JoinOperator` with an in-memory hash join. The inner (build) side is drained once into a `HashMap` keyed by the equality conjuncts; the outer (probe) side then streams through once. After a hash-table lookup, the full original condition is re-evaluated on every candidate, so residual non-equality conjuncts (e.g. `A.x = B.x AND A.y > 3`) work correctly. Output order — outer-major, inner order preserved within each key bucket — is identical to the nested-loop join. `EXPLAIN` shows it as `HashJoin[<condition>]`.
-
-**Auto-selection rule:** the planner chooses hash join when `Constants.useHashJoin` is `true` (the default) **and** the join condition contains at least one column-to-column equality conjunct (e.g. `Student.A = Enrolled.A`). Cross products (no condition) and pure non-equi joins (e.g. `A.x > B.y` only) always use nested-loop join.
-
-**Toggle:** set `Constants.useHashJoin = false` at program start (or in tests) to force nested-loop for all joins.
-
-### Benchmarks
-
-Performance was measured with a JMH 1.37 benchmark suite in the `bench/` package (`engine/src/test/java/com/github/jinba1/cuckoodb/bench/`). The suite is compiled in CI but never run there; run it locally with:
-
-```bash
-./mvnw -pl engine -q test-compile exec:exec -Dexec.executable=java -Dexec.classpathScope=test \
-  "-Dexec.args=-cp %classpath org.openjdk.jmh.Main .*Benchmark"
-```
-
-**Results** (OpenJDK 21.0.5, Intel Core i9-13900HX, 32 logical cores, Linux under WSL2):
-
-| Benchmark | matchesPerKey | rowsPerSide | useHashJoin | Mode | Cnt | Score | Error | Units |
-|-----------|--------------|-------------|-------------|------|-----|-------|-------|-------|
-| EndToEndJoinBenchmark.planAndDrain | N/A | N/A | true | avgt | 3 | 1.028 | ± 0.288 | ms/op |
-| EndToEndJoinBenchmark.planAndDrain | N/A | N/A | false | avgt | 3 | 315.523 | ± 36.021 | ms/op |
-| JoinAlgorithmBenchmark.hashJoin | 1 | 1000 | N/A | avgt | 5 | 0.270 | ± 0.011 | ms/op |
-| JoinAlgorithmBenchmark.hashJoin | 1 | 5000 | N/A | avgt | 5 | 1.382 | ± 0.154 | ms/op |
-| JoinAlgorithmBenchmark.hashJoin | 10 | 1000 | N/A | avgt | 5 | 2.160 | ± 0.109 | ms/op |
-| JoinAlgorithmBenchmark.hashJoin | 10 | 5000 | N/A | avgt | 5 | 10.661 | ± 0.840 | ms/op |
-| JoinAlgorithmBenchmark.nestedLoopJoin | 1 | 1000 | N/A | avgt | 5 | 202.621 | ± 25.313 | ms/op |
-| JoinAlgorithmBenchmark.nestedLoopJoin | 1 | 5000 | N/A | avgt | 5 | 5027.912 | ± 370.564 | ms/op |
-| JoinAlgorithmBenchmark.nestedLoopJoin | 10 | 1000 | N/A | avgt | 5 | 194.786 | ± 4.916 | ms/op |
-| JoinAlgorithmBenchmark.nestedLoopJoin | 10 | 5000 | N/A | avgt | 5 | 4785.620 | ± 212.683 | ms/op |
+| Tool | What it does |
+|---|---|
+| `list_tables` | list the available tables |
+| `describe_table` | a table's column names and types |
+| `sample_rows` | preview rows without writing SQL |
+| `explain_query` | preview a query's plan and cost before running it |
+| `query` | run a read-only `SELECT`, budget-bounded |
 
-`EndToEndJoinBenchmark` joins two 1 000-row CSV tables through the full planner pipeline; nested-loop re-parses the inner CSV once per outer row, so the gap (≈ 307×) reflects both the algorithmic difference and I/O cost. `JoinAlgorithmBenchmark` uses in-memory `CachedOperator` inputs to isolate the join algorithm itself; at 5 000 rows/side the operator-level gap is ≈ 3 600×.
+Every tool routes through the same guarded execution path as the REST API, so agent traffic inherits the read-only guarantee, the tuple/time budget, and concurrency limits (with audit hooks in place) — there is no way to bypass them.
 
-Benchmarks are compiled in CI but never executed there.
+## REST API
 
-## Demo
+| Endpoint | |
+|---|---|
+| `POST /queries` | plan + execute one read-only query → JSON columns/rows, or an `EXPLAIN` plan |
+| `GET /tables` | list table names |
+| `GET /tables/{name}` | a table's typed schema |
+| `/swagger-ui.html` | interactive OpenAPI docs |
 
-**Input table** (`engine/samples/db/data/Student.csv`):
+Queries are **budget-bounded and fail-closed**: the server always attaches a budget, so an unbounded query is unreachable. A result that would exceed the tuple budget returns `429` (retry with a tighter `LIMIT`); one that exceeds the time budget returns `504`.
 
-```
-A, B, C, D
-1, 200, 50, 33
-2, 200, 200, 44
-3, 100, 105, 44
-4, 100, 50, 11
-5, 100, 500, 22
-6, 300, 400, 11
-```
+### EXPLAIN
 
-**Query** (`engine/samples/input/query4.sql`):
+Any query can be planned without executing it — prefix `EXPLAIN` over REST, or call the `explain_query` tool. The plan is shown as written and after optimisation:
 
-```sql
-SELECT * FROM Student WHERE Student.A < 3;
 ```
+=== Plan (as written) ===
+Project[Student.B, Student.C]
+  Select[Student.D > 30]
+    HashJoin[Student.A = Enrolled.A]
+      Scan[Student]
+      Scan[Enrolled]
 
-**Command:**
-
-```bash
-java -cp engine/target/cuckoodb-engine-1.0.0-jar-with-dependencies.jar \
-  com.github.jinba1.cuckoodb.CuckooDB \
-  engine/samples/db engine/samples/input/query4.sql output.csv
+=== Plan (optimized) ===
+Project[Student.B, Student.C]
+  HashJoin[Student.A = Enrolled.A]
+    Select[Student.D > 30]
+      Scan[Student]
+    Project[Enrolled.A]
+      Scan[Enrolled]
 ```
 
-To limit resource usage, add optional budget flags:
+The optimiser pushes the `Select` below the join (selection pushdown) and projects the inner scan down to just the key it needs; the planner picked a hash join for the equi-condition. See the [engine README](engine/README.md#explain) for the full treatment.
 
-```bash
-java -cp engine/target/cuckoodb-engine-1.0.0-jar-with-dependencies.jar \
-  com.github.jinba1.cuckoodb.CuckooDB \
-  engine/samples/db engine/samples/input/query4.sql output.csv --max-tuples=10000 --timeout-ms=5000
-```
+## How it works
 
-**Output** (`output.csv`):
-
-```
-a,b,c,d
-1,200,50,33
-2,200,200,44
 ```
-
-## Running Examples
-
-The `engine/samples/` directory ships with 20 queries and a small dataset (Student, Course, Enrolled, Staff tables). Expected output lives in `engine/samples/expected_output/`.
-
-Run all 20 through the bundled runner, which diffs each result against the expected output and reports pass/fail. It is launched via `exec:exec` (not `exec:java`) so it runs with the engine module as the working directory — `exec:java` would keep the working directory at the reactor root and fail to find `samples/`:
-
-```bash
-./mvnw -pl engine -q test-compile exec:exec -Dexec.executable=java -Dexec.classpathScope=test \
-  "-Dexec.args=-cp %classpath com.github.jinba1.cuckoodb.SampleQueryRunner"
+SQL → JSqlParser → QueryPlanner → optimizer → operator tree → results
 ```
 
-Or run each query through the CLI and diff manually:
-
-```bash
-# Run all sample queries and diff against expected output
-for i in $(seq 1 20); do
-  java -cp engine/target/cuckoodb-engine-1.0.0-jar-with-dependencies.jar \
-    com.github.jinba1.cuckoodb.CuckooDB \
-    engine/samples/db "engine/samples/input/query${i}.sql" "/tmp/out${i}.csv"
-  diff "engine/samples/expected_output/query${i}.csv" "/tmp/out${i}.csv" && echo "query${i}: OK"
-done
-```
+The engine is a from-scratch Volcano/iterator executor — typed values, hash and nested-loop joins, selection pushdown, tuple/time budgets. The server wraps it behind a single `QueryService` choke point that applies the budget, a concurrency permit, and audit; **both** the REST controllers and the MCP tools go through it, so the guardrails can't be bypassed and apply uniformly. Engine internals — architecture, join algorithms, benchmarks, CLI — are in the **[engine README](engine/README.md)**.
 
-## Testing
+## Build and test
 
 ```bash
-./mvnw test
+./mvnw clean verify    # builds + tests both modules: engine (419 tests) + server (90 tests)
 ```
 
-The test suite covers individual operators, the query planner, the optimiser, expression evaluation, query budgets, EXPLAIN, hash join, and end-to-end integration scenarios (339 tests).
+The 20 sample queries are a golden-output regression gate (see the engine README to run them). CI builds, tests, and publishes the container image to GHCR on every merge to `main`.
 
-## Project Structure
+## Project structure
 
 ```
-├── pom.xml                                          # Parent POM (aggregator: engine + server; Java 17, dep/plugin management)
-├── engine/                                          # Pure query engine — zero Spring dependencies
-│   ├── pom.xml                                      # cuckoodb-engine (JSqlParser 4.7, commons-csv 1.14.1, JMH 1.37 test-scope)
-│   ├── src/main/java/com/github/jinba1/cuckoodb/    # Core engine (35 files)
-│   │   └── operator/                                # Volcano operators (11 files, incl. HashJoinOperator)
-│   ├── src/test/java/com/github/jinba1/cuckoodb/    # JUnit 5 tests (339 tests across 33 files)
-│   └── samples/
-│       ├── db/data/                                 # CSV data files (header row + data rows)
-│       ├── input/query[1-20].sql                    # Sample queries
-│       └── expected_output/query[1-20].csv          # Expected results
-├── server/                                          # cuckoodb-server — Spring Boot REST + MCP gateway over the engine
-│   ├── pom.xml                                      # Spring Boot 4 (web MVC), springdoc/OpenAPI, Spring AI MCP server
-│   └── src/main/java/com/github/jinba1/cuckoodb/server/   # web/ controllers, query/ service, catalog/ facade, mcp/ agent tools, config
-├── mvnw / mvnw.cmd                                  # Maven Wrapper
-└── LICENSE
+├── engine/   # pure query engine — Java 17, zero Spring (see engine/README.md)
+└── server/   # Spring Boot 4 gateway — REST + MCP over the engine
 ```
 
 ## Background
 
-Originally built as a university project for the Advanced Database Systems course at the University of Edinburgh, subsequently extended with additional query optimisation and expanded test coverage.
+Originally built as a university project for the Advanced Database Systems course at the University of Edinburgh, then extended into a guarded, agent-facing gateway — REST and MCP interfaces, query budgets, and additional optimisation and test coverage.
 
 ## License
 
-This project is released under the MIT License. See [LICENSE](LICENSE) for details.
+Released under the MIT License. See [LICENSE](LICENSE).
diff --git a/engine/README.md b/engine/README.md
new file mode 100644
index 0000000..9e93b01
--- /dev/null
+++ b/engine/README.md
@@ -0,0 +1,166 @@
+# cuckooDB — query engine
+
+The query engine under the [cuckooDB gateway](../README.md): an in-memory relational query engine on the Volcano/iterator model. It parses SQL via JSqlParser, builds an operator tree, optimises it, and executes tuple-by-tuple against CSV files. Pure Java 17, **zero Spring dependencies**.
+
+## Architecture
+
+```
+SQL → JSqlParser → QueryPlanner → QueryPlanOptimizer → Operator Tree → Results
+```
+
+| Component | Role |
+|-----------|------|
+| `QueryPlanner` | Parses SQL and builds the operator pipeline |
+| `QueryPlanOptimizer` | Selection pushdown, trivial operator removal |
+| `DBCatalog` | Schema and table metadata (singleton) |
+| `Value` | Typed tuple values (sealed interface: `IntValue`, `StringValue`) |
+| `ExpressionEvaluator` | Evaluates WHERE/HAVING conditions per tuple |
+| `ExpressionPreprocessor` | Resolves column references to indices |
+| `ConditionSplitter` | Separates join predicates from selection predicates |
+
+**Operator hierarchy** (all extend `Operator`):
+
+`ScanOperator` → `SelectOperator` → `ProjectOperator` → `JoinOperator` / `HashJoinOperator` → `SortOperator` → `AggregateOperator` → `DuplicateEliminationOperator` → `LimitOperator`
+
+## Scope
+
+Read-only SQL-over-CSV: `SELECT`/`FROM`/`WHERE`, inner joins, `GROUP BY` with `SUM`/`COUNT`/`AVG`/`MIN`/`MAX`, `ORDER BY`, `DISTINCT`, `LIMIT n`, and nested arithmetic/comparison expressions. Values are typed int or string, inferred per column from the data. Tables are discovered from CSV files with header rows; no separate schema file. No transactions, indexes, data modification, persistence, or full SQL dialect — the focus is query planning, optimisation, and the Volcano execution model.
+
+## Build and run (CLI)
+
+Run from the repository root (uses the Maven Wrapper):
+
+```bash
+./mvnw -pl engine -DskipTests clean package
+
+java -cp engine/target/cuckoodb-engine-1.0.0-jar-with-dependencies.jar \
+  com.github.jinba1.cuckoodb.CuckooDB \
+  <database_dir> <input_file> <output_file> [--max-tuples=N] [--timeout-ms=N]
+```
+
+`<database_dir>` is a directory containing a `data/` subdir of `.csv` tables. `--max-tuples` and `--timeout-ms` are optional and independent — use one, both, or neither.
+
+### Demo
+
+**Input** (`engine/samples/db/data/Student.csv`):
+
+```
+A, B, C, D
+1, 200, 50, 33
+2, 200, 200, 44
+3, 100, 105, 44
+4, 100, 50, 11
+5, 100, 500, 22
+6, 300, 400, 11
+```
+
+**Command** (`engine/samples/input/query4.sql` is `SELECT * FROM Student WHERE Student.A < 3;`):
+
+```bash
+java -cp engine/target/cuckoodb-engine-1.0.0-jar-with-dependencies.jar \
+  com.github.jinba1.cuckoodb.CuckooDB \
+  engine/samples/db engine/samples/input/query4.sql output.csv
+```
+
+**Output** (`output.csv`):
+
+```
+a,b,c,d
+1,200,50,33
+2,200,200,44
+```
+
+## Query budgets
+
+The engine enforces **total-work semantics**: every tuple emitted by any operator counts against the budget, including intermediate tuples later filtered or joined. A cross-product explosion that never produces output rows still hits the tuple limit. The timeout clock starts lazily at the first tuple emission. On a breach the partial output file is deleted, `Error: <message>` is written to stderr, and the process exits 1.
+
+## EXPLAIN
+
+Prefix any query with `EXPLAIN` to inspect the plan without executing it. The output has two sections — as written, then after optimisation:
+
+```
+=== Plan (as written) ===
+Aggregate[group by: Student.B; calls: SUM(Student.c)]
+  Project[Enrolled.A, Student.A, Student.B, Student.C, Student.D]
+    Select[Student.D > 30]
+      HashJoin[Student.A = Enrolled.A]
+        Scan[Student]
+        Scan[Enrolled]
+
+=== Plan (optimized) ===
+Aggregate[group by: Student.B; calls: SUM(Student.c)]
+  Project[Enrolled.A, Student.A, Student.B, Student.C, Student.D]
+    HashJoin[Student.A = Enrolled.A]
+      Select[Student.D > 30]
+        Scan[Student]
+      Project[Enrolled.A]
+        Scan[Enrolled]
+```
+
+The optimiser pushes the `Select` below the join (selection pushdown) and inserts a projection on the inner scan; the planner picked a hash join for the equi-condition. No execution occurs for `EXPLAIN`.
+
+## Join algorithms
+
+The planner selects between two join algorithms automatically.
+
+**Nested-loop** (`JoinOperator`): for every outer tuple the inner child is rewound and scanned in full. Handles any condition — equality, inequality, arbitrary expression, or cross product. Shown in `EXPLAIN` as `Join[<condition>]`.
+
+**Hash** (`HashJoinOperator extends JoinOperator`): the inner (build) side is drained once into a `HashMap` keyed by the equality conjuncts; the outer (probe) side streams through once. After a lookup, the full original condition is re-evaluated on every candidate, so residual non-equality conjuncts (e.g. `A.x = B.x AND A.y > 3`) work. Output order is identical to nested-loop (outer-major, inner order preserved per key bucket). Shown as `HashJoin[<condition>]`.
+
+**Auto-selection:** hash join is used when `Constants.useHashJoin` is `true` (default) **and** the condition has at least one column-to-column equality conjunct. Cross products and pure non-equi joins always use nested-loop. Set `Constants.useHashJoin = false` to force nested-loop everywhere.
+
+### Benchmarks
+
+A JMH 1.37 suite lives in `engine/src/test/java/com/github/jinba1/cuckoodb/bench/`. It is compiled in CI but never run there; run it locally from the repository root:
+
+```bash
+./mvnw -pl engine -q test-compile exec:exec -Dexec.executable=java -Dexec.classpathScope=test \
+  "-Dexec.args=-cp %classpath org.openjdk.jmh.Main .*Benchmark"
+```
+
+**Results** (OpenJDK 21.0.5, Intel Core i9-13900HX, 32 logical cores, Linux under WSL2):
+
+| Benchmark | matchesPerKey | rowsPerSide | useHashJoin | Mode | Cnt | Score | Error | Units |
+|-----------|--------------|-------------|-------------|------|-----|-------|-------|-------|
+| EndToEndJoinBenchmark.planAndDrain | N/A | N/A | true | avgt | 3 | 1.028 | ± 0.288 | ms/op |
+| EndToEndJoinBenchmark.planAndDrain | N/A | N/A | false | avgt | 3 | 315.523 | ± 36.021 | ms/op |
+| JoinAlgorithmBenchmark.hashJoin | 1 | 1000 | N/A | avgt | 5 | 0.270 | ± 0.011 | ms/op |
+| JoinAlgorithmBenchmark.hashJoin | 1 | 5000 | N/A | avgt | 5 | 1.382 | ± 0.154 | ms/op |
+| JoinAlgorithmBenchmark.hashJoin | 10 | 1000 | N/A | avgt | 5 | 2.160 | ± 0.109 | ms/op |
+| JoinAlgorithmBenchmark.hashJoin | 10 | 5000 | N/A | avgt | 5 | 10.661 | ± 0.840 | ms/op |
+| JoinAlgorithmBenchmark.nestedLoopJoin | 1 | 1000 | N/A | avgt | 5 | 202.621 | ± 25.313 | ms/op |
+| JoinAlgorithmBenchmark.nestedLoopJoin | 1 | 5000 | N/A | avgt | 5 | 5027.912 | ± 370.564 | ms/op |
+| JoinAlgorithmBenchmark.nestedLoopJoin | 10 | 1000 | N/A | avgt | 5 | 194.786 | ± 4.916 | ms/op |
+| JoinAlgorithmBenchmark.nestedLoopJoin | 10 | 5000 | N/A | avgt | 5 | 4785.620 | ± 212.683 | ms/op |
+
+`EndToEndJoinBenchmark` joins two 1 000-row CSV tables through the full pipeline; nested-loop re-parses the inner CSV once per outer row, so the ≈ 307× gap reflects both algorithm and I/O. `JoinAlgorithmBenchmark` uses in-memory inputs to isolate the algorithm; at 5 000 rows/side the operator-level gap is ≈ 3 600×.
+
+## Sample queries
+
+`engine/samples/` ships 20 queries and a small dataset (Student, Course, Enrolled, Staff). The bundled runner diffs each result against `engine/samples/expected_output/` — the golden-output regression gate. Launch via `exec:exec` (not `exec:java`) so it runs with the engine module as the working directory:
+
+```bash
+./mvnw -pl engine -q test-compile exec:exec -Dexec.executable=java -Dexec.classpathScope=test \
+  "-Dexec.args=-cp %classpath com.github.jinba1.cuckoodb.SampleQueryRunner"
+```
+
+## Testing
+
+```bash
+./mvnw -pl engine test
+```
+
+419 tests across operators, the planner, the optimiser, expression evaluation, query budgets, EXPLAIN, hash join, and end-to-end integration scenarios.
+
+## Layout
+
+```
+engine/
+├── src/main/java/com/github/jinba1/cuckoodb/   # core engine (45 files)
+│   └── operator/                                # Volcano operators (11 files, incl. HashJoinOperator)
+├── src/test/java/com/github/jinba1/cuckoodb/    # JUnit 5 tests (419 across 41 files)
+└── samples/
+    ├── db/data/                                 # CSV tables (header row + data rows)
+    ├── input/query[1-20].sql
+    └── expected_output/query[1-20].csv
+```

From 45385ea4932b141828b762bb659ce3dfc93d66bf Mon Sep 17 00:00:00 2001
From: JinBa1 <72070041+JinBa1@users.noreply.github.com>
Date: Fri, 7 Aug 2026 23:05:46 +0100
Subject: [PATCH 2/2] docs: sync AGENTS.md and docs with QueryConfig and server
 state

AGENTS.md was generated before the server module and the QueryConfig /
PlanContext refactor, so it still described static mutable flags in
Constants, a DBCatalog that owned intermediate schemas, and a no-op
EXPLAIN plan root. Regenerate it against the current tree: server module
and its dependencies, the new config/context/result types, the reworked
catalog and budget APIs, current test counts, the packaging-based CI job,
and the GHCR publish workflow.

Also correct the surrounding docs: the README's blanket "read-only"
framing predates the opt-in CSV upload endpoint and is narrowed to the
query path, BENCHMARK_RESULTS.md no longer claims benchmarks flip a
mutable global, and engine/README.md points at QueryConfig instead of
Constants for join selection.
---
 AGENTS.md            | 66 +++++++++++++++++++++++++-------------------
 BENCHMARK_RESULTS.md | 12 ++++----
 README.md            | 20 ++++++++------
 engine/README.md     |  4 +--
 4 files changed, 58 insertions(+), 44 deletions(-)

diff --git a/AGENTS.md b/AGENTS.md
index 6fd1876..513b7cb 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -1,12 +1,12 @@
 # PROJECT KNOWLEDGE BASE
 
-**Generated:** 2026-06-12
-**Commit:** 158fe91
-**Branch:** feat/sql-aggregates-limit
+**Generated:** 2026-07-02
+**Commit:** 1abe17f
+**Branch:** docs/readme-rewrite
 
 ## OVERVIEW
 
-CuckooDB (engine module artifactId: `cuckoodb-engine`; parent: `cuckoodb-parent`) is an in-memory relational query engine built on the Volcano/iterator model. It parses SQL via JSqlParser, builds an operator tree, and executes tuple-at-a-time over CSV data. The project targets Java 17 and uses JUnit Jupiter 5.10.2, JSqlParser 4.7, and JaCoCo 0.8.12.
+CuckooDB (engine module artifactId: `cuckoodb-engine`; server module artifactId: `cuckoodb-server`; parent: `cuckoodb-parent`) is an in-memory relational query engine plus a Spring Boot REST/MCP gateway. The engine parses SQL via JSqlParser, builds a Volcano/iterator operator tree, and executes tuple-at-a-time over CSV data. The server exposes guarded query execution through REST and MCP with fail-closed budgets, concurrency limits, audit hooks, and an opt-in CSV upload endpoint. The project targets Java 17 and uses JSqlParser 4.7, commons-csv 1.14.1, JUnit Jupiter 5.10.2 in the engine, JMH 1.37 for benchmarks, Spring Boot 4.0.7, springdoc 3.0.3, Spring AI 2.0.0, and JaCoCo 0.8.12.
 
 ## STRUCTURE
 
@@ -14,7 +14,8 @@ CuckooDB (engine module artifactId: `cuckoodb-engine`; parent: `cuckoodb-parent`
 cuckoodb-parent/                       # repo root (git slug: java-query-engine)
 ├── pom.xml                            # Parent POM: packaging=pom, modules engine+server, dependency/plugin management (Java 17)
 ├── .gitignore                         # ignores target/, *.class, .iml, IDE metadata, engine/ test-output dirs
-├── .github/workflows/ci.yml           # push/PR to main: ./mvnw clean compile + test + Codecov (engine/target/site/jacoco/jacoco.xml)
+├── .github/workflows/ci.yml           # push/PR to main: ./mvnw clean package + Codecov (engine/target/site/jacoco/jacoco.xml)
+├── .github/workflows/docker-publish.yml # PR image build validation; push main/tags publishes GHCR image
 ├── mvnw / mvnw.cmd / .mvn/            # Maven Wrapper (shared, at root)
 ├── engine/                            # cuckoodb-engine — pure query engine, ZERO Spring deps
 │   ├── pom.xml                        # JSqlParser 4.7, commons-csv 1.14.1, JUnit 5.10.2, JMH 1.37 (test); exec + assembly + jacoco
@@ -23,16 +24,17 @@ cuckoodb-parent/                       # repo root (git slug: java-query-engine)
 │   │   ├── input/query[1-20].sql      # 20 sample queries
 │   │   └── expected_output/query[1-20].csv   # expected results
 │   └── src/
-│       ├── main/java/com/github/jinba1/cuckoodb/   # 35 core files (CuckooDB entry, planner, optimizer, catalog, budgets, Value/Tuple)
+│       ├── main/java/com/github/jinba1/cuckoodb/   # 34 core files (CuckooDB entry, planner, optimizer, catalog, budgets, Value/Tuple, result DTOs)
 │       │   └── operator/                           # 11 Volcano operators (base + Scan/Select/Project/Join/HashJoin/Sort/Aggregate/Limit/DuplicateElimination + Accumulator)
-│       └── test/java/com/github/jinba1/cuckoodb/   # 33 test files (339 tests)
+│       └── test/java/com/github/jinba1/cuckoodb/   # 41 test files (419 tests)
 │           ├── CuckooDBTest.java                   # incl. testAllSampleQueries — the 20-sample byte-identical gate
 │           ├── ConcurrentQueryExecutionTest.java, DBCatalogTest.java, ...   # planner / optimizer / budget / EXPLAIN / end-to-end
 │           ├── operator/                           # operator-level tests + CachedOperator test utility
 │           └── bench/                              # JMH benchmarks (compiled in CI, never run there): EndToEndJoinBenchmark, JoinAlgorithmBenchmark
 └── server/                            # cuckoodb-server — Spring Boot 4 REST + MCP gateway over the engine
     ├── pom.xml                        # depends on cuckoodb-engine; Spring Boot 4.0.7 (web MVC), springdoc/OpenAPI, Spring AI 2.0.0 MCP server
-    └── src/main/java/com/github/jinba1/cuckoodb/server/   # web/ controllers + GlobalExceptionHandler, query/ QueryService+budget+concurrency, catalog/ CatalogFacade, mcp/ CuckooMcpTools (5 @McpTool tools over the QueryService choke point) + TableNameValidator/CatalogMapper, audit/ sink, config/ (90 server tests)
+    ├── src/main/java/com/github/jinba1/cuckoodb/server/   # web/ controllers + GlobalExceptionHandler, query/ QueryService+budget+concurrency, catalog/ CatalogFacade, mcp/ CuckooMcpTools (5 @McpTool tools over the QueryService choke point) + TableNameValidator/CatalogMapper, audit/ sink, config/
+    └── src/test/java/com/github/jinba1/cuckoodb/server/   # 13 server test classes (90 tests)
 ```
 
 Per-file responsibilities are in the WHERE TO LOOK table below.
@@ -41,14 +43,17 @@ Per-file responsibilities are in the WHERE TO LOOK table below.
 
 | File | Package | Notes |
 |------|---------|-------|
-| `CuckooDB.java` | `com.github.jinba1.cuckoodb` | Entry point. `public static void main(String[] args)` takes db-dir, input SQL file, output CSV path, and optional `--max-tuples=N` / `--timeout-ms=N` flags. `static int run(String[] args)` is main minus System.exit (used in tests). Parses via `QueryPlanner.planQuery`, attaches a `QueryBudget` when flags are present, executes via `execute()`, and writes RFC 4180 output (LF line endings). On budget exceeded: deletes partial output, writes `Error: <message>` to stderr, exits 1. |
-| `QueryPlanner.java` | `com.github.jinba1.cuckoodb` | Translates a SQL file into a `PlannedQuery` via `planQuery(filename)`. For EXPLAIN-prefixed queries, renders before/after operator trees via `PlanPrinter`; returned root is a no-op for EXPLAIN. Builds the scan/join/select/project/group-by/sort/distinct/limit pipeline. Auto-selects `HashJoinOperator` when `Constants.useHashJoin` is true and the join condition contains a column=column equality conjunct; falls back to `JoinOperator` otherwise. |
-| `QueryBudget.java` | `com.github.jinba1.cuckoodb` | Holds per-query kill limits: `QueryBudget(Long maxTuples, Long timeoutMs)` — both nullable (no limit on that axis). `charge()` increments the tuple counter and checks the wall-clock timeout (lazy start at first call); throws `QueryBudgetExceededException` when either limit is exceeded. `processed()` returns the total tuple count so far. |
+| `CuckooDB.java` | `com.github.jinba1.cuckoodb` | CLI entry point. `public static void main(String[] args)` takes db-dir, input SQL file, output CSV path, and optional `--max-tuples=N` / `--timeout-ms=N` flags. `static int run(String[] args)` is main minus System.exit (used in tests). Parses via `QueryPlanner.planQuery(..., QueryConfig.defaults())`; EXPLAIN writes plan text and executes nothing; executed queries attach a `QueryBudget` only when CLI flags are present. `execute()` writes RFC 4180 output (LF line endings) and returns `QueryResult`; `executeToResultSet()` is the in-memory REST/library path. On execution failure: deletes partial output, writes `Error: <message>` to stderr, exits 1. |
+| `QueryPlanner.java` | `com.github.jinba1.cuckoodb` | Translates a SQL file into a `PlannedQuery` via `planQuery(filename[, QueryConfig])`; `planSql(sql, QueryConfig)` is the REST/library string-SQL path. For EXPLAIN-prefixed queries, renders before/after operator trees via `PlanPrinter`; returned root is still the optimized executable tree, while callers short-circuit when `explainText` is non-null. Builds the scan/join/select/project/group-by/sort/distinct/limit pipeline. Auto-selects `HashJoinOperator` when `ctx.config().useHashJoin()` is true and the join condition contains a column=column equality conjunct; falls back to `JoinOperator` otherwise. |
+| `QueryConfig.java` | `com.github.jinba1.cuckoodb` | Immutable per-query planner config: `useQueryOptimization`, `useHashJoin`. `defaults()` enables both. Replaces the former mutable static flags so concurrent queries and benchmarks do not race on global state. |
+| `PlanContext.java` | `com.github.jinba1.cuckoodb` | One per planned query. Owns intermediate schemas, schema parents, column origins, and the query's `QueryConfig`; delegates base-table lookups to `DBCatalog`. Prevents intermediate-schema accumulation across queries and isolates concurrent plans. |
+| `QueryBudget.java` | `com.github.jinba1.cuckoodb` | Holds per-query kill limits: `QueryBudget(Long maxTuples, Long timeoutMs)` — both nullable in the engine (no limit on that axis). `charge()` increments the tuple counter and checks the wall-clock timeout (lazy start at first call); `checkDeadline()` checks time without counting a tuple for blocking phases; throws `QueryBudgetExceededException` when either limit is exceeded. `processed()` returns the total tuple count so far. |
 | `QueryBudgetExceededException.java` | `com.github.jinba1.cuckoodb` | Unchecked exception thrown by `QueryBudget.charge()` when either the tuple limit or timeout is exceeded. Carries a human-readable message stating which limit was hit. |
-| `PlannedQuery.java` | `com.github.jinba1.cuckoodb` | Record: `PlannedQuery(Operator root, String explainText)`. `explainText` is non-null only for EXPLAIN queries; `root` is a no-op for EXPLAIN queries. |
+| `PlannedQuery.java` | `com.github.jinba1.cuckoodb` | Record: `PlannedQuery(Operator root, String explainText)`. `explainText` is non-null only for EXPLAIN queries; `root` is the optimized executable plan in all cases, but callers skip execution when `explainText` is present. |
 | `PlanPrinter.java` | `com.github.jinba1.cuckoodb` | Utility: `static String print(Operator root)` walks the operator tree depth-first and renders it as an indented text plan using each operator's `describe()` line. |
-| `QueryPlanOptimizer.java` | `com.github.jinba1.cuckoodb` | Optimization passes: selection pushdown, trivial project/select removal, consecutive-select merging, projection pushdown. Toggled by Constants.useQueryOptimization. |
-| `DBCatalog.java` | `com.github.jinba1.cuckoodb` | Mutable singleton: CSV-header table discovery + INT/STRING type inference at init. Table-to-path map (dbLocations), table schemas (dbSchemata), column types (dbColumnTypes). initDBCatalog(dir) / resetDBCatalog(); getColumnTypes(), getOrderedColumnNames(). No schema.txt. |
+| `QueryPlanOptimizer.java` | `com.github.jinba1.cuckoodb` | Optimization passes: selection pushdown, trivial project/select removal, consecutive-select merging, projection pushdown. Runs when `QueryConfig.useQueryOptimization()` is true. |
+| `DBCatalog.java` | `com.github.jinba1.cuckoodb` | Engine singleton for durable base-table metadata. Discovers CSV-header tables under `data/`, infers INT/STRING types, stores one immutable `TableMeta` per table in a `ConcurrentHashMap`, and supports runtime `registerTable` for the server upload path. Per-query intermediate schemas live in `PlanContext`, not here. initDBCatalog(dir) / resetDBCatalog(); no schema.txt. |
+| `TableMeta.java` | `com.github.jinba1.cuckoodb` | Record grouping a table's backing `Path`, schema map, and column types so catalog readers see one atomic registration. |
 | `ExpressionEvaluator.java` | `com.github.jinba1.cuckoodb` | JSqlParser visitor evaluating WHERE/HAVING expressions against a Tuple (boolean and Value evaluation). Type-checked comparisons; string literals via StringValue. |
 | `ExpressionPreprocessor.java` | `com.github.jinba1.cuckoodb` | JSqlParser visitor that separates two-table join predicates from single-table selection predicates during planning. |
 | `ConditionSplitter.java` | `com.github.jinba1.cuckoodb` | Splits a join condition into outer-only, inner-only, and true join predicate parts (used by optimizer pushdown). Uses Constants.INTERMEDIATE_SCHEMA_PREFIX. |
@@ -57,7 +62,7 @@ Per-file responsibilities are in the WHERE TO LOOK table below.
 | `Tuple.java` | `com.github.jinba1.cuckoodb` | Row of typed values: wraps List<Value>; getAttribute(i) returns Value, toString() (comma-space separated), equals/hashCode. |
 | `TupleComparator.java` | `com.github.jinba1.cuckoodb` | Comparator<Tuple> for multi-column lexicographic sorting by column indices. |
 | `SchemaTransformationType.java` | `com.github.jinba1.cuckoodb` | Enum marking the kind of schema transformation an operator performs. |
-| `Constants.java` | `com.github.jinba1.cuckoodb` | App constants: useQueryOptimization (boolean, default true), useHashJoin (boolean, default true — set false to force nested-loop for all joins), INTERMEDIATE_SCHEMA_PREFIX = "temp_", DATA_DIRECTORY_NAME = "data". |
+| `Constants.java` | `com.github.jinba1.cuckoodb` | App constants only: `INTERMEDIATE_SCHEMA_PREFIX = "temp_"`, `DATA_DIRECTORY_NAME = "data"`. Query flags live in `QueryConfig`. |
 | `AggregateFunction.java` | `com.github.jinba1.cuckoodb` | Enum of supported aggregate functions: SUM, COUNT, AVG, MIN, MAX. `fromFunctionName(String)` maps SQL function names (case-insensitive) to enum values; returns null for unrecognised names. |
 | `AggregateCall.java` | `com.github.jinba1.cuckoodb` | Record holding one parsed aggregate call from the SELECT list: `function` (AggregateFunction), `argument` (JSqlParser Expression; null for COUNT(*)), and `schemaKey` (the output column name as registered). |
 | `SampleQueryRunner.java` | `com.github.jinba1.cuckoodb` | Standalone main that runs all 20 sample queries against samples/db and diffs each output against samples/expected_output/, reporting pass/fail. |
@@ -65,13 +70,16 @@ Per-file responsibilities are in the WHERE TO LOOK table below.
 | `IntValue.java` | `com.github.jinba1.cuckoodb` | Record implementing Value; wraps int v(); compareTo orders numerically. |
 | `StringValue.java` | `com.github.jinba1.cuckoodb` | Record implementing Value; wraps String v(); compareTo orders lexicographically. |
 | `ColumnType.java` | `com.github.jinba1.cuckoodb` | Enum: INT, STRING. Used by DBCatalog and ScanOperator. |
+| `ColumnMeta.java` | `com.github.jinba1.cuckoodb` | REST/library output-column metadata for `QueryResultSet`: bare name, optional qualified origin, best-effort runtime type. Rows remain positional because result column names can duplicate. |
+| `QueryResult.java` | `com.github.jinba1.cuckoodb` | CLI execution metadata: row count, truncation flag, and refine hint when LIMIT cut the result short. |
+| `QueryResultSet.java` | `com.github.jinba1.cuckoodb` | Fully materialized in-memory result for REST/library callers: column metadata, positional rows, truncation flag, and hint. |
 | `QueryExecutionException.java` | `com.github.jinba1.cuckoodb` | Unchecked exception for data/type errors at runtime; messages state operation, column/literal, and both types for agent-legible diagnostics. |
-| `Operator.java` | `com.github.jinba1.cuckoodb.operator` | Abstract base for all Volcano operators. Defines `getNextTuple()`, `reset()`, `describe()`, schema methods (`propagateSchemaId`, `registerSchema`, `ensureSchemaRegistered`, `updateSchema`). Holds `protected Operator child`, a schemaRegistered flag, `protected long tupleCounter` (benchmarking via `getTupleCount()`/`resetTupleCount()`). Budget methods: `attachBudget(QueryBudget)` propagates the budget to the whole subtree; `protected final void countTuple()` must be called by each concrete operator's `getNextTuple()` on every non-null tuple — this is what enforces total-work semantics. Abstract `describe()` returns a one-line description for `PlanPrinter`. |
+| `Operator.java` | `com.github.jinba1.cuckoodb.operator` | Abstract base for all Volcano operators. Defines `getNextTuple()`, `reset()`, `describe()`, schema methods (`propagateSchemaId`, `registerSchema`, `ensureSchemaRegistered`, `updateSchema`). Holds `protected Operator child`, a `PlanContext`, a schemaRegistered flag, `protected long tupleCounter` (benchmarking via `getTupleCount()`/`resetTupleCount()`). Budget methods: `attachBudget(QueryBudget)` propagates the budget to the whole subtree; `protected final void countTuple()` must be called by each concrete operator's `getNextTuple()` on every non-null tuple; `checkBudgetDeadline()` covers blocking build/drain loops without double-counting. Abstract `describe()` returns a one-line description for `PlanPrinter`. |
 | `ScanOperator.java` | `com.github.jinba1.cuckoodb.operator` | Leaf operator; reads CSV via commons-csv RFC 4180, skips header row, emits typed tuples using column types from DBCatalog. |
 | `SelectOperator.java` | `com.github.jinba1.cuckoodb.operator` | Unary filter; applies a WHERE Expression via ExpressionEvaluator, passing through matching tuples. |
 | `ProjectOperator.java` | `com.github.jinba1.cuckoodb.operator` | Unary; projects a subset of columns, rewriting the schema. |
 | `JoinOperator.java` | `com.github.jinba1.cuckoodb.operator` | Binary nested-loop join; has outerChild and child (inner); optional join condition; propagates merged schema. Used for cross products and pure non-equi joins. |
-| `HashJoinOperator.java` | `com.github.jinba1.cuckoodb.operator` | Extends `JoinOperator`. Build phase drains the inner child into a `HashMap` keyed by equality-conjunct column values; probe phase streams the outer child and probes the map. Re-evaluates the full original condition on every candidate to handle residual non-equi conjuncts. Output order and EXPLAIN label (`HashJoin[...]`) differ from `JoinOperator` (`Join[...]`); auto-selected by `QueryPlanner` for equi-joins when `Constants.useHashJoin` is true. |
+| `HashJoinOperator.java` | `com.github.jinba1.cuckoodb.operator` | Extends `JoinOperator`. Build phase drains the inner child into a `HashMap` keyed by equality-conjunct column values; probe phase streams the outer child and probes the map. Re-evaluates the full original condition on every candidate to handle residual non-equi conjuncts. Output order and EXPLAIN label (`HashJoin[...]`) differ from `JoinOperator` (`Join[...]`); auto-selected by `QueryPlanner` for equi-joins when `QueryConfig.useHashJoin()` is true. |
 | `SortOperator.java` | `com.github.jinba1.cuckoodb.operator` | Unary; materializes child tuples then sorts via TupleComparator by ORDER BY columns. |
 | `AggregateOperator.java` | `com.github.jinba1.cuckoodb.operator` | Blocking operator for GROUP BY + SUM/COUNT/AVG/MIN/MAX. Groups tuples by key columns, creates one Accumulator per aggregate call per group, then emits one output tuple per group. |
 | `Accumulator.java` | `com.github.jinba1.cuckoodb.operator` | Package-private interface for per-group, per-call aggregate state. `add(Value)` folds one row's argument; `result()` returns the final Value. Static factory `create(AggregateCall)` dispatches to IntSumAccumulator (SUM/AVG), CountAccumulator, or MinMaxAccumulator. |
@@ -81,22 +89,22 @@ Per-file responsibilities are in the WHERE TO LOOK table below.
 ## CONVENTIONS
 
 - **Iterator model:** All operators extend `Operator` and implement `getNextTuple()` / `reset()`. Unary operators use `protected Operator child`; `JoinOperator` adds `outerChild` for its outer input.
-- **Schema tracking:** Operators register schema transformations with `DBCatalog`; intermediate schemas get the `Constants.INTERMEDIATE_SCHEMA_PREFIX` (`"temp_"`) prefix. `INTERMEDIATE_SCHEMA_PREFIX` is used in production code in `QueryPlanOptimizer`, `ConditionSplitter`, and `JoinOperator`.
+- **Schema tracking:** Operators register schema transformations with their per-query `PlanContext`; durable base-table metadata stays in `DBCatalog`. Intermediate schemas get the `Constants.INTERMEDIATE_SCHEMA_PREFIX` (`"temp_"`) prefix. `INTERMEDIATE_SCHEMA_PREFIX` is used in production code in `PlanContext`, `QueryPlanOptimizer`, `ConditionSplitter`, `CuckooDB`, and `JoinOperator`.
 - **Lazy schema registration:** The `schemaRegistered` flag and `ensureSchemaRegistered()` pattern is used across operators to defer schema registration until first use.
-- **No dependency injection:** `DBCatalog` is a mutable singleton. Tests call `resetDBCatalog()` in their `@BeforeEach` setup (JUnit 5).
+- **Catalog lifecycle:** The engine still uses the `DBCatalog` singleton for base-table metadata; engine tests generally call `resetDBCatalog()` in `@BeforeEach`. The server wraps the singleton behind Spring-managed `CatalogFacade` / `CatalogInitializer`; server integration tests isolate catalog state with Spring test contexts.
 - **Typed values:** `Tuple` stores `List<Value>` (sealed: `IntValue`, `StringValue`). Column types inferred at catalog init (INT iff every field parses as int). Column names stored lowercase.
 - **Output format:** Query output begins with a header row (column names, plain commas), followed by data rows (plain comma-separated, LF line endings, RFC 4180). Output is round-trippable as input.
 
 ## COMMANDS
 
 ```bash
-# Run the full test suite (339 tests)
+# Run the full test suite (419 engine tests + 90 server tests)
 ./mvnw test
 
 # Build the fat JAR (engine module; assembly is bound to the package phase)
 ./mvnw -pl engine -DskipTests clean package
 
-# Or build the whole reactor (engine fat JAR + server skeleton), running tests
+# Or build/package the whole reactor (engine fat JAR + server Boot JAR), running tests
 ./mvnw clean package
 
 # The fat JAR is always produced at:
@@ -136,20 +144,22 @@ java -cp engine/target/cuckoodb-engine-1.0.0-jar-with-dependencies.jar \
 The project ships `.github/workflows/ci.yml` which triggers on push and pull_request to `main`. It runs on `ubuntu-latest` with JDK 17 (Temurin distribution) and caches `~/.m2`.
 
 Steps:
-1. **Build** — `./mvnw clean compile`
-2. **Test** — `./mvnw test` (339 tests; JMH benchmark classes compile here but are not executed)
-3. **Coverage upload** — uploads JaCoCo coverage reports to Codecov via `codecov/codecov-action@v5` (token from `secrets.CODECOV_TOKEN`). JaCoCo plugin version 0.8.12 generates the coverage report.
+1. **Build, test, package** — `./mvnw clean package` for both modules; this also exercises the Spring Boot repackaging step so a broken server jar cannot ship.
+2. **Coverage upload** — uploads JaCoCo coverage reports to Codecov via `codecov/codecov-action@v5` (token from `secrets.CODECOV_TOKEN`). JaCoCo plugin version 0.8.12 generates the coverage report.
 
-The README displays CI, Coverage (Codecov), and Dependencies badges at the top.
+The project also ships `.github/workflows/docker-publish.yml`: PRs build the server container without pushing; pushes to `main` and `v*` tags publish `ghcr.io/<owner>/cuckoodb` with branch/tag/sha tags and `latest` on the default branch. The README displays CI, Coverage (Codecov), and Dependencies badges at the top.
 
 ## NOTES
 
-- The test suite currently passes 339 tests with zero failures or errors: `Tests run: 339, Failures: 0, Errors: 0, Skipped: 0`.
+- The test suite currently passes 419 engine tests and 90 server tests with zero failures or errors.
 - The benchmarking/tuple-counter infrastructure was introduced in commit `ef92ca1` ("feat: add query optimization benchmark suite with tuple counters"): `Operator` gained `protected long tupleCounter` with `getTupleCount()` / `resetTupleCount()`, and `QueryOptimizationBenchmarkTest` was added as one of the test files.
 - Budget enforcement reuses the tuple-counter slot: `countTuple()` (called per emitted tuple) increments `tupleCounter` and delegates to `QueryBudget.charge()` when a budget is attached. This means every operator in the tree counts — total-work semantics.
 - `SampleQueryRunner.java` provides an automated 20-query diff runner: it runs all queries in `samples/input/` against `samples/db/` and diffs each result against `samples/expected_output/`, reporting pass/fail. There is no need to diff manually.
 - A `.gitignore` exists at the repository root. It ignores `target/`, `*.iml`, `.DS_Store`, `*.class`, `.omo/`, and the engine-module test-resource output directories (`engine/src/test/resources/test_integration_output/`, `engine/src/test/resources/test_sample_output/`, `engine/src/test/resources/test_integration_queries/`).
 - Query output includes a header row (column names, plain commas) followed by data rows (plain comma-separated, LF). Output is RFC 4180 and is round-trippable as input to this engine.
-- EXPLAIN queries write the two-section plan text to the output file and do not execute the query. The plan root returned by `QueryPlanner.planQuery` for EXPLAIN is a no-op; `CuckooDB.run` short-circuits before operator iteration when `explainText` is non-null.
-- Hash join is the default for equi-joins (`Constants.useHashJoin = true`). Set it to `false` in tests (via `@BeforeEach`/`@AfterEach`) to exercise the nested-loop path. `HashJoinOperator.hasEquiConjunct(Expression)` is the planner's check for at least one cross-side column=column equality in the condition.
+- EXPLAIN queries write the two-section plan text to the output file and do not execute the query. `QueryPlanner.planQuery` still returns the optimized executable root for EXPLAIN; `CuckooDB.run` and `QueryService` short-circuit before operator iteration when `explainText` is non-null.
+- Hash join is the default for equi-joins (`QueryConfig.defaults().useHashJoin() == true`). Pass `new QueryConfig(true, false)` to planner tests/benchmarks to exercise the nested-loop path without mutating global state. `HashJoinOperator.hasEquiConjunct(Expression)` is the planner's check for at least one cross-side column=column equality in the condition.
 - The JMH 1.37 benchmark suite lives in `engine/src/test/java/com/github/jinba1/cuckoodb/bench/`. It is compiled as part of `test-compile` in CI but never executed there. `CachedOperator` (in `engine/src/test/java/.../operator/`) is a test utility operator that replays a fixed in-memory tuple list — used by `JoinAlgorithmBenchmark` to isolate join algorithm cost from CSV I/O. The `bench/` package classes are JMH benchmarks and do not contain JUnit tests.
+- REST endpoints: `POST /queries`, `GET /tables`, `GET /tables/{name}`, and opt-in `POST /tables/{name}` raw `text/csv` upload. Upload is disabled by default (`cuckoodb.upload.enabled=false`), enforces strict table names, streamed byte cap, table-count cap, malformed-CSV rejection, and `409` on name clash.
+- MCP endpoint: `/mcp` Streamable HTTP, SYNC tools, five `@McpTool` methods (`list_tables`, `describe_table`, `sample_rows`, `explain_query`, `query`) in `CuckooMcpTools`. Query-running tools route through `QueryService`.
+- Docker image layout: mount CSVs at `/cuckoodb/data`; `CUCKOODB_DATA_DIR=/cuckoodb`, `CUCKOODB_WORK_DIR=/cuckoodb/work`; runtime image uses a non-root `cuckoo` user.
diff --git a/BENCHMARK_RESULTS.md b/BENCHMARK_RESULTS.md
index 8570e2b..6106fa6 100644
--- a/BENCHMARK_RESULTS.md
+++ b/BENCHMARK_RESULTS.md
@@ -75,20 +75,20 @@ This document contains quantifiable performance results from the query optimizat
 
 ## Summary for Resume
 
-> "Reduced scan-level tuple processing by **23-57%** on multi-table join queries through rule-based query optimisations including selection pushdown and projection pushdown, validated by 225 JUnit tests covering operators, optimiser rules, and end-to-end query output."
+> "Reduced scan-level tuple processing by **23-57%** on multi-table join queries through rule-based query optimisations including selection pushdown and projection pushdown, validated by focused optimizer benchmarks and guarded by the current 419-test engine suite."
 
 ## Implementation Details
 
 - **Benchmark Class**: `QueryOptimizationBenchmarkTest.java`
-- **Tuple Counters**: Added to all 7 operator types (`Scan`, `Select`, `Project`, `Join`, `Sort`, `DuplicateElimination`, `Sum`)
-- **Optimization Toggle**: `Constants.useQueryOptimization` changed from `final` to mutable for benchmark comparison
-- **New Sample Queries**: query13.sql through query16.sql demonstrating optimization scenarios
-- **Test Count**: 225 total (190 original + 35 new benchmark tests)
+- **Tuple Counters**: All concrete operators count emitted tuples through `Operator.countTuple()` (`Scan`, `Select`, `Project`, `Join` / `HashJoin`, `Sort`, `Aggregate`, `DuplicateElimination`, `Limit`)
+- **Optimization Toggle**: Benchmarks pass per-query `QueryConfig` values to `QueryPlanner` instead of mutating global flags
+- **Sample Queries**: query13.sql through query16.sql demonstrate optimization scenarios; query20 demonstrates EXPLAIN
+- **Current Test Gate**: 419 engine tests, including 36 `QueryOptimizationBenchmarkTest` cases, plus the 20-query byte-identical sample gate
 
 ## Running Benchmarks
 
 ```bash
-./mvnw test -Dtest=QueryOptimizationBenchmarkTest
+./mvnw -pl engine test -Dtest=QueryOptimizationBenchmarkTest
 ```
 
 Or run all tests:
diff --git a/README.md b/README.md
index 62410ef..3741dcb 100644
--- a/README.md
+++ b/README.md
@@ -4,15 +4,15 @@
 ![Coverage](https://codecov.io/gh/JinBa1/java-query-engine/branch/main/graph/badge.svg)
 ![Dependencies](https://img.shields.io/badge/dependencies-up%20to%20date-brightgreen)
 
-**A self-hosted gateway that gives AI agents safe, read-only, budgeted SQL access to your CSV files — no database required.**
+**A self-hosted gateway that gives AI agents safe, budgeted SQL query access to your CSV files — no database required.**
 
-Everyone has CSVs — exports, dumps, logs — and AI agents increasingly need to query them. Embedding a database in every agent environment hands over raw file access; what you actually want is a *guarded window* onto the data: an endpoint that is read-only by construction, resource-budgeted, and auditable. cuckooDB is that gateway, built on a from-scratch query engine and exposed over both a **REST API** and the **Model Context Protocol (MCP)**, so an agent can discover tables, preview data, check a query's cost, and run SQL — without writing SQL blind or bypassing the guardrails.
+Everyone has CSVs — exports, dumps, logs — and AI agents increasingly need to query them. Embedding a database in every agent environment hands over raw file access; what you actually want is a *guarded window* onto the data: read-only query execution, resource budgets, and audit hooks around a narrow data surface. cuckooDB is that gateway, built on a from-scratch query engine and exposed over both a **REST API** and the **Model Context Protocol (MCP)**, so an agent can discover tables, preview data, check a query's cost, and run SQL — without writing SQL blind or bypassing the guardrails.
 
 ## Features
 
 | Capability | |
 |---|:--:|
-| Read-only SQL over CSV — `SELECT` / `WHERE` / `JOIN` / `GROUP BY` / `ORDER BY` / `LIMIT` / `DISTINCT` | ✅ |
+| Read-only SQL query execution over CSV — `SELECT` / `WHERE` / `JOIN` / `GROUP BY` / `ORDER BY` / `LIMIT` / `DISTINCT` | ✅ |
 | Aggregates — `COUNT` / `SUM` / `AVG` / `MIN` / `MAX` | ✅ |
 | Hash + nested-loop joins (planner auto-selects) | ✅ |
 | Typed columns (int / string), CSV headers | ✅ |
@@ -20,8 +20,9 @@ Everyone has CSVs — exports, dumps, logs — and AI agents increasingly need t
 | Tuple + time budgets, fail-closed | ✅ |
 | **REST API** + OpenAPI / Swagger | ✅ |
 | **MCP server** — five agent tools, Streamable-HTTP | ✅ |
+| Opt-in CSV upload endpoint, disabled by default | ✅ |
 | Runs as a container (published to GHCR) | ✅ |
-| Writes / transactions / indexes / persistence | ❌ read-only by design |
+| SQL writes / transactions / indexes / database persistence | ❌ query path is read-only by design |
 
 ## Quick start
 
@@ -58,7 +59,7 @@ The server exposes a Model Context Protocol endpoint at `http://localhost:8080/m
 | `explain_query` | preview a query's plan and cost before running it |
 | `query` | run a read-only `SELECT`, budget-bounded |
 
-Every tool routes through the same guarded execution path as the REST API, so agent traffic inherits the read-only guarantee, the tuple/time budget, and concurrency limits (with audit hooks in place) — there is no way to bypass them.
+Every query-running tool routes through the same guarded execution path as the REST query API, so agent traffic inherits the read-only query guarantee, the tuple/time budget, and concurrency limits (with audit hooks in place) — there is no way to bypass them.
 
 ## REST API
 
@@ -67,10 +68,13 @@ Every tool routes through the same guarded execution path as the REST API, so ag
 | `POST /queries` | plan + execute one read-only query → JSON columns/rows, or an `EXPLAIN` plan |
 | `GET /tables` | list table names |
 | `GET /tables/{name}` | a table's typed schema |
+| `POST /tables/{name}` | opt-in `text/csv` upload as a process-lifetime table; disabled by default |
 | `/swagger-ui.html` | interactive OpenAPI docs |
 
 Queries are **budget-bounded and fail-closed**: the server always attaches a budget, so an unbounded query is unreachable. A result that would exceed the tuple budget returns `429` (retry with a tighter `LIMIT`); one that exceeds the time budget returns `504`.
 
+Uploads are off unless a deployment sets `cuckoodb.upload.enabled=true`. When enabled, uploads enforce table-name validation, a streamed byte cap, a process-wide table cap, malformed-CSV rejection, and `409` on name clashes.
+
 ### EXPLAIN
 
 Any query can be planned without executing it — prefix `EXPLAIN` over REST, or call the `explain_query` tool. The plan is shown as written and after optimisation:
@@ -100,15 +104,15 @@ The optimiser pushes the `Select` below the join (selection pushdown) and projec
 SQL → JSqlParser → QueryPlanner → optimizer → operator tree → results
 ```
 
-The engine is a from-scratch Volcano/iterator executor — typed values, hash and nested-loop joins, selection pushdown, tuple/time budgets. The server wraps it behind a single `QueryService` choke point that applies the budget, a concurrency permit, and audit; **both** the REST controllers and the MCP tools go through it, so the guardrails can't be bypassed and apply uniformly. Engine internals — architecture, join algorithms, benchmarks, CLI — are in the **[engine README](engine/README.md)**.
+The engine is a from-scratch Volcano/iterator executor — typed values, hash and nested-loop joins, selection pushdown, tuple/time budgets. The server wraps query execution behind a single `QueryService` choke point that applies the budget, a concurrency permit, and audit; **both** the REST query controller and the MCP query tools go through it, so query guardrails can't be bypassed and apply uniformly. Engine internals — architecture, join algorithms, benchmarks, CLI — are in the **[engine README](engine/README.md)**.
 
 ## Build and test
 
 ```bash
-./mvnw clean verify    # builds + tests both modules: engine (419 tests) + server (90 tests)
+./mvnw clean package    # builds + tests/packages both modules: engine (419 tests) + server (90 tests)
 ```
 
-The 20 sample queries are a golden-output regression gate (see the engine README to run them). CI builds, tests, and publishes the container image to GHCR on every merge to `main`.
+The 20 sample queries are a golden-output regression gate (see the engine README to run them). The CI workflow builds, tests, and packages on every push/PR to `main`; the Docker workflow validates image builds on PRs and publishes `ghcr.io/jinba1/cuckoodb` on pushes to `main` and `v*` tags.
 
 ## Project structure
 
diff --git a/engine/README.md b/engine/README.md
index 9e93b01..6a120b5 100644
--- a/engine/README.md
+++ b/engine/README.md
@@ -11,7 +11,7 @@ SQL → JSqlParser → QueryPlanner → QueryPlanOptimizer → Operator Tree →
 | Component | Role |
 |-----------|------|
 | `QueryPlanner` | Parses SQL and builds the operator pipeline |
-| `QueryPlanOptimizer` | Selection pushdown, trivial operator removal |
+| `QueryPlanOptimizer` | Selection pushdown, projection pushdown, trivial operator removal, consecutive-select merging |
 | `DBCatalog` | Schema and table metadata (singleton) |
 | `Value` | Typed tuple values (sealed interface: `IntValue`, `StringValue`) |
 | `ExpressionEvaluator` | Evaluates WHERE/HAVING conditions per tuple |
@@ -107,7 +107,7 @@ The planner selects between two join algorithms automatically.
 
 **Hash** (`HashJoinOperator extends JoinOperator`): the inner (build) side is drained once into a `HashMap` keyed by the equality conjuncts; the outer (probe) side streams through once. After a lookup, the full original condition is re-evaluated on every candidate, so residual non-equality conjuncts (e.g. `A.x = B.x AND A.y > 3`) work. Output order is identical to nested-loop (outer-major, inner order preserved per key bucket). Shown as `HashJoin[<condition>]`.
 
-**Auto-selection:** hash join is used when `Constants.useHashJoin` is `true` (default) **and** the condition has at least one column-to-column equality conjunct. Cross products and pure non-equi joins always use nested-loop. Set `Constants.useHashJoin = false` to force nested-loop everywhere.
+**Auto-selection:** hash join is used when the per-query `QueryConfig.useHashJoin` flag is `true` (the production default) **and** the condition has at least one column-to-column equality conjunct. Cross products and pure non-equi joins always use nested-loop. Tests and benchmarks can pass `new QueryConfig(true, false)` to `QueryPlanner` to force nested-loop everywhere without mutating global state.
 
 ### Benchmarks