From 21899e0a375629d1bd1c1b19045499eae0a05a6a Mon Sep 17 00:00:00 2001
From: Halleluyah Oludele <halleluyaholudele@gmail.com>
Date: Sun, 2 Aug 2026 23:06:57 +0100
Subject: [PATCH] docs: add bench/ and docs/evaluations/, with the ICDAR 2013
 result

Benchmark harnesses and measurement findings had no home. The ICDAR 2013
harness was living in a temp directory and every result from this work
existed only in the issue tracker, where it records that something was
decided rather than what was measured.

bench/ is a SEPARATE Go module. Nested modules are excluded from the
parent ./... , so the library build ignores it and a consumer of pdftable
never fetches its dependencies. The first version of this commit did not
do that and go list ./... showed bench/icdar2013 inside the library
module -- exactly the leak the split exists to prevent.

Datasets are not committed. run.py fetches ICDAR 2013 into a cache
directory, builds the extractor as its own module against the working
tree, and scores pdftable alongside pdfplumber -- "is 0.36 good?" is
unanswerable without a baseline.

docs/evaluations/ holds dated reports. Each states the commit it ran
against, the external oracle used, the metric precisely, what the number
is NOT comparable to, and what remains untested. That last part is the
point: a benchmark number without its scope reads as a general claim, and
0.36 here is end-to-end detection plus structure, not the structure-only
task papers report 0.85-0.95 on.

Reports are append-only, so a regression shows up as a diff between two
dated files rather than disappearing into a rewrite.

The library source stays in the repository root. That is the import path
for a single-package Go library, and moving it would break every
consumer; Go also deliberately keeps _test.go files beside the code they
cover. The README now documents the layout so the root is legible rather
than merely conventional.
---
 README.md                                     |  38 +++-
 bench/README.md                               |  33 ++++
 bench/go.mod                                  |  11 ++
 bench/icdar2013/README.md                     |  57 ++++++
 bench/icdar2013/diag.py                       |  86 ++++++++
 bench/icdar2013/extract.go                    |  70 +++++++
 bench/icdar2013/run.py                        | 112 +++++++++++
 bench/icdar2013/score.py                      | 184 ++++++++++++++++++
 docs/README.md                                |  39 ++++
 ...6-08-02-font-metrics-and-table-fidelity.md | 124 ++++++++++++
 .../2026-08-02-icdar2013-table-structure.md   |  90 +++++++++
 11 files changed, 838 insertions(+), 6 deletions(-)
 create mode 100644 bench/README.md
 create mode 100644 bench/go.mod
 create mode 100644 bench/icdar2013/README.md
 create mode 100644 bench/icdar2013/diag.py
 create mode 100644 bench/icdar2013/extract.go
 create mode 100644 bench/icdar2013/run.py
 create mode 100644 bench/icdar2013/score.py
 create mode 100644 docs/README.md
 create mode 100644 docs/evaluations/2026-08-02-font-metrics-and-table-fidelity.md
 create mode 100644 docs/evaluations/2026-08-02-icdar2013-table-structure.md

diff --git a/README.md b/README.md
index 4e49166..f1ef9b3 100644
--- a/README.md
+++ b/README.md
@@ -603,12 +603,38 @@ stdlib-only.
   and a `pdftable` CLI mirroring pdfplumber's surface.
 - `v0.4.x` — bundle the standard-14 AFM metrics so word bboxes (and
   therefore cell text) match pdfplumber on standard fonts. **Done**: the
-  Adobe Core 14 metrics ship, and Symbol/ZapfDingbats decode with their
-  own built-in encodings. The golden position envelope is still asserted
-  at 15pt pending a re-measure, so the "within 1 point" claim is not yet
-  evidenced.
-- `v0.5.x` — performance pass: parser benchmarking against
-  pdfminer.six and pdfplumber on a representative document corpus.
+  Adobe Core 14 metrics ship, Symbol/ZapfDingbats decode with their own
+  built-in encodings, and measured position drift against pdfplumber is
+  **0.0000pt on both axes** — the golden envelope is now asserted at
+  0.01pt. See
+  [the evaluation](docs/evaluations/2026-08-02-font-metrics-and-table-fidelity.md).
+- `v0.5.x` — **table detection**. The ICDAR 2013 benchmark puts
+  end-to-end F1 at 0.362, level with pdfplumber's 0.370, and the
+  diagnostic is unambiguous: precision 0.865, recall 0.229. What we
+  extract is right; we miss three quarters of the tables, because the
+  `lines` strategy needs *intersecting* rulings and a horizontally-ruled
+  table produces none. See
+  [the evaluation](docs/evaluations/2026-08-02-icdar2013-table-structure.md).
+- `v0.6.x` — performance pass: parser speed against pdfminer.six and
+  pdfplumber on a representative corpus.
+
+## Repository layout
+
+`pdftable` is a single Go package, so its source lives in the repository
+root — that is the import path, and Go keeps `_test.go` files next to the
+code they cover.
+
+| | |
+| --- | --- |
+| `*.go` | the library |
+| `internal/pdf/` | content-stream interpreter, fonts, encodings |
+| `internal/layout/` | edge/intersection/cell geometry |
+| `cmd/pdftable/` | CLI |
+| `bench/` | accuracy benchmarks against public datasets — a **separate Go module**, so it never adds a dependency here |
+| `docs/evaluations/` | dated measurement reports, with their caveats |
+| `scripts/` | fixture and golden-file generators |
+| `testdata/golden/` | fixtures compared against pdfplumber |
+| `testdata/fonts/` | fixtures where pdfplumber is wrong, asserted directly |
 
 ## License
 
diff --git a/bench/README.md b/bench/README.md
new file mode 100644
index 0000000..d79e8d4
--- /dev/null
+++ b/bench/README.md
@@ -0,0 +1,33 @@
+# Benchmarks
+
+Accuracy benchmarks against public datasets with published ground truth.
+
+These are **not** `go test` benchmarks (those live next to the code as
+`*_bench_test.go` and measure speed). These measure **correctness** against
+an external reference, and they are deliberately outside the library's Go
+module so their dependencies never reach a consumer of `pdftable`.
+
+Datasets are **not committed** — they are large and separately licensed.
+Each harness downloads what it needs into a scratch directory.
+
+| Benchmark | Dataset | Measures | Report |
+| --- | --- | --- | --- |
+| [`icdar2013/`](icdar2013/) | ICDAR 2013 Table Competition (125 PDFs) | table detection + structure | [2026-08-02](../docs/evaluations/2026-08-02-icdar2013-table-structure.md) |
+
+## Why the numbers live in `docs/evaluations/`
+
+A benchmark result is only meaningful with its date, the code version it
+ran against, and the caveats on the metric. A bare number in a README rots
+the moment either side changes. Every run gets a dated report; the table
+above links to the most recent.
+
+## Adding a benchmark
+
+1. `bench/<name>/` with its own `go.mod` if it needs Go.
+2. A `run.py` that fetches the dataset and prints a result table.
+3. A dated report in `docs/evaluations/`, and a row in the table above.
+
+State the metric precisely, and say what it is **not** comparable to.
+Published table-extraction scores in particular are usually
+structure-only — the system is handed the table location — and are not
+comparable to an end-to-end number.
diff --git a/bench/go.mod b/bench/go.mod
new file mode 100644
index 0000000..f592b91
--- /dev/null
+++ b/bench/go.mod
@@ -0,0 +1,11 @@
+// Separate module on purpose: the benchmark harness must never add a
+// dependency to the library it measures. Nested modules are excluded from
+// the parent's ./... , so go build ./... at the repo root ignores this
+// entirely and a consumer of pdftable never fetches it.
+module github.com/hallelx2/pdftable/bench
+
+go 1.25.0
+
+require github.com/hallelx2/pdftable v0.0.0
+
+replace github.com/hallelx2/pdftable => ../
diff --git a/bench/icdar2013/README.md b/bench/icdar2013/README.md
new file mode 100644
index 0000000..fb7531c
--- /dev/null
+++ b/bench/icdar2013/README.md
@@ -0,0 +1,57 @@
+# ICDAR 2013 Table Competition benchmark
+
+Measures **table detection + structure recognition** end to end against
+125 born-digital PDFs from EU and US government sources, with per-cell
+ground truth.
+
+```sh
+pip install pdfplumber
+python bench/icdar2013/run.py            # full run, ~10 min
+python bench/icdar2013/run.py --limit 5  # quick check while iterating
+python bench/icdar2013/run.py --diag     # detection vs structure breakdown
+```
+
+The dataset (~12 MB) downloads to `~/.cache/pdftable-bench` (override with
+`PDFTABLE_BENCH_DIR`). It is not committed.
+
+## Metric: adjacency relations
+
+From Göbel et al., the metric the competition itself used. For every
+non-empty cell, take its nearest non-empty neighbour to the right and
+below; each pair is a relation `(text_a, text_b, direction)`. Score the
+detected multiset against ground truth.
+
+Cell-by-cell grid comparison would be the obvious alternative and is
+worse: two tools can grid the same table differently — one emits a spacer
+column, the other does not — and still convey identical structure. A grid
+diff calls that a failure. Adjacency asks the question a reader actually
+cares about: *is this value next to that label?* It is unforgiving about
+genuinely wrong structure and forgiving about harmless disagreements.
+
+Documents are scored whole, with relations unioned across pages and
+tables. That sidesteps matching detected tables to ground-truth tables, a
+step which would need its own arbitrary thresholds and would make the
+number depend on them.
+
+## Reading the result
+
+**This is an end-to-end number and is NOT comparable to published ICDAR
+2013 scores of 0.85–0.95.** Those evaluate structure recognition with the
+table region *given*; this has to find the table first. Compare runs of
+this harness to each other, and to the pdfplumber column, not to papers.
+
+`--diag` splits the two failure modes, which need opposite work:
+
+- **the table was never detected** — a detection problem
+- **it was detected but gridded wrongly** — a structure problem
+
+Latest results: [`docs/evaluations/`](../../docs/evaluations/).
+
+## Files
+
+| | |
+| --- | --- |
+| `run.py` | fetches the dataset, builds the extractor, runs everything |
+| `extract.go` | dumps every table pdftable finds as JSON; built as its own module so the benchmark adds no dependency to the library |
+| `score.py` | parses ground-truth XML, computes precision/recall/F1 |
+| `diag.py` | separates detection failures from structure failures |
diff --git a/bench/icdar2013/diag.py b/bench/icdar2013/diag.py
new file mode 100644
index 0000000..c8d5dc8
--- /dev/null
+++ b/bench/icdar2013/diag.py
@@ -0,0 +1,86 @@
+"""Separate the two ways table extraction can fail on ICDAR 2013.
+
+Low recall has two very different causes and the headline F1 cannot tell
+them apart:
+
+  (a) the table was never detected at all — a detection problem
+  (b) it was detected but gridded wrongly — a structure problem
+
+(a) is fixed by better "is this a table?" logic; (b) by better cell
+geometry. They need opposite work, so it is worth knowing which one we
+have before touching anything.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import subprocess
+import sys
+from collections import Counter
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from score import gt_relations, norm, relations_from_grid, score  # noqa: E402
+
+
+def main() -> int:
+    root, exe = sys.argv[1], sys.argv[2]
+    pairs = []
+    for dirpath, _, files in os.walk(root):
+        for f in sorted(files):
+            if f.endswith("-str.xml"):
+                pdf = os.path.join(dirpath, f.replace("-str.xml", ".pdf"))
+                if os.path.exists(pdf):
+                    pairs.append((pdf, os.path.join(dirpath, f)))
+    pairs.sort()
+
+    zero_detect = 0
+    gt_regions_total = 0
+    detected_tables_total = 0
+    pages_total = 0
+    per_doc = []
+
+    for pdf, xml in pairs:
+        gt = gt_relations(xml)
+        out = subprocess.run(
+            [exe, "-strategy", "lines", pdf], capture_output=True, timeout=120
+        ).stdout
+        tables = json.loads(out or b"[]")
+        detected_tables_total += len(tables)
+        rels: Counter = Counter()
+        for t in tables:
+            rels += relations_from_grid([[norm(c) for c in r] for r in t["rows"]])
+        c, nd, ng = score(gt, rels)
+        if nd == 0:
+            zero_detect += 1
+        gt_regions_total += 1
+        per_doc.append((os.path.basename(pdf), ng, nd, c, len(tables)))
+
+    print(f"documents                     : {len(pairs)}")
+    print(f"documents where we found NO table: {zero_detect}"
+          f"  ({100*zero_detect/len(pairs):.0f}%)")
+    print(f"tables detected (total)       : {detected_tables_total}")
+    print()
+
+    # Restrict to documents where we DID detect something: if structure is
+    # good, precision AND recall should both be high on this subset.
+    tot = [0, 0, 0]
+    for _, ng, nd, c, _ in per_doc:
+        if nd:
+            tot[0] += c
+            tot[1] += nd
+            tot[2] += ng
+    p = tot[0] / tot[1] if tot[1] else 0
+    r = tot[0] / tot[2] if tot[2] else 0
+    f = 2 * p * r / (p + r) if (p + r) else 0
+    print("--- documents where a table WAS detected ---")
+    print(f"  precision {p:.3f}   recall {r:.3f}   F1 {f:.3f}")
+    print()
+    print("worst 8 documents by missed relations:")
+    for name, ng, nd, c, nt in sorted(per_doc, key=lambda x: x[1] - x[3])[-8:]:
+        print(f"  {name:<18} gt={ng:<6} detected={nd:<6} correct={c:<6} tables={nt}")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/bench/icdar2013/extract.go b/bench/icdar2013/extract.go
new file mode 100644
index 0000000..ba010bd
--- /dev/null
+++ b/bench/icdar2013/extract.go
@@ -0,0 +1,70 @@
+// Emit every table pdftable finds in a PDF, as JSON, for benchmarking.
+package main
+
+import (
+	"encoding/json"
+	"flag"
+	"fmt"
+	"os"
+
+	"github.com/hallelx2/pdftable"
+)
+
+type tableOut struct {
+	Page int        `json:"page"`
+	Rows [][]string `json:"rows"`
+}
+
+func main() {
+	strategy := flag.String("strategy", "lines", "lines | text | fallback")
+	merge := flag.Bool("merge", false, "TableSettings.MergeSplitTokens")
+	flag.Parse()
+
+	doc, err := pdftable.OpenFile(flag.Arg(0))
+	if err != nil {
+		// A file we cannot open scores zero rather than aborting the run;
+		// the harness needs a result for every document.
+		json.NewEncoder(os.Stdout).Encode([]tableOut{})
+		fmt.Fprintln(os.Stderr, "open:", err)
+		return
+	}
+	defer doc.Close()
+
+	mk := func(v, h pdftable.TableStrategy) pdftable.TableSettings {
+		s := pdftable.DefaultTableSettings()
+		s.VerticalStrategy, s.HorizontalStrategy = v, h
+		s.MergeSplitTokens = *merge
+		return s
+	}
+	lines := mk(pdftable.StrategyLines, pdftable.StrategyLines)
+	text := mk(pdftable.StrategyText, pdftable.StrategyText)
+
+	var attempts []pdftable.TableSettings
+	switch *strategy {
+	case "lines":
+		attempts = []pdftable.TableSettings{lines}
+	case "text":
+		attempts = []pdftable.TableSettings{text}
+	default: // fallback: ruled cells first, whitespace alignment if none
+		attempts = []pdftable.TableSettings{lines, text}
+	}
+
+	out := []tableOut{}
+	for i := 1; i <= doc.NumPages(); i++ {
+		p, err := doc.Page(i)
+		if err != nil {
+			continue
+		}
+		for _, s := range attempts {
+			tables, err := p.ExtractTables(s)
+			if err != nil || len(tables) == 0 {
+				continue
+			}
+			for _, t := range tables {
+				out = append(out, tableOut{Page: i, Rows: t.Rows})
+			}
+			break
+		}
+	}
+	json.NewEncoder(os.Stdout).Encode(out)
+}
diff --git a/bench/icdar2013/run.py b/bench/icdar2013/run.py
new file mode 100644
index 0000000..7b578f7
--- /dev/null
+++ b/bench/icdar2013/run.py
@@ -0,0 +1,112 @@
+"""Run the ICDAR 2013 table benchmark end to end.
+
+    pip install pdfplumber
+    python bench/icdar2013/run.py
+
+Downloads the dataset (~12 MB) into a scratch directory, builds the Go
+extractor, scores pdftable against the ground truth, and prints a result
+table. pdfplumber is scored alongside as a reference point — the question
+"is 0.36 good?" is unanswerable without a baseline, and pdfplumber is the
+implementation pdftable is a port of.
+
+Pass --limit N to score only the first N documents while iterating.
+"""
+
+from __future__ import annotations
+
+import argparse
+import os
+import shutil
+import subprocess
+import sys
+import tarfile
+import urllib.request
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+REPO = os.path.abspath(os.path.join(HERE, "..", ".."))
+
+# Brandon Smock's corrected edition of the ICDAR 2013 competition set.
+# The original release had known ground-truth errors; this is the version
+# current table-structure papers evaluate against.
+URL = (
+    "https://huggingface.co/datasets/bsmock/"
+    "ICDAR-2013-Table-Competition-Corrected/resolve/main/"
+    "ICDAR-2013-Table-Competition-Corrected.tar.gz"
+)
+DIRNAME = "ICDAR-2013-Table-Competition-Corrected"
+
+
+def scratch() -> str:
+    d = os.environ.get("PDFTABLE_BENCH_DIR") or os.path.join(
+        os.path.expanduser("~"), ".cache", "pdftable-bench"
+    )
+    os.makedirs(d, exist_ok=True)
+    return d
+
+
+def fetch(dest: str) -> str:
+    root = os.path.join(dest, DIRNAME)
+    if os.path.isdir(root):
+        print(f"dataset already present: {root}")
+        return root
+    tgz = os.path.join(dest, "icdar2013.tar.gz")
+    if not os.path.exists(tgz):
+        print(f"downloading {URL}")
+        urllib.request.urlretrieve(URL, tgz)
+    print("extracting...")
+    with tarfile.open(tgz) as t:
+        t.extractall(dest)
+    return root
+
+
+def build_extractor(dest: str) -> str:
+    exe = os.path.join(dest, "bench-extract.exe" if os.name == "nt" else "bench-extract")
+    mod = os.path.join(dest, "extractor")
+    os.makedirs(mod, exist_ok=True)
+    shutil.copy(os.path.join(HERE, "extract.go"), os.path.join(mod, "main.go"))
+
+    # Its own module, pointed at the working tree: the benchmark must never
+    # add a dependency to the library it is measuring.
+    gomod = os.path.join(mod, "go.mod")
+    if not os.path.exists(gomod):
+        subprocess.run(["go", "mod", "init", "benchextract"], cwd=mod, check=True,
+                       capture_output=True)
+    subprocess.run(
+        ["go", "mod", "edit", "-replace",
+         f"github.com/hallelx2/pdftable={REPO.replace(os.sep, '/')}"],
+        cwd=mod, check=True, capture_output=True)
+    subprocess.run(
+        ["go", "mod", "edit", "-require", "github.com/hallelx2/pdftable@v0.0.0"],
+        cwd=mod, check=True, capture_output=True)
+    subprocess.run(["go", "mod", "tidy"], cwd=mod, check=True, capture_output=True)
+    subprocess.run(["go", "build", "-o", exe, "."], cwd=mod, check=True)
+    print(f"built {exe}")
+    return exe
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--limit", type=int, default=0,
+                    help="score only the first N documents")
+    ap.add_argument("--diag", action="store_true",
+                    help="also run the detection-vs-structure diagnostic")
+    args = ap.parse_args()
+
+    dest = scratch()
+    root = fetch(dest)
+    exe = build_extractor(dest)
+
+    cmd = [sys.executable, os.path.join(HERE, "score.py"), root, exe]
+    if args.limit:
+        cmd.append(str(args.limit))
+    subprocess.run(cmd, check=True)
+
+    if args.diag:
+        print()
+        subprocess.run(
+            [sys.executable, os.path.join(HERE, "diag.py"), root, exe], check=True)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/bench/icdar2013/score.py b/bench/icdar2013/score.py
new file mode 100644
index 0000000..1e3fdb8
--- /dev/null
+++ b/bench/icdar2013/score.py
@@ -0,0 +1,184 @@
+"""Benchmark table extraction on the ICDAR 2013 table competition set.
+
+Metric: adjacency relations (Goebel et al., ICDAR 2013) — the standard for
+this dataset. For every non-empty cell, take its nearest non-empty
+neighbour to the right and below; each such pair is a relation
+(text_a, text_b, direction). Score the detected relation multiset against
+ground truth.
+
+Why this metric rather than comparing grids cell-by-cell: two tools can
+grid a table differently — one emits a spacer column, another does not —
+and still convey exactly the same structure. Adjacency asks the question
+that actually matters for a reader: "is this value next to that label?"
+It is unforgiving about genuinely wrong structure and forgiving about
+harmless disagreements over gridding.
+
+Documents are scored as a whole (relations unioned across pages/tables),
+which sidesteps having to match detected tables to ground-truth tables —
+a matching step that would itself need arbitrary thresholds.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import re
+import subprocess
+import sys
+import xml.etree.ElementTree as ET
+from collections import Counter
+
+WS = re.compile(r"\s+")
+
+
+def norm(s: str | None) -> str:
+    return WS.sub(" ", (s or "")).strip()
+
+
+def relations_from_grid(grid: list[list[str]]) -> Counter:
+    """Adjacency relations from a dense row/col grid."""
+    rels: Counter = Counter()
+    nrows = len(grid)
+    for r in range(nrows):
+        ncols = len(grid[r])
+        for c in range(ncols):
+            a = grid[r][c]
+            if not a:
+                continue
+            # nearest non-empty to the right, skipping blanks
+            for c2 in range(c + 1, ncols):
+                if grid[r][c2]:
+                    rels[(a, grid[r][c2], "H")] += 1
+                    break
+            # nearest non-empty below
+            for r2 in range(r + 1, nrows):
+                row2 = grid[r2]
+                if c < len(row2) and row2[c]:
+                    rels[(a, row2[c], "V")] += 1
+                    break
+    return rels
+
+
+def gt_relations(xml_path: str) -> Counter:
+    """Ground truth: build a dense grid per region from start/end row+col."""
+    rels: Counter = Counter()
+    root = ET.parse(xml_path).getroot()
+    for region in root.iter("region"):
+        cells = []
+        for cell in region.findall("cell"):
+            sr = int(cell.get("start-row", 0))
+            sc = int(cell.get("start-col", 0))
+            er = int(cell.get("end-row", sr))
+            ec = int(cell.get("end-col", sc))
+            content = cell.find("content")
+            text = norm(content.text if content is not None else "")
+            cells.append((sr, sc, er, ec, text))
+        if not cells:
+            continue
+        nrows = max(c[2] for c in cells) + 1
+        ncols = max(c[3] for c in cells) + 1
+        grid = [["" for _ in range(ncols)] for _ in range(nrows)]
+        for sr, sc, er, ec, text in cells:
+            # A spanning cell occupies every position it covers, which is
+            # what makes adjacency work across merged headers.
+            for r in range(sr, er + 1):
+                for c in range(sc, ec + 1):
+                    if r < nrows and c < ncols:
+                        grid[r][c] = text
+        rels += relations_from_grid(grid)
+    return rels
+
+
+def score(gt: Counter, got: Counter) -> tuple[int, int, int]:
+    correct = sum((gt & got).values())
+    return correct, sum(got.values()), sum(gt.values())
+
+
+def prf(correct: int, ndet: int, ngt: int) -> tuple[float, float, float]:
+    p = correct / ndet if ndet else 0.0
+    r = correct / ngt if ngt else 0.0
+    f = 2 * p * r / (p + r) if (p + r) else 0.0
+    return p, r, f
+
+
+def run_pdftable(exe: str, pdf: str, strategy: str, merge: bool) -> Counter:
+    cmd = [exe, "-strategy", strategy]
+    if merge:
+        cmd.append("-merge")
+    cmd.append(pdf)
+    try:
+        out = subprocess.run(cmd, capture_output=True, timeout=120).stdout
+        tables = json.loads(out or b"[]")
+    except Exception:
+        return Counter()
+    rels: Counter = Counter()
+    for t in tables:
+        rels += relations_from_grid([[norm(c) for c in row] for row in t["rows"]])
+    return rels
+
+
+def run_pdfplumber(pdf: str, strategy: str) -> Counter:
+    import pdfplumber
+
+    settings = {"vertical_strategy": strategy, "horizontal_strategy": strategy}
+    rels: Counter = Counter()
+    try:
+        with pdfplumber.open(pdf) as doc:
+            for page in doc.pages:
+                for tbl in page.extract_tables(settings):
+                    rels += relations_from_grid(
+                        [[norm(c) for c in row] for row in tbl]
+                    )
+    except Exception:
+        return Counter()
+    return rels
+
+
+def main() -> int:
+    root = sys.argv[1]
+    exe = sys.argv[2]
+    limit = int(sys.argv[3]) if len(sys.argv) > 3 else 0
+
+    pairs = []
+    for dirpath, _, files in os.walk(root):
+        for f in sorted(files):
+            if not f.endswith("-str.xml"):
+                continue
+            pdf = os.path.join(dirpath, f.replace("-str.xml", ".pdf"))
+            if os.path.exists(pdf):
+                pairs.append((pdf, os.path.join(dirpath, f)))
+    pairs.sort()
+    if limit:
+        pairs = pairs[:limit]
+    print(f"scoring {len(pairs)} documents\n", flush=True)
+
+    systems = {
+        "pdftable (lines)": lambda p: run_pdftable(exe, p, "lines", False),
+        "pdftable (fallback)": lambda p: run_pdftable(exe, p, "fallback", False),
+        "pdftable (fallback+merge)": lambda p: run_pdftable(exe, p, "fallback", True),
+        "pdfplumber (lines)": lambda p: run_pdfplumber(p, "lines"),
+        "pdfplumber (text)": lambda p: run_pdfplumber(p, "text"),
+    }
+    totals = {k: [0, 0, 0] for k in systems}
+
+    for i, (pdf, xml) in enumerate(pairs, 1):
+        gt = gt_relations(xml)
+        for name, fn in systems.items():
+            c, nd, ng = score(gt, fn(pdf))
+            totals[name][0] += c
+            totals[name][1] += nd
+            totals[name][2] += ng
+        if i % 10 == 0:
+            print(f"  ...{i}/{len(pairs)}", flush=True)
+
+    print(f"\n{'system':<28} {'precision':>10} {'recall':>10} {'F1':>10}")
+    print("-" * 62)
+    for name in systems:
+        p, r, f = prf(*totals[name])
+        print(f"{name:<28} {p:>10.3f} {r:>10.3f} {f:>10.3f}")
+    print(f"\nground-truth relations: {totals['pdftable (lines)'][2]}")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/docs/README.md b/docs/README.md
new file mode 100644
index 0000000..41c9e96
--- /dev/null
+++ b/docs/README.md
@@ -0,0 +1,39 @@
+# Documentation
+
+| | |
+| --- | --- |
+| [`evaluations/`](evaluations/) | dated accuracy findings — what was measured, against what, and what it does not prove |
+
+User-facing documentation lives in the root [`README.md`](../README.md);
+release history in [`CHANGELOG.md`](../CHANGELOG.md).
+
+## Evaluations
+
+| date | subject | headline |
+| --- | --- | --- |
+| [2026-08-02](evaluations/2026-08-02-icdar2013-table-structure.md) | ICDAR 2013 table detection + structure | F1 0.362 end-to-end; the bottleneck is **detection**, not cell accuracy |
+| [2026-08-02](evaluations/2026-08-02-font-metrics-and-table-fidelity.md) | font metrics and table fidelity | position drift 11.99pt → **0.0000pt**; negative-sign loss 19% → **0%** |
+
+### Conventions
+
+Each report states:
+
+- the **date** and the **commit** it ran against,
+- the **dataset** and the **external reference** used as an oracle,
+- the **metric**, precisely, and **what the number is not comparable to**,
+- **what remains untested**.
+
+That last section is the point. A benchmark result without its scope reads
+as a general claim, and every number here is narrower than it looks —
+these results say nothing about scanned documents or CJK text, for
+instance.
+
+Reports are append-only. Re-running a benchmark adds a new dated file
+rather than editing an old one, so a regression is visible as a diff
+between two reports instead of vanishing into a rewrite.
+
+### Why findings live here and not only in the issue tracker
+
+An issue records that something was *decided*. A report records what was
+*measured*, so the next person can tell whether a number still holds
+without re-deriving it — and can see the caveats that made it honest.
diff --git a/docs/evaluations/2026-08-02-font-metrics-and-table-fidelity.md b/docs/evaluations/2026-08-02-font-metrics-and-table-fidelity.md
new file mode 100644
index 0000000..f2ede0c
--- /dev/null
+++ b/docs/evaluations/2026-08-02-font-metrics-and-table-fidelity.md
@@ -0,0 +1,124 @@
+# Font metrics and table fidelity — audit and fixes
+
+**Date:** 2026-08-02
+**Commits:** `325e628` … `0ca65ca` (PRs #9, #13–#19)
+**References:** rendered pages via `pdftoppm`, poppler `pdftotext -layout`, pdfplumber 0.11.9
+**Real-world corpus:** 3M Company 2018 Form 10-K (FinanceBench), pages 56–60
+
+## Headline
+
+| | before | after |
+|---|---|---|
+| Horizontal (X) drift vs pdfplumber | 11.99pt max / 4.79pt mean | **0.0000pt** |
+| Vertical (Y) drift vs pdfplumber | 4.968pt @24pt, 2.484pt @12pt | **0.0000pt** |
+| Golden position tolerance | 15pt | **0.01pt, both axes** |
+| Negative signs preserved (3M, 5 statements) | 83/103 (81%) | **103/103** |
+| Font coverage in the test corpus | 1 font, 26 words | **12 fonts, 3 sizes, 170 words** |
+
+## Defects found and fixed
+
+### 1. Flat 500 width for the standard 14 fonts
+
+PDF 1.7 §9.6.2.2 lets the 14 standard fonts omit `/Widths`; a consumer is
+expected to already know their metrics. pdftable did not, and fell through
+to a flat 500/1000 guess for every glyph — `i` (222) and `m` (833) got
+identical widths, and the error accumulated along each line.
+
+Because the `text` and `lines_strict` strategies infer column boundaries
+from word positions, this was upstream of table structure, not merely
+cosmetic.
+
+### 2. Symbol and ZapfDingbats decoded as Latin
+
+Both fonts carry their own built-in encoding and neither declares
+`/Encoding`, so `readFont` handed them StandardEncoding. Symbol code
+`0x61` decoded as `a` rather than `alpha` — a *text*-correctness failure,
+not a metrics one. Any document using Symbol for Greek extracted nonsense.
+
+pdfplumber 0.11.9 still exhibits this; pdftable now does not.
+
+### 3. Glyph boxes sat `descent × size` too high
+
+Two independent causes, the second of which would have survived fixing the
+first:
+
+- The same exemption that omits `/Widths` also omits `/FontDescriptor`,
+  so `Ascent`/`Descent` were 0 and the glyph box collapsed to
+  `[baseline, baseline+size]`.
+- Descent was scaled by `0.001` but **not** by the font size, so even
+  fonts that *did* supply a descriptor were short by a factor of the font
+  size — 12× at 12pt.
+
+Measured error was exactly `0.207 × size`, matching Helvetica's −207/1000
+descender. Rows are derived from word Y extents, so this could merge or
+split table rows.
+
+### 4. 19% of negative numbers lost their sign
+
+Cell assignment gives a glyph to whichever cell contains its centre. That
+is correct at an *interior* boundary — it settles which of two candidates
+owns a straddling glyph. At the table's *outer* edge there is no competing
+cell, so the rule stopped disambiguating and started deleting.
+
+On 3M page 58 the `)` of `(16,048)` had its centre at x=537.879 while the
+last column ended at x=537.871 — **a shortfall of 0.008pt**, about a
+nine-thousandth of an inch. Accounting notation for −16,048 was read back
+as +16,048.
+
+Across the five financial statements this flipped the sign of **20 of 103
+negatives (19%)** while leaving every magnitude correct — which is what
+made it dangerous. A missing value is detectable downstream; a plausible
+wrong sign is not.
+
+**pdfplumber has the same defect.** On the same row it returns `16,048`
+without the closing paren. pdftable is now strictly better than its
+reference here.
+
+### 5. Small type over-merged into single runs
+
+`DefaultWordOpts()` did not honour explicit space glyphs, so word
+boundaries were inferred from inter-glyph gaps alone. At 8pt a space is
+`278/1000 × 8 = 2.22pt`, under the 3pt `XTolerance`, so a whole line
+collapsed into one word. Body type in real documents is routinely 8–9pt.
+
+pdfplumber ends a word *at* a whitespace glyph before any gap test runs.
+Matching that reproduces its output word-for-word and coordinate-for-coordinate.
+
+## A finding that was wrong
+
+`ExtractText` returns `"Consolidated Balance Shee t"` for 3M page 58. This
+was filed as a defect. **It is not** — that gap is in the source document.
+The rendered page shows it, and poppler extracts it identically. 3M's
+filing agent typeset it that way, which is common in SEC EDGAR documents.
+
+The error was methodological: the original comparison was pdftable's table
+output against pdftable's *own* text output. Internal agreement proves the
+two code paths disagree, not which one is right. Correcting it required an
+external reference — a rendered page and a second extractor.
+
+**Any fidelity claim needs an oracle the project did not write.**
+
+## Verification method
+
+Findings were confirmed against three independent sources before being
+accepted:
+
+1. the page rendered to PNG (`pdftoppm -r 150`) and read directly,
+2. poppler `pdftotext -layout`,
+3. pdfplumber 0.11.9.
+
+The citation geometry (`BBox.Viewport` / `BBox.Normalized`) was verified by
+projecting real cell bboxes onto the rendered page and confirming the
+highlights land on the intended rows.
+
+## What remains untested
+
+- **scanned PDFs** — no text layer at all; the geometric approach cannot
+  apply and an OCR path is required
+- **CID/Type0 fonts** — CJK and modern subsetted fonts
+- **embedded subset fonts** with their own `/Widths` — the common
+  production case
+- **rotated and multi-column layouts**
+
+And separately, table *structure* accuracy, measured in the
+[ICDAR 2013 evaluation](2026-08-02-icdar2013-table-structure.md).
diff --git a/docs/evaluations/2026-08-02-icdar2013-table-structure.md b/docs/evaluations/2026-08-02-icdar2013-table-structure.md
new file mode 100644
index 0000000..13ecccd
--- /dev/null
+++ b/docs/evaluations/2026-08-02-icdar2013-table-structure.md
@@ -0,0 +1,90 @@
+# ICDAR 2013 — table detection and structure
+
+**Date:** 2026-08-02
+**Commit:** `0ca65ca` (after the font-metric and table-fidelity work of HAL-480/481/510/520/548/511)
+**Harness:** [`bench/icdar2013`](../../bench/icdar2013/) — `python bench/icdar2013/run.py`
+**Dataset:** ICDAR 2013 Table Competition, Smock's corrected edition — 125 born-digital PDFs, 39,524 ground-truth adjacency relations
+**Reference:** pdfplumber 0.11.9
+
+## Result
+
+| system | precision | recall | F1 |
+| --- | --- | --- | --- |
+| pdftable (`lines`) | 0.865 | 0.229 | **0.362** |
+| pdfplumber (`lines`) | 0.868 | 0.235 | **0.370** |
+| pdftable (fallback `lines`→`text`) | 0.223 | 0.557 | 0.318 |
+| pdftable (fallback + `MergeSplitTokens`) | 0.471 | 0.275 | 0.347 |
+| pdfplumber (`text`) | 0.167 | 0.679 | 0.267 |
+
+## Two findings
+
+### 1. Parity holds at the table level
+
+0.362 vs 0.370, with near-identical precision (0.865 vs 0.868). Everything
+before this validated *text* fidelity — word positions, glyph widths,
+signs. This is the first measurement of *table* behaviour, and pdftable
+tracks its reference implementation there too.
+
+### 2. The bottleneck is detection, not structure
+
+```
+documents                          : 125
+documents where NO table was found : 28  (22%)
+tables detected (total)            : 306
+
+restricted to documents where a table WAS detected:
+  precision 0.865   recall 0.409   F1 0.556
+```
+
+Precision 0.865 says **what we extract is right**. Recall 0.229 says **we
+miss three quarters of it**. Those are very different problems and they
+need opposite work.
+
+The cause is structural. The `lines` strategy builds cells from
+**intersecting** rulings. A table ruled only horizontally — booktabs
+style, ubiquitous in government and academic documents — yields no
+intersections and is invisible. Confirmed on `us-018.pdf`: 46 ruling lines
+on page 1, zero rectangles, zero tables detected.
+
+The `text` strategy sees those tables (recall 0.56–0.68) but fabricates
+tables on prose pages (precision 0.17–0.22). Neither library has a usable
+"is this actually a table?" decision, which is why the naive fallback
+scores *worse* overall than `lines` alone.
+
+## Caveat on the number
+
+**0.36 is not comparable to the 0.85–0.95 reported in table-structure
+papers.** Those benchmarks hand the system the table region and score only
+the gridding. This runs the harder end-to-end task. The 0.556 figure on
+documents where detection succeeded is closer to a structure-only
+comparison, and still contains intra-document detection misses.
+
+The honest one-line summary: **cell-level extraction is strong, table
+detection is weak.**
+
+## What this implies
+
+Detection — *where is the table* — is a vision problem, and the thing
+CV/VLM models are built for. Content fidelity — the values, signs and
+coordinates — is a text-layer problem, where the deterministic parser is
+provably better (it preserves accounting minus signs that pdfplumber
+drops; see the [font-metrics evaluation](2026-08-02-font-metrics-and-table-fidelity.md)).
+
+That argues for splitting the work by which half is weak, rather than
+replacing either:
+
+- **layout/VLM model → table region and row/column structure** (attacks
+  the 0.229 recall)
+- **pdftable text layer → cell contents and coordinates** (keeps exact
+  values and citation geometry, which a generative model cannot provide)
+
+Two cheaper wins should be tried first, since that is where the points
+are:
+
+1. Support horizontally-ruled-only tables — derive column edges from word
+   alignment *within* the rule band instead of requiring vertical rulings.
+2. Give the `text` strategy a confidence signal, so a fallback can reject
+   prose instead of emitting a table for every page.
+
+Tracked as HAL-568. Re-run this harness after each and record a new dated
+report here.