fturader — Python API (deep reference)

The README covers installation, the one-call segment_ftu quick start, and the CLI. This document is the deep Python-API reference: how to choose a loader and inference call, the low-level FTURader model, the multiplex loaders, pseudo-H&E recipes, and the full organ × modality × entry support matrix.

Runnable end-to-end examples live in the walkthrough notebooks — walkthrough_python.ipynb (full Python-API tour: loaders, recipe comparison, tile_grid, bundle I/O) and its shell companion walkthrough_cli.ipynb. See notebooks/README.md for which dataset each one needs.

Choosing an entry point

For most users segment_ftu (above) is enough. When you need finer control there are two layers of choice: (A) how you hand us the image (loader) and (B) which inference call runs (single-shot vs. tiled). The CLI hides both; the Python API exposes them.

A. Loader decision tree (Python API)

Pick by the shape of your input, not the modality:

Is the input already ONE assembled image (a file or a numpy array)?
│
├─ YES  ── SingleImageDataset.from_image(...)
│           │
│           ├─ Fluorescence panel? choose how the eosin channel is built:
│           │     recipe="auto"        → auto-pick channels per panel  (default)
│           │     recipe=["Cytokeratin","Vimentin"]  → YOUR channels, taken verbatim
│           │     recipe="ck"          → single Cytokeratin (epithelial organs only)
│           │
│           └─ Is it a big MULTI-Z image stitched from acquisition tiles
│              (one image, but each region focused independently)?
│                 └─ add tile_grid=(rows, cols)  → per-block best-focus z
│
└─ NO, I have a GRID of separate tile files + their geometry
        │
        ├─ Generic tiles (CODEX / MIBI / autofluor; tileMap or explicit positions)
        │     └─ TiledDataset.from_tiles(...)        ← first-class
        │
        └─ A *verified* HuBMAP/Akoya reg{NNN}_X##_Y##.tif processed dir
              └─ CodexDataset(...)   ⚠ DEPRECATED — delegates to from_tiles

Input you have	Entry point	Notes
One assembled image (file or numpy array)	`SingleImageDataset.from_image`	The general front door. OME-TIFF reads names + pixel size from its header.
One big multi-z image (per-region focus)	`from_image(..., tile_grid=(r, c))`	Block-wise per-block best-focus z. No-op for single-z images. Still one image, not a grid.
A grid of separate tile files + geometry	`TiledDataset.from_tiles`	Modality-agnostic via `reader_factory`; geometry via `tilemap=` or `positions=`.
Verified HuBMAP/Akoya `reg{NNN}_X##_Y##.tif` dir	`CodexDataset` (deprecated)	Thin wrapper, delegates to the `from_tiles` stitch engine. Only the exact verified layout is guaranteed. New code should use `from_tiles`.

load_dataset(path) is a best-effort convenience factory on top of these — it sniffs the layouts we routinely handle (processed CODEX dir / single TIFF) and raises pointing you to from_image + manual metadata when it cannot recognize the input. It is not the main entry point.

All loaders converge on one synthesis core (global percentile normalization + Beer-Lambert), so colours and detections are consistent regardless of entry point. Always call ds.describe() first to confirm how the input was understood (and which channels the recipe selected).

B. `infer_image` vs `infer_wsi`

Call	When	What it does
`FTURader.infer_image`	A single image that fits the model canvas (≈ ≤ 1024² after scaling)	One forward pass, no tiling.
`FTURader.infer_wsi`	Large images (multiplex mosaics ~85 MP scale to ~9 MP ≫ 1024²)	Large-image tiling engine — `sv.InferenceSlicer` cuts overlapping `1024²` tiles, runs each, then merges across tiles with `NON_MAX_MERGE` + `IOS`.

infer_wsi is not the everyday user-facing entry point — it is the large-image tiling engine that makes big mosaics tractable. Both segment_ftu(...) and infer_multiplex(ds, model, recipe=...) route pseudo-H&E mosaics through it automatically. Multiplex mosaics are structurally large (~85 MP raw → ~9 MP after pixel-size scaling), so they require tiling. Use infer_image directly only for genuinely small crops.

CLI vs Python-API boundary

The CLI (fturader) takes a single --input (one image file, or a processed-CODEX directory routed through load_dataset). The multichannel / multi-tile loaders are Python-API only: TiledDataset.from_tiles with custom geometry, tile_grid block-focus, in-memory numpy arrays, and custom reader_factory are not exposed as CLI flags. For those, use the Python API. The CLI always runs infer_wsi internally regardless of image size.

Low-level model — `FTURader.infer_image` / `infer_wsi`

from pathlib import Path
from fturader.pipeline import FTURader

weights = Path.home() / ".deepcell/models/ftusam_v0.1"
model = FTURader(
    rf_detr_checkpoint=weights / "RFDETRLarge_resolution_1024_kidney.pth",
    sam2_checkpoint=weights / "sam2.1_hiera_b+_epoch_300.pth",
    min_mask_area=5000,           # kidney default
)

# Small single image → one forward pass
detections = model.infer_image(
    "path/to/small_crop.tif",
    pixel_size_um=0.4,
    score_threshold=0.25,         # kidney default
)

# Large image → tiled engine (sv.InferenceSlicer, 50% overlap, NON_MAX_MERGE + IOS)
import numpy as np
from PIL import Image
large = np.array(Image.open("large.tif").convert("RGB"))
detections = model.infer_wsi(
    large,
    pixel_size_um=0.4,
    score_threshold=0.25,
    overlap_ratio=0.5,
)

# Results
# detections.xyxy        — (N, 4) bounding boxes in original image coordinates
# detections.mask        — CompactMask (RLE crops) by default; dense (N, H, W) bool
#                          only when called with compact_masks=False
# detections.confidence  — (N,) combined score = √(rf_score × sam_score)

Multiplex loaders (Python API)

(1) One assembled image — from_image (format-aware TIFF / OME-TIFF / numpy):

from fturader.multiplex import SingleImageDataset, infer_multiplex

ds = SingleImageDataset.from_image(
    array_or_tiff,                                  # file path or numpy array
    channel_names=["DAPI", "PanCK", "Vimentin"],    # list, or path to channelnames.txt
    pixel_size_um=0.377,                            # µm/px (warned if omitted)
    axes="CYX",        # arrays only; inferred for 2D/3D, required for >=4D (ZCYX/TZCYX)
    he=None,           # None=auto · True=already-H&E passthrough · False=force synthesis
    recipe="auto",     # default pseudo-H&E recipe stored on the dataset
)
print(ds.describe())   # sanity-check how the input was understood BEFORE inference

# Prefer to pick the eosin channels yourself? Pass a marker list (taken verbatim):
ds = SingleImageDataset.from_image(
    array_or_tiff,
    channel_names=["DAPI", "PanCK", "Vimentin"],
    pixel_size_um=0.377,
    recipe=["Cytokeratin", "Vimentin"],            # your channels; nuclear still auto-classified
)

# OME-TIFF carrying its own channel names + pixel size needs no extra metadata:
ds = SingleImageDataset.from_image("stitched.ome.tif")

# Big multi-z image stitched from acquisition tiles? Process block-wise so each block
# gets its OWN best-focus z (a single global z would defocus the other regions):
ds = SingleImageDataset.from_image("big_stack.ome.tif", tile_grid=(rows, cols))

detections = infer_multiplex(ds, model, recipe="auto")   # pseudo-H&E -> infer_wsi

(2) Separate tiles — from_tiles (modality-agnostic; declare geometry with a tilemap or explicit positions):

from fturader.multiplex import TiledDataset

ds = TiledDataset.from_tiles(
    tiles="/path/to/processed",                     # tile dir (tileMap mode) …
    channel_names="/path/to/channelnames.txt",
    pixel_size_um=0.377,
    tilemap="/path/to/tileMap.txt", region=1,       # … or positions=[(x,y), …] for a file list
    tile_pattern="reg004_X{x:02d}_Y{y:02d}.tif",    # absorbs HBM288's reg/RegionNumber mismatch
)
detections = infer_multiplex(ds, model, recipe="auto")

Deprecated: CodexDataset / the load_dataset factory remain for the verified HuBMAP reg{NNN}_X##_Y##.tif layout and delegate to the from_tiles engine; new code should use from_image / from_tiles. See the "Loading boundary" section of cli.md. These multiplex loaders are Python API only (the CLI takes a single --input; tile_grid and custom from_tiles geometry are not flags).

Pseudo-H&E recipes (`recipe=`)

The recipe decides which markers become the eosin (pink, structure/cytoplasm) source; the nuclear (hematoxylin) channel is always a single auto-classified channel. Three ways to specify it, plus deprecated legacy aliases:

`recipe=`	Eosin channel(s)	Use for
`"auto"` (default)	Gated auto-selection (see below)	Any panel — the most general choice
`"auto:mean"` / `"auto:max"`	Same as `auto`, with the eosin reduce made explicit (default is `mean`)	When you want to control how multiple matched channels combine
`["Cytokeratin", "Vimentin"]` (a list)	Your channels, taken verbatim	When you know which markers carry your structure of interest
`"ck"`	`["Cytokeratin"]` (single, sharpest crypts/tubules/alveoli)	Epithelial organs that carry Cytokeratin only — a legacy good choice, not a general default
`PseudoHEConfig(...)` / `PseudoHEConfig.auto(names, eosin_reduce="max")`	Full control	Custom reduce / Beer-Lambert
`"v1"` / `"v2"` (deprecated)	`v1`→`["Cytokeratin"]` (= `ck`, byte-for-byte); `v2`→`["Cytokeratin","CollIV","Vimentin"]`	Backward-compatible aliases; emit a `DeprecationWarning`

What auto does (gated by whether the panel carries a cytokeratin-family marker):

Panel HAS Cytokeratin (e.g. large intestine) → name-based classification keeps all matched structural markers and reduces them (mean by default). Because that multi-channel mean blends the sharp epithelial signal with diffuse ECM/mesenchymal channels, auto can detect slightly fewer FTUs than the single-Cytokeratin ck, whose tight crypt boundary gives the tightest masks. This is an explainable channel difference, not non-determinism.
Panel LACKS Cytokeratin (e.g. spleen — lymphoid/stromal, no epithelium) → auto falls through to a pixel-evidence SNR selector that drops near-noise channels and picks the single strongest structural channel (Vimentin on spleen).

"ck" (and the deprecated v1/v2) fix the eosin list to literally include "Cytokeratin", so they hard-KeyError on a panel that has no such channel — by design they are epithelial-only, not a universal default. Full detail: multiplex_pseudo_he.md.

Support matrix (organ × modality × entry)

Validated on the HuBMAP datasets we hold. ✓ = works end-to-end; ✗ = not supported; untested = not yet validated (planned feature support, not a confirmed failure).

Organ	Modality	Loader entry	Recipe	Status	Notes
Kidney	H&E brightfield	`from_image(he=True)` / `--he true`	— (passthrough)	✓	No synthesis; RGB fed directly.
Lung	H&E brightfield	`from_image(he=True)` / `--he true`	— (passthrough)	✓ crop only	Full slide is 1471 MP; `NON_MAX_MERGE` mask densify OOMs → run on a crop.
Large intestine	CODEX fluorescence	`from_tiles` (tiles) / `from_image` (OME)	`auto` / manual list / `ck`	✓	Has Cytokeratin → `auto` keeps the name-based multi-channel mean; `ck` (single Cytokeratin) gives the tightest masks; a manual list lets you pick channels.
Spleen	CODEX fluorescence	`from_image(axes="CYX")` (stack mosaics)	`auto` → SNR → Vimentin	✓	No Cytokeratin → use `auto` (falls through to the SNR selector). `ck` / `v1` / `v2` `KeyError`.
Kidney	PAS brightfield	`from_image(he=True)`	—	untested	Not validated on PAS — the H&E-trained weights are not validated on PAS. An exploratory native-resolution check returned no detections (confounded by stain-domain shift + glomerulus scale); treat as planned feature support (may or may not transfer), not a confirmed failure.

HuBMAP source datasets (portal links verified 2026-06-22):

HBM id	Organ / type	HuBMAP portal
HBM269.FMTJ.275	Kidney H&E (Visium histology component)	https://portal.hubmapconsortium.org/browse/dataset/3a7480e53dc874ce2199cb73b31b4bdd
HBM988.MNCM.878	Lung H&E (image pyramid)	https://portal.hubmapconsortium.org/browse/dataset/fffaec22fb69f78a104731ad6854cefa
HBM547.GNRV.624	Lung H&E (histology; same slide as HBM988)	https://portal.hubmapconsortium.org/browse/dataset/7311e2312471ddf9986f13818c441588
HBM573.TTLG.748	Large intestine CODEX	https://portal.hubmapconsortium.org/browse/dataset/8eeca1a98041348bb14fd652b766661e
HBM288.BBKK.828	Large intestine CODEX (2nd sample)	https://portal.hubmapconsortium.org/browse/dataset/7500e7f02589f4b4e55641f09723c3de
HBM685.PCCJ.427	Large intestine CODEX (Cytokit + SPRM output)	https://portal.hubmapconsortium.org/browse/dataset/c8fa615fb884c7b51d08c0cf35debfee
HBM543.RSRV.265	Spleen CODEX	https://portal.hubmapconsortium.org/browse/dataset/3d14dcc3d7c3e0cd339c9366e34b37c7

The HuBMAP portal links above are the source for each dataset's specs; the from_image / from_tiles examples earlier in this document show how each is loaded.

Navigation Menu

Uh oh!

fturader — Python API (deep reference)

Choosing an entry point

A. Loader decision tree (Python API)

B. `infer_image` vs `infer_wsi`

CLI vs Python-API boundary

Low-level model — `FTURader.infer_image` / `infer_wsi`

Multiplex loaders (Python API)

Pseudo-H&E recipes (`recipe=`)

Support matrix (organ × modality × entry)

Uh oh!

FilesExpand file tree

python_api.md

Latest commit

History

python_api.md

File metadata and controls

fturader — Python API (deep reference)

Choosing an entry point

A. Loader decision tree (Python API)

B. infer_image vs infer_wsi

CLI vs Python-API boundary

Low-level model — FTURader.infer_image / infer_wsi

Multiplex loaders (Python API)

Pseudo-H&E recipes (recipe=)

Support matrix (organ × modality × entry)

B. `infer_image` vs `infer_wsi`

Low-level model — `FTURader.infer_image` / `infer_wsi`

Pseudo-H&E recipes (`recipe=`)