Skip to content

Feature/gaussian point model - #22

Open
sridhs21 wants to merge 11 commits into
mainfrom
feature/gaussian-point-model
Open

Feature/gaussian point model#22
sridhs21 wants to merge 11 commits into
mainfrom
feature/gaussian-point-model

Conversation

@sridhs21

@sridhs21 sridhs21 commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Reworks the X-point detector to train on Gaussian heatmap targets and evaluates it at the point level instead of per pixel. The hyperparameter-tuning work this branch was originally stacked on has since merged in #21, so this PR now contains only the point-level and Gaussian changes.

Gaussian heatmap model (XPointMLTest.py)

  • --targetType gaussian renders each ground-truth X-point as a 2D Gaussian peak (--gaussianSigma, default 3) instead of the dilated 9×9 binary mask; binary is still the default
  • --lossFunction heatmap_focal: CornerNet/CenterNet-style penalty-reduced focal loss, with --heatmapAlpha / --heatmapBeta

Point-level evaluation

  • point_metrics.py: extract_peaks (vectorized connected-component NMS, one peak per region above the threshold) and match_points (greedy one-to-one matching within a radius, highest confidence first), plus aggregation over frames
  • run_hessian_and_build_cache.py: also writes a ground-truth {N}_xpts.csv (row, col, class) for every cached frame
  • backfill_xpts_csv.py: writes those CSVs for caches built earlier, with --output-dir for caches that aren't writable
  • predict_points.py: runs a checkpoint over cached frames and writes per-frame predicted {N}_xpts.csv files with confidences
  • score_point_predictions.py: matches predictions to ground truth and reports F1 / precision / recall / TP / FP / FN per dataset

Tuning

  • optuna_tuner_gaussian.py: Optuna tuner for the Gaussian model that maximizes point-level F1 on the validation frames rather than minimizing validation loss
  • The peak-extraction threshold is swept inside each trial's F1 evaluation (--threshold-grid) instead of being fixed at 0.3, so every configuration is scored at its own best operating point; the winning threshold is recorded per trial

Data and configuration

  • Second, independent PKPM run (PKPMv2) supported in cache building, prediction, and scoring
  • Dataset paths in predict_points.py and score_point_predictions.py are built from RC_EXTRACT_DIR / RC_CACHE_BASE / RC_PKPM_V0_ROOT (the same convention as run_hessian_and_build_cache.py), with the shared DeltaAI locations as defaults, so launchers can point them at another environment without code changes
  • test_xpoint_transfer.py: THRESHOLD environment variable for the pixel-level transfer evaluation

README

  • Documents the new target and loss options, the Gaussian tuner and threshold sweep, PKPMv2 and the CSV outputs, the point-level evaluation pipeline, and the path environment variables

Swaroop Sridhar and others added 8 commits June 17, 2026 18:23
…w, col, class columns for X/Omax/Omin points), with a backfill pass that writes CSVs for already-cached frames without re-running the classifier.
…d truth at the point level. match_points takes two coordinate arrays and a matching radius, builds a grid index on the gt side, and does greedy 1-to-1 nearest-neighbor matching to figure out tp/fp/fn (with optional confidence ordering so high-confidence preds claim matches first). evaluate_point_predictions runs that across many frames and gives back a global summary plus per-frame metrics. load_xpts_csv and load_xpts_csvs_for_frames are tiny helpers for reading the CSV format the cache builder emits.
…oint-list CSVs onto disk. backfill_xpts_csv.py walks a cache directory and writes per-frame {N}_xpts.csv files from existing {N}_xpts.npy plus optsMax/optsMin files, with an --output-dir flag for cases where the source cache isn't writable (like the PKPM cache, which is owned by another user). predict_points.py loads a trained checkpoint, runs inference over the cached frames of a dataset, post-processes the predicted heatmap with connected-component NMS to extract one peak per blob above threshold, then writes the predicted points as per-frame CSVs with row/col/confidence columns. Between them they cover both sides of the comparison: ground-truth points from the deterministic classifier, and predicted points from a neural net.
…st scoring together. Takes a predictions root directory (subdirs per dataset), loads the predicted xpts CSVs with their confidences plus the matching ground-truth CSVs from the per-dataset cache mirror, and runs evaluate_point_predictions from point_metrics on each dataset. Prints a per-dataset F1/precision/recall/TP/FP/FN table for the given checkpoint, and optionally writes the full per-frame breakdown to JSON via --json-out. Takes --radius and --datasets flags so you can tweak the matching radius or pick which subset of datasets to score without editing the script. Together with predict_points.py this closes the loop from checkpoint through NMS extraction through point-level comparison.
…tmapLoss, --gaussianSigma/--targetType/--heatmap* args) onto the point-list branch so a single working tree can both train at varying sigma and score with the point-list metric, for the sigma sweep experiment.
…PKPM run through cache build, prediction, and scoring
@cwsmith

cwsmith commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Thank you. With the change to the model, are there changes needed in the top level README (e.g., new command line args, etc.)?

@sridhs21
sridhs21 marked this pull request as draft August 21, 2026 17:17
@sridhs21
sridhs21 force-pushed the feature/gaussian-point-model branch from d5979f1 to 074899a Compare September 4, 2026 17:01
@sridhs21
sridhs21 force-pushed the feature/gaussian-point-model branch from 074899a to deb3d38 Compare September 4, 2026 17:11
@sridhs21
sridhs21 marked this pull request as ready for review September 11, 2026 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants