Skip to content

Support input masks in panorama reconstruction - #4755

Open
trueoneplusone wants to merge 1 commit into
colmap:mainfrom
trueoneplusone:feat/panorama-input-masks
Open

trueoneplusone wants to merge 1 commit into
colmap:mainfrom
trueoneplusone:feat/panorama-input-masks

Conversation

@trueoneplusone

Copy link
Copy Markdown
Contributor

Panorama reconstruction currently creates masks to avoid duplicate features across virtual cameras, but cannot incorporate user masks in the original panoramas. This adds per-image and shared input masks to the Python API and panorama_sfm.py, addressing #4346.

  • Add input_mask_path and input_camera_mask_path to PanoramaReconstructionOptions and the example CLI. Per-image paths follow ImageReader's appended .png convention and replacement-extension fallback.
  • In perspective modes, read masks with COLMAP's grayscale reader, validate dimensions, project them with nearest-neighbor sampling, and intersect them with the virtual-camera ownership masks. Longitude wraps at the panorama seam; latitude is clamped at the poles. A shared mask is cached without modification by per-image masks.
  • In spherical mode, pass the original masks to ImageReaderOptions.
  • Document the workflow and its limits: masks filter COLMAP keypoints, RGB images and GPS metadata are preserved, and downstream Gaussian-splatting tools must independently consume masks for their training losses.

Validation

30 tests pass across python/pycolmap/panorama_test.py, python/pycolmap/panorama_mask_test.py, and python/examples/panorama_sfm_test.py. The new API/CLI tests failed before implementation. Coverage includes both perspective layouts, exact polar pixels, longitude seams, zero/nonzero mask semantics, mask intersections, nested filenames and fallback precedence, missing/corrupt/wrong-size masks, threaded per-image processing, RGB/GPS preservation, and CLI forwarding.

The integration tests run actual rendering, native CPU SIFT and database IO in all three reconstruction modes. They check that the retained keypoints and descriptors equal the exact expected subset of an unmasked extraction. Matching and mapping are bypassed; no full reconstruction accuracy claim is made.

A separate deterministic check used three synthetic textured panoramas (two 1024×512, one 2048×1024) with masks crossing the seam and excluding the lower hemisphere:

Mode Baseline features Excluded Retained
Perspective overlapping 64,186 40,006 24,180
Perspective non-overlapping 42,530 26,545 15,985
Spherical 43,999 27,431 16,568

Across all cases, zero excluded features survived, no expected valid features were lost, and retained descriptors were identical.

Ruff 0.15.20 check/format, mypy on the four changed Python files, and git diff --check pass. These checks used the Python code from this checkout (4.3.0.dev0) with the native PyCOLMAP 4.2.0 Windows wheel, Python 3.13.5, OpenCV 4.12.0, and NumPy 2.5.3. The C++ extension was not rebuilt and the full C++/reconstruction suites were not run.

AI assistance: OpenAI Codex researched the issue, implemented the changes, and ran the tests and measurements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant