Old-vs-new mask comparison on real data (tiles + exposures)

[cailmdaley]

The gate for retiring ShapePipe's internal masking (#847 keeps the old path config-selectable until this lands). Part of epic #845.

Goal. Run both mask paths on the same real data — the Nibi P3 run (#808: 10 tiles, ~1960 single-CCD exposures) is the natural testbed — and understand, visually and quantitatively, what each mask piece does and where it comes from, before the old path is removed.

Deliverables:

• Side-by-side visual overlays of old (WeightWatcher/GSC star halos+spikes, NGC/Messier regions) vs new (healsparse-rasterized maskforce) masks on real image pixels, for several tiles and exposure CCDs. The question each panel answers: do the masks land on the artifacts, and which system catches what?
• Component-provenance audit. A table of every mask ingredient and where it enters each path: saturation / bleeding / bad columns (instrument flag files from CFHT preprocessing — summed in at the exposure level via USE_EXT_FLAG), cosmic rays (Stephen's masking), bright-star halos + spikes (old: GSC-2.3 + WeightWatcher; new: PhotoPipe bright-star masks), extended galaxies (old: NGC/Messier region files; new: 2MASS manual masks), MaxiMask. Per Axel (Slack 2026-07-16), the exposure level is where pixel masks matter — the audit should make explicit what the exposure-level flag image contains under each path.
• Quantitative comparison: masked-area fractions, IMAFLAGS_ISO population changes, PSF star-selection differences (counts + size–magnitude locus), and PSF model quality on the affected CCDs.

Non-goal: deciding halo/spike sizes — that validation is the tangential-shear test (separate sub-issue), though the two share infrastructure.

— Fable, on behalf of Cail

[cailmdaley]

Old vs new masks on real data — 10 tiles + 9 exposure CCDs from the P3 run (#808).

All panels: CFIS r-band pixels (zscale + asinh stretch); percentages = masked fraction per component. These figures compare only what gets masked in ShapePipe: the NEW panels show the external healsparse masks as #847 rasterizes them into the pipeline — in current production those are applied post-production, at the catalogue level.

• Tile figures (OLD | NEW). OLD = ShapePipe's internal tile pipeline_flag from this run: GSC-2.3 star halos + diffraction spikes rasterized by WeightWatcher (Messier/NGC masking enabled but empty on these tiles; no instrument content at tile level). NEW = the external healsparse reason maps (nside 131072 ≈ 1.6″) queried at each pixel's sky position: faint star halo (bit 1), bright star halo (2), bright stars (4), manual galaxy mask (8), MaxiMask (1024).
• Exposure figures (INSTRUMENT | OLD | NEW). One full 2112×4644 MegaCam CCD each. INSTRUMENT = the CFHT/Elixir flag file (bad pixels, saturation, bleed trails); in production it is summed into both old and new pipeline_flag, so it is shown separately — the OLD/NEW panels isolate what each masking system itself adds. OLD caveat: bit 2 is ambiguous (halo=2 collides with instrument saturated=2 under summing).

Tiles

Exposure CCDs

— Fable, on behalf of Cail

[aguinot]

If I may make a few comments:

• Using MaxiMask on the coadds is hard to justify. MaxiMask was not trained on coadds but on single exposures.
• More generally, I don't see a justification to use MaxiMask at all. It was trained on old and outdated CFHT images and the outcome is redundant in some case, or just bad.
• We don't need pixel level masks on the coadds.
• The bright stars/objects can be constructed at the single exposure level and combined to a full sky mask at posteriori.

A mask like this is the best way to inject systematics in the data

[martinkilbinger]

Something went very wrong with the ShapePipe tile masks, none of the star masks and actual star coordinates match.

[cailmdaley]

@martinkilbinger i don't think that's true, there are stars under the old ShapePipe tile masks. it's just that not all stars are masked.

[AstroMike]

Ludo identified one case where the external (galaxy mask) did not look good.

Can you @cailmdaley / Claude maybe draw 20 tiles or CCDs randomly from P3 that have at least one external mask present, and overlay just the mask with the greyscale so we can see how well or poorly it fits? (It looks better if its just an outline not a filled-in coloured polygon).

I will let you turn this into a separate subtask on github - I claim ignorance.

[AstroMike]

@martinkilbinger i don't think that's true, there are stars under the old ShapePipe tile masks. it's just that not all stars are masked.

I believe the GSC (guide star catalogue) intentionally rejected close pairs of stars (these not good for HST to guide on), so its far from complete.

[cailmdaley]

More generally, I don't see a justification to use MaxiMask at all.

@aguinot what should we do for the streaks (satellites?) that appear occasionally then? the instrument mask doesn't seem to capture them:

[cailmdaley]

Can you @cailmdaley / Claude maybe draw 20 tiles or CCDs randomly from P3 that have at least one external mask present, and overlay just the mask with the greyscale so we can see how well or poorly it fits? (It looks better if its just an outline not a filled-in coloured polygon).

sure, check out #855 @AstroMike

[aguinot]

Satellite strickes could be a reason to use MaxiMMask but just for this purpose only. Also, maybe the mask would need to be rework a bit before use to enlarge and "smooth" it. The dented pattern is probably not great in terms of systematic.