Quickmaps for IMAP-Lo#3179
Conversation
tmplummer
left a comment
tmplummer
left a comment
There was a problem hiding this comment.
Jon was right that there would be some opinions on the attitude and reinventing the wheel for much of this code. So, I will state them here and act as if this is a normal PR.
- Much of this code duplicates thing we already have in place.
- Why does Lo need to extract it's own mean spin-axis?
- Why does it need to write it's own tools to do vector transforms instead of using the industry standard, SPICE?
- Much of this implementation is at odds with the "one mission" philosophy of the IMAP mission.
We really can't merge this without test coverage. The throw it into production and then come back and write meaningful tests logic is flawed. Writing functions with limited scope and writing concise unit tests for them can save time improve code reliability tremendously.
|
@tmplummer - is there a way for us to test the pipeline with this code? (on dev or something)? The products need to be generated so that they can be validated in parallel by the team with the refactorings you're requesting here (which are reasonable). EDIT: Or if there's a way to tell what dependency json the pipeline would have used for certain invocations, we could generate a bunch of |
If you wanted to spot check one or two pointings, it would be feasible in dev. We can build a docker image for a certain branch and push that to dev. Then you would need to sync all the upstream dependencies required to dev as well. We don't have a good way to do a bulk sync AFAIK. For running locally, you would have to generate the dependency json files locally and run the If neither of those options are sufficient, I think that we could merge past the codecov check if you create issue(s) for all the PR comments such that they don't get lost and forgotten. @tech3371 @bryan-harter what do you think? |
|
@tmplummer - no worries then. I just wanted to make sure there wasn't a shortcut we could take. Let me incorporate your suggestions tomorrow, which will take comparable time to what you guys will have to go through. This will also help me get familiar with more of the existing code. |
|
An update - the |
|
As a note, it is possible to deploy into dev from an unmerged branch using our existing deploy to dev GitHub action. You can specify a GitHub URL to deploy from. So it is simple enough to test in dev. |
9f3afed to
74b6aea
Compare
… offset) angle, instead of a np.roll after the fact
…ions Generalize the Lo L1C pointing-direction geometry so it can be reused by other products (e.g. the l1c quickmap) without duplicating the DPS az/el -> sky transform. - Add compute_pointing_directions(epoch, pivot_angle, spin_angles=None, off_angles=None, to_frame=IMAP_HAE): the pure-geometry core returning the raw (n_spin, n_off, 2) lon/lat array for an arbitrary spin/off-angle grid and destination frame. - set_pointing_directions now delegates to it with the PSET defaults (3600x40 grid, IMAP_DPS -> IMAP_HAE) and keeps its DataArray output, so the PSET path is unchanged. - Add unit tests for the defaults, custom grid/frame, and the delegation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
# Conflicts: # imap_processing/tests/lo/test_lo_l1c.py
…ections Replace the quaternion-derived mean-spin-axis + analytic create_ra_dec pointing with a SPICE IMAP_DPS -> ECLIPJ2000 transform at the good-time midpoint, using the reusable compute_pointing_directions helper. - Drop the quaternion_datasets parameter; lo_l1c_quickmap now takes only sci_dependencies. Update the CLI quickmap branch accordingly (no longer loads spacecraft quaternion files). - Remove create_ra_dec and the assemble_quaternions / Rotation / cartesian_to_spherical / spherical_to_cartesian imports it needed. - Keep nep_bin_angles solely for the cosalpha (RAM projection) factor. - Integration tests: drop the synthetic quaternion dataset and mock frame_transform_az_el with an identity DPS->ecliptic transform, since a CK/attitude kernel is not available in the test environment. Caveat documented in-code: routing through IMAP_DPS assumes the L1B histogram spin-bin index is the DPS spin angle (mounting handled by the frame kernel). This has not been validated against a known-good sky map; if the histogram is in raw spacecraft spin phase the map would be rotated by the spin-phase offset. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…et convention The L1B histogram spin bins (spin_bin_6 <- L1A azimuth_6 = onboard bin indices 0..59) are hardware spin-phase bins referenced to the spacecraft spin pulse, not the instrument/DPS spin angle. The L1B star-sensor product converts the same onboard angle to the instrument frame by adding the spacecraft->instrument spin-phase offset (spin_angle = start_angle_offset + sample_centers); the L1B DE product needs no offset only because its spin_bin comes from a SPICE HAE->IMAP_DPS transform of the actual look direction. Accordingly, feed compute_pointing_directions the offset-applied DPS azimuth (sc_bin_centers + offset) rather than the raw bin centers, matching the legacy quickmap (which rolled the histogram by the same offset). Update the in-code note from "unvalidated assumption" to the resolved convention, leaving only a sky-map cross-check of the DPS azimuth zero-point as a follow-up. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…validation frame_transform_az_el squeezes singleton leading dims, so compute_pointing_directions returned (n_spin, 2) rather than (n_spin, 1, 2) when called with a single boresight off-angle (as the quickmap does). Reshape back to the documented (n_spin, n_off, 2) contract so az_el[:, 0, :] indexing works with real SPICE (the identity mock in the tests did not reproduce the squeeze). Also upgrade the in-code note: with the real IMAP_DPS/attitude CK furnished (tools/verify_lo_quickmap_equivalence.py), the SPICE pointing reproduces the legacy quickmap counts cell-for-cell (<= 2-count differences in 3 of 7 ESA levels, from sub-cell spin-axis rounding), confirming the spin-phase offset and the IMAP_DPS azimuth zero-point match the legacy NEP convention. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eeze The identity mock previously returned (n_spin, 1, 2), hiding the singleton-dim squeeze that real frame_transform_az_el performs. Return (n_spin, 2) instead so the reshape in compute_pointing_directions is actually exercised and a regression would fail the test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Thanks for the comments @tmplummer - I'm closing this PR since I'll open up another one shortly, with hopefully a lot of these comments incorporated (using existing functions, tests, avoid loops). In any case using this branch and continuing to review it here will cause confusion. Closing this and opening another PR soon.. |
3 participants
Closes issue #3153
Change Summary
Overview
Added a
quickmapproduct for Lo/L1C.This product uses quaternions. I understand there may be strong opinions on using other approaches to get attitude (ephemeris_* etc) (@jtniehof mentioned this) that other instruments are using - perhaps we can open an issue on this and iterate on it once this is merged..
File changes
lo_l1b.py- added alo_l1c_quickmapfunction that takes in sci dependencies and quaternion files and runs the algorithm identified in the issue.Testing
No testing yet ! Once products start getting produced from
sds-data-manager, I'll iterate with the team to weed out any bugs and write meaningful test cases.