Skip to content

VV: Read Oxford Instr. EBSD Data (.ctf) fully V&V'ed - #1692

Open
imikejackson wants to merge 1 commit into
BlueQuartzSoftware:developfrom
imikejackson:vv/ctf_reader
Open

VV: Read Oxford Instr. EBSD Data (.ctf) fully V&V'ed#1692
imikejackson wants to merge 1 commit into
BlueQuartzSoftware:developfrom
imikejackson:vv/ctf_reader

Conversation

@imikejackson

Copy link
Copy Markdown
Contributor

Summary

Full V&V of ReadCtfDataFilter per the v2 MTR V&V policy, structured like the ReadAngDataFilter cycle (#1657). Correctness was established against an independent Class 1 (Analytical) + Class 4 (Invariant) oracle before any legacy comparison; EbsdLib 3.1.0's CtfReader is the trusted Class 2 boundary and is not re-tested.

  • Bugs fixed: 3D multi-slice .ctf files were silently truncated to slice 0 (legacy read the full volume — support restored, bit-identical to 6.5.171 on the 3D fixture); Euler +30°/degrees-to-radians math now uses double-precision intermediates (correctly-rounded float32 results, bit-identical to 6.5.171 — 543,950 of 660,000 production-scan Euler values match exactly, the remainder being the documented unindexed-point deviation).
  • Robustness: six malformed-input guards (-19600-19605) replace paths where 6.5.171 demonstrably segfaults (missing data column; Phases 0 header — both exit 139) or silently emits heap-dependent output (out-of-range phase value).
  • Deviations from 6.5.171: 4 documented entries (vv/deviations/ReadCtfDataFilter.md) — D1 unindexed points keep Phases == 0 (deliberate, PR BUG: ReadCTFData-Remove phase data adjustment. #937), D2 consequent +30° difference on unindexed φ2, D3 malformed-input behavior, D4 3D truncation (fixed/retired). No legacy patch required; every root cause proven from source plus demonstration runs.
  • Tests: the legacy-oracle exemplar test and 6_6_read_ctf_data_2.tar.gz are retired; replaced by 12 TEST_CASEs on inline hand-authored fixtures with hand-derived expected values, including Euler angles chosen where float32 and double roundings differ (pins the precision fix), an exact-boundary phase-range value, and a new SIMPL 6.4/6.5 conversion test.
  • Deliverables: report (vv/ReadCtfDataFilter.md, READY FOR REVIEW), deviations, and provenance sidecar documenting the archive retirement. Filter docs updated (φ2 wording fix, 3D/unindexed/memory/migration sections).
  • Hardened via adversarial, senior-engineer, CPU, memory, and out-of-core review passes; ReadAngData.hpp aligned to the FileSystemPathParameter::ValueType alias convention.

Test Plan

  • ctest -R "OrientationAnalysis::ReadCtfDataFilter" — 12/12 pass (in-core simplnx-Rel)
  • Same 12/12 pass in the out-of-core build (simplnx-ooc-Rel)
  • Full OrientationAnalysis suite — 263/263 pass
  • A/B vs official DREAM3D 6.5.171 PipelineRunner on toy, production (Cugrid 550×400), and 3D fixtures — bit-identical outside documented deviations (artifacts archived to OneDrive per policy, not committed)

Summary:
- Found and fixed 2 bugs (3D multi-slice .ctf silently truncated to
  slice 0; float32 Euler conversion drifted 1 ULP from the correctly-
  rounded double-precision results) and added 6 malformed-input guards
  (-19600..-19605) replacing 2 demonstrated segfault paths and 1 silent
  out-of-bounds read;
- documented 4 deviations from DREAM3D 6.5.171 (D1 phase-0 remap
  removed, D2 +30deg on unindexed phi2, D3 malformed-input crashes,
  D4 3D truncation - retired/fixed);
- retired 1 test (legacy-oracle exemplar comparison vs
  6_6_read_ctf_data_2.tar.gz, download_test_data entry removed);
- unit tests replaced with 12 inlined *Class 1 (Analytical) + Class 4
  (Invariant)* test fixtures, including double-precision-pinning Euler
  angles and a SIMPL 6.4/6.5 backwards-compatibility test;
- added 3 V&V source-tree deliverables (report, deviations, provenance);
- fixed filter doc errors (phi2 wording, markup) and added 3D,
  unindexed-point, memory, and migration-note sections; aligned
  ReadAngData.hpp InputValues to the parameter ValueType alias
  convention.

Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
@imikejackson
imikejackson requested a review from JDuffeyBQ July 24, 2026 19:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant