Skip to content

Round half floats up into the next power of two, and read small ones at their value - #334

Merged
LogicalError merged 1 commit into
mainfrom
tachyonui/half-float-rounding
Nov 21, 2026
Merged

LogicalError merged 1 commit into
mainfrom
tachyonui/half-float-rounding

Conversation

@LogicalError

Copy link
Copy Markdown
Owner

FloatToHalf could lose a carry when rounding, and HalfToFloat read the smallest values at half their size. The styles use both to pack a gradient's rotation into 16-bit floats, so a gradient at an angle near an axis pointed the wrong way.

Cause

  • FloatToHalf: it rounded the mantissa up, then masked it to 10 bits. A value that rounded past the largest mantissa lost the carry instead of moving to the next power of two:
    • 0.99995 came out as 0.5
    • 1.9999 came out as 1.0
    • 65530 came out as 32768
  • HalfToFloat: it normalized a subnormal half with the exponent off by one, so every subnormal read as half its value.

What changes

  • FloatToHalf: it now adds the rounding to the exponent and mantissa together. Rounding past the largest mantissa carries into the exponent, and rounding past the largest half gives infinity.
  • HalfToFloat: it now normalizes a subnormal with the right exponent.
  • Changelog: an entry for both fixes.

Tests

  • New: rounding up carries into the exponent, for 0.99995, 1.9999 and -1.9999.
  • New: 65519 rounds down to the largest half, and 65530 rounds up to infinity.
  • New: subnormals read as their value.
  • New: every finite half comes back as itself through a float. That test is what found the HalfToFloat bug.

Validation

  • Full Debug build: no errors or warnings.
  • gui_core_tests, gui_vulkan_tests, gui_gpu_tests, gui_d3d12_tests and gui_sdlgpu_tests all pass on Windows.

…at their value

FloatToHalf rounded the mantissa up and then masked it to 10 bits, so a
value that rounded past the largest mantissa lost the carry: 0.99995
became 0.5, 1.9999 became 1.0, and 65530 became 32768. A gradient's
rotation is packed this way, so one at an angle near an axis pointed the
wrong way. HalfToFloat read subnormal halves as half their value.

- FloatToHalf adds the rounding to exponent and mantissa together: it
  carries into the exponent, and to infinity past the largest half.
- HalfToFloat normalizes a subnormal with the right exponent.
- Tests: rounding up carries into the exponent, rounding past the largest
  half is infinity, subnormals read as their value, and every finite half
  comes back as itself through a float.
@LogicalError
LogicalError merged commit a4fe98c into main Nov 21, 2026
8 checks passed
@LogicalError
LogicalError deleted the tachyonui/half-float-rounding branch November 21, 2026 21:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant