Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 36 additions & 36 deletions Docs/ChangeLog-6x.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,15 @@ at 3.9 GHz, running `astcenc` using AVX2 and 12 threads.
<!-- ---------------------------------------------------------------------- -->
## 6.0.0

**Status:** In development.
**Status:** Released November 2026.

The 6.0.0 release is a major release, adding support for Rate-Distortion
Optimization (RDO) to both CLI and library, and support for AVX-512 as a
library backend. There is also significant performance uplifit across all
backends, and numerous small improvements to image quality and stability.
The 6.0.0 release is a major release, adding support for rate-distortion
optimization (RDO) to both the CLI and the library, and support for AVX-512 as
a library backend. There is also a significant performance uplift across all
backends, along with numerous small improvements to image quality and
stability.

Thank you to Epic Games for the contribution of the LZ RDO implementation, and
Thank you to Epic Games for the contribution of the LZ RDO implementation and
to AMD for the contribution of the AVX-512 backend.

This release includes changes to the public interface in the `astcenc.h`
Expand All @@ -29,28 +30,26 @@ header from the same release to avoid compatibility issues.
* **API Change:** Use `size_t` for image dimensions, context thread counts,
and thread indices in the library API.
* **API Change:** Add support for rate-distortion optimization to reduce
size of bundled ASTC texture assets. Enable by setting the config flag
`ASTCENC_FLG_USE_LZ_RDO` and setting the appropriate `lz_rdo_lambda`
level (0 = no RDO, 20 = light RDO, 40 = moderate RDO ...).
the size of bundled ASTC texture assets. Enable it by setting the config
flag `ASTCENC_FLG_USE_LZ_RDO` and setting the appropriate `lz_rdo_lambda`
value (0 = no RDO, 20 = light RDO, 40 = moderate RDO, etc.).
* **API Change:** Decompressing images containing invalid encodings, or HDR
encodings in an LDR profile, now return an identifiable
encodings in an LDR profile, now returns an identifiable
`ASTCENC_ERR_BAD_BLOCK` return code. The image will still be decompressed,
using the specified block error color for the ASTC profile in use. The
command line tool will write the image, emit a warning, and exit with code
2 for these failures. Functional errors continue to use exit code 1.
* **CLI Change:** Add support for rate-distortion optimization to reduce
size of bundled ASTC texture assets. Enable by passing the `-rdo-lambda`
command line option. See API Change above for details.
the size of bundled ASTC texture assets. Enable it by passing the
`-rdo-lambda` command line option. See the API change above for details.
* **Feature:** Add an AVX-512 SIMD backend to the library, with a
corresponding command line tool variant operating on 512-bit vectors.
AVX-512 performance can be slower than AVX2 for smaller block sizes.
* **Build Change:** AVX2 builds now also require BMI and BMI2.
functions with new standard library operations where possible.
* **Build Change:** Require C++20 to build the codec, replacing local library
functions with new standard library operations where possible.
* **Build Change:** Minimum supported compiler versions increased to Clang
18, GCC 13, and Visual Studio 2022. Minimum supported CMake version
increased to 3.21.
* **Build Change:** Require Clang 18 or newer, GCC 13 or newer, or Visual
Studio 2022 or newer. Require CMake 3.21 or newer.
* **Codec library updates:**
* **Bug fix:** Disable dual-plane encoding at a zero correlation cutoff.
* **Bug fix:** Preserve HDR endpoint mode bits when quantization rounds down
Expand All @@ -63,20 +62,19 @@ header from the same release to avoid compatibility issues.
* **Bug fix:** Preserve the error threshold inherited by child contexts,
avoiding a second conversion from dB.
* **Image quality:** Reduce visible HDR artifacts with perceptual mode
enabled by weighting logarithmic channel errors according to geomean
of channel brightness.
* **Optimization:** Restructuture partition candidate selection by processing
enabled by weighting logarithmic channel errors according to the geometric
mean of channel brightness.
* **Optimization:** Restructure partition candidate selection by processing
multiple candidates in parallel on wider vector hardware, one candidate for
every 128-bits in the available SIMD vectors. This significantly improves
every 128 bits in the available SIMD vectors. This significantly improves
performance on 256-bit and 512-bit vector hardware.
* **Optimization:** Specialize a simpler partition candidate selection for
luminance-only blocks, as we know that direction is always through the
origin.
* **Optimization:** Specialize partition candidate selection for
luminance-only blocks, whose color values lie on a line through the origin.
* **Optimization:** Specialize many data paths for luminance+alpha data,
which is commonly used for normal maps.
* **Optimization:** Use analytical calculation of weight change error diffs
* **Optimization:** Use analytical calculations of weight change error deltas
in `realign_weights_decimated()` and `realign_weights_undecimated()`.
Changes compute simpler linear and quadratic error coefficients, and then
Compute simpler linear and quadratic error coefficients, then use them to
compute the up and down candidate deltas.
* **Optimization:** Precompute an infilled weight set for the realign loop
in `realign_weights_decimated()`, rebuilding set entries if a decimated
Expand All @@ -87,34 +85,36 @@ header from the same release to avoid compatibility issues.
scan.
* **Optimization:** Skip RGB+lumashift endpoint error calculations for LDR
blocks, as this encoding is only available for HDR.
* **Optimization:** Interpolate deltas in `lerp_color_int()` and unrolled
* **Optimization:** Interpolate deltas in `lerp_color_int()` and its unrolled
specialization, saving one multiply.
* **Optimization:** Directly copy non-decimated weight grids when unpacking
symbolic blocks, avoiding redundant interpolation.
* **Optimization:** Use SoA-style SIMD batches for k-means cluster assignment.
* **Optimization:** Avoid constructing three- and four-partition
endpoint-format combinations that exceed the allowed endpoint integer limit.
endpoint-format combinations that exceed the allowed endpoint integer
limit.
* **Optimization:** Split multi-partition endpoint-format searches by
quantization level to skip unreachable high-precision combinations.
* **Optimization:** Specialize ISE decoding at compile-time into functions
per quant level to help with RDO performance.
* **Optimization:** Optimized implementaiton of unorm16_to_sf16 to help with
RDO performance.
* **Optimization:** Specialize ISE decoding at compile time into functions
for each quantization level to improve RDO performance.
* **Optimization:** Optimize the implementation of `unorm16_to_sf16()` to
improve RDO performance.
* **Optimization:** Use NEON combined shift-and-insert for byte packing in
`interleave_rgba8()`.
* **Optimization:** Use SVE narrowing stores in the 4-wide implementation of
`pack_and_store_low_bytes()`.
* **Command line tool updates:**
* **CLI Change:** Writing to an LDR output file format, such as PNG, when
test compressing an HDR image is now supported. The tool will output
compressing an HDR image in test mode is now supported. The tool will output
multiple files, one for each active integer f-stop exposure level.
* **Feature:** Report SSIM for LDR images and exposure-averaged mSSIM for
HDR images in test mode. HDR exposures follow the `-mpsnr` range.
* **Feature:** Report DEFLATE compression when using RDO. Note - requires
CLI to be built with zlib and is not enabled in our release binaries.
* **Feature:** Report DEFLATE compression when using RDO. This requires
building the CLI with zlib support, which is not enabled in our release
binaries.
* **Change:** Automatically center HDR image quality metric exposures on the
original image's geometric mean luminance, covering two stops either side.
An explicit `-mpsnr` range overrides automatic exposure selection.
original image's geometric mean luminance, covering two stops on either
side. An explicit `-mpsnr` range overrides automatic exposure selection.
* **Feature:** Add crop region support for compression. Crop regions are
extracted in the image coordinate space before flipping or preprocessing.
* **Build system updates:**
Expand Down
55 changes: 27 additions & 28 deletions Docs/RDO.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,35 +3,35 @@
Rate-distortion optimization (RDO) trades image quality for smaller files after
applying additional lossless compression to compressed ASTC files. It is used
to reduce the size of application distribution bundles, providing finer-grained
control over storage size than simply reducing ASTC block size.
control over storage size than simply changing the ASTC block size.

> [!NOTE]
> The RDO implementation in `astcenc` currently only supports LDR textures.

## How RDO works

ASTC compression is a fixed-rate compression that encodes every block into 16
bytes of data. For a given image size and block footprint, the ASTC payload
data has a fixed size. RDO is a post-process that changes the compressed ASTC
bitstream to make byte patterns more similar to nearby bytes in the stream.
This makes the data more compressible when applying subsequent lossless
compression, such as ZIP or LZ4.
ASTC uses fixed-rate compression, encoding every block into 16 bytes of data.
For a given image size and block footprint, the ASTC payload data has a fixed
size. RDO is a post-process that changes the compressed ASTC bitstream to make
byte patterns more similar to nearby patterns in the stream. This makes the
data more compressible when applying subsequent lossless compression, such as
ZIP or LZ4.

The RDO transform is lossy, sacrificing image quality to improve data
compressibility. The amount of allowed quality loss is controlled by a lambda
setting, `-rdo-lambda` for the command line tool, where a higher lambda values
allow more image distortion in exchange for a more compressible pattern. A
lambda value of zero disables the RDO transform. We recommend using a value
of 20 as a starting point, which give a reasonable tradeoff between quality
and storage. Normal maps are more sensitive to introduced error, so a lower
lambda value might be required.

The ASTC blocks after RDO has been applied are still standard ASTC data blocks.
RDO does not require a special GPU decoder, and it does not change the number of
blocks or the size of each block. To realize the storage saving, additional
setting (`-rdo-lambda` in the command line tool). Higher lambda values allow
more image distortion in exchange for more compressible data. A lambda value
of zero disables the RDO transform. We recommend using a value of 20 as a
starting point, which gives a reasonable tradeoff between quality and storage.
Normal maps are more sensitive to introduced error, so a lower lambda value
might be required.

The blocks remain standard ASTC data blocks after RDO has been applied. RDO
does not require a special GPU decoder, and it does not change the number of
blocks or the size of each block. To realize the storage savings, additional
lossless compression must be applied in your asset packaging pipeline. For
Android, this is compression is commonly applied when generating an Application
APK (application bundle) or AAB (asset bundle).
Android, this compression is commonly applied when generating an APK or
Android App Bundle (AAB).

### Using RDO versus changing the ASTC block size

Expand All @@ -40,18 +40,17 @@ can choose to use a larger ASTC block size, which simply reduces the amount of
data that needs to be packaged, or you can choose to use RDO to make data more
compressible.

Where possible we recommend using a larger ASTC block size. This gives you
additional savings to run-time memory bandwidth, in addition to the packging
size reduction.
Where possible, we recommend using a larger ASTC block size. This reduces
runtime memory bandwidth requirements as well as packaged file size.

We recommend using RDO to fine-tune packaging size when dropping to the next
lowest ASTC bitrate is not an option. This could be because the next bitrate
option loses too much detail, or suffers from more visible block artifacts.
We recommend using RDO to fine-tune packaged file size when moving to the next
lower ASTC bitrate is not an option. This could be because the next bitrate
setting loses too much detail or introduces more visible block artifacts.

## Compressing with RDO

Use the `-rdo-lambda` command line option to enable RDO and set its strength.
Setting 0 disables RDO, higher values increase the amount of quality loss
A value of 0 disables RDO. Higher values increase the amount of quality loss
permitted.

```shell
Expand All @@ -66,8 +65,8 @@ build-rdo/Source/astcenc-avx2 -cl input.png rdo.astc 5x5 -thorough \

The RDO lambda value is a heuristic to control tradeoffs, not a specific target
bitrate or PSNR threshold. Lower positive values favor fidelity; higher values
place more weight on storage savings. You much measure quality and check that
it is suitable for your content.
place more weight on storage savings. You must measure quality and check that
the results are suitable for your content.

- - -

Expand Down
26 changes: 26 additions & 0 deletions Test/Images/HDRIHaven/astc_reference-6.0-avx2_fast_results.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
Image Set,Block Size,Name,PSNR,SSIM,Total Time,Coding Time,Coding Rate
HDRIHaven,4x4,hdr-rgb-arboretum.hdr,42.0149,0.9944,0.3071,0.0312,18.8865
HDRIHaven,4x4,hdr-rgb-bellparkpier.hdr,53.4759,0.9983,0.3033,0.0291,20.2772
HDRIHaven,4x4,hdr-rgb-canarywharf.hdr,45.6729,0.9954,0.3113,0.0327,18.0557
HDRIHaven,4x4,hdr-rgb-eveningroad.hdr,49.4453,0.9953,0.3071,0.0303,19.4757
HDRIHaven,4x4,hdr-rgb-riverwalk.hdr,31.1255,0.9734,0.3233,0.0416,14.1785
HDRIHaven,5x5,hdr-rgb-arboretum.hdr,37.7087,0.9871,0.3067,0.0270,21.8525
HDRIHaven,5x5,hdr-rgb-bellparkpier.hdr,49.7809,0.9968,0.3052,0.0258,22.8878
HDRIHaven,5x5,hdr-rgb-canarywharf.hdr,42.0143,0.9902,0.3039,0.0276,21.3604
HDRIHaven,5x5,hdr-rgb-eveningroad.hdr,46.5177,0.9913,0.3020,0.0251,23.5410
HDRIHaven,5x5,hdr-rgb-riverwalk.hdr,27.2399,0.9369,0.3096,0.0328,18.0000
HDRIHaven,6x6,hdr-rgb-arboretum.hdr,33.8423,0.9718,0.3041,0.0252,23.4131
HDRIHaven,6x6,hdr-rgb-bellparkpier.hdr,46.9213,0.9945,0.3022,0.0253,23.3391
HDRIHaven,6x6,hdr-rgb-canarywharf.hdr,39.2445,0.9824,0.3030,0.0272,21.6745
HDRIHaven,6x6,hdr-rgb-eveningroad.hdr,44.0075,0.9851,0.3029,0.0260,22.6490
HDRIHaven,6x6,hdr-rgb-riverwalk.hdr,25.0114,0.8984,0.3089,0.0296,19.9198
HDRIHaven,8x8,hdr-rgb-arboretum.hdr,29.2843,0.9240,0.3118,0.0357,16.5143
HDRIHaven,8x8,hdr-rgb-bellparkpier.hdr,43.3356,0.9887,0.3130,0.0341,17.2862
HDRIHaven,8x8,hdr-rgb-canarywharf.hdr,35.9927,0.9629,0.3141,0.0360,16.4054
HDRIHaven,8x8,hdr-rgb-eveningroad.hdr,40.8319,0.9688,0.3136,0.0352,16.7793
HDRIHaven,8x8,hdr-rgb-riverwalk.hdr,22.5462,0.8259,0.3178,0.0378,15.6215
HDRIHaven,12x12,hdr-rgb-arboretum.hdr,25.7063,0.8278,0.3254,0.0448,13.1751
HDRIHaven,12x12,hdr-rgb-bellparkpier.hdr,39.1784,0.9739,0.3187,0.0419,14.0686
HDRIHaven,12x12,hdr-rgb-canarywharf.hdr,32.9256,0.9250,0.3236,0.0440,13.3993
HDRIHaven,12x12,hdr-rgb-eveningroad.hdr,37.8554,0.9374,0.3248,0.0438,13.4583
HDRIHaven,12x12,hdr-rgb-riverwalk.hdr,20.3186,0.7073,0.3276,0.0466,12.6621
26 changes: 26 additions & 0 deletions Test/Images/HDRIHaven/astc_reference-6.0-avx2_fastest_results.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
Image Set,Block Size,Name,PSNR,SSIM,Total Time,Coding Time,Coding Rate
HDRIHaven,4x4,hdr-rgb-arboretum.hdr,41.8147,0.9941,0.3016,0.0197,29.8737
HDRIHaven,4x4,hdr-rgb-bellparkpier.hdr,53.1427,0.9982,0.3001,0.0198,29.8632
HDRIHaven,4x4,hdr-rgb-canarywharf.hdr,44.9703,0.9949,0.2993,0.0199,29.5769
HDRIHaven,4x4,hdr-rgb-eveningroad.hdr,49.2381,0.9951,0.3001,0.0195,30.1949
HDRIHaven,4x4,hdr-rgb-riverwalk.hdr,30.7680,0.9715,0.3032,0.0228,25.8955
HDRIHaven,5x5,hdr-rgb-arboretum.hdr,37.5899,0.9867,0.2997,0.0177,33.3743
HDRIHaven,5x5,hdr-rgb-bellparkpier.hdr,49.4068,0.9966,0.2967,0.0182,32.3441
HDRIHaven,5x5,hdr-rgb-canarywharf.hdr,41.5788,0.9892,0.2954,0.0180,32.7149
HDRIHaven,5x5,hdr-rgb-eveningroad.hdr,46.2062,0.9907,0.2941,0.0183,32.2450
HDRIHaven,5x5,hdr-rgb-riverwalk.hdr,26.7882,0.9295,0.3002,0.0203,29.0041
HDRIHaven,6x6,hdr-rgb-arboretum.hdr,33.7636,0.9713,0.2982,0.0177,33.2782
HDRIHaven,6x6,hdr-rgb-bellparkpier.hdr,46.6616,0.9941,0.2998,0.0172,34.2205
HDRIHaven,6x6,hdr-rgb-canarywharf.hdr,39.0303,0.9814,0.2951,0.0180,32.8504
HDRIHaven,6x6,hdr-rgb-eveningroad.hdr,43.6676,0.9838,0.2954,0.0169,34.9960
HDRIHaven,6x6,hdr-rgb-riverwalk.hdr,24.8221,0.8937,0.2952,0.0188,31.3204
HDRIHaven,8x8,hdr-rgb-arboretum.hdr,29.1970,0.9226,0.3050,0.0251,23.4718
HDRIHaven,8x8,hdr-rgb-bellparkpier.hdr,42.8763,0.9872,0.3048,0.0236,24.9438
HDRIHaven,8x8,hdr-rgb-canarywharf.hdr,35.7346,0.9610,0.3059,0.0251,23.4580
HDRIHaven,8x8,hdr-rgb-eveningroad.hdr,40.4613,0.9662,0.3040,0.0236,24.9642
HDRIHaven,8x8,hdr-rgb-riverwalk.hdr,22.4793,0.8233,0.3081,0.0277,21.3249
HDRIHaven,12x12,hdr-rgb-arboretum.hdr,25.6284,0.8253,0.3062,0.0307,19.2237
HDRIHaven,12x12,hdr-rgb-bellparkpier.hdr,38.6036,0.9712,0.3086,0.0283,20.8337
HDRIHaven,12x12,hdr-rgb-canarywharf.hdr,32.6532,0.9217,0.3082,0.0293,20.1374
HDRIHaven,12x12,hdr-rgb-eveningroad.hdr,37.6871,0.9357,0.3092,0.0307,19.2169
HDRIHaven,12x12,hdr-rgb-riverwalk.hdr,20.2840,0.7051,0.3142,0.0334,17.6436
26 changes: 26 additions & 0 deletions Test/Images/HDRIHaven/astc_reference-6.0-avx2_medium_results.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
Image Set,Block Size,Name,PSNR,SSIM,Total Time,Coding Time,Coding Rate
HDRIHaven,4x4,hdr-rgb-arboretum.hdr,42.1577,0.9946,0.3251,0.0439,13.4476
HDRIHaven,4x4,hdr-rgb-bellparkpier.hdr,53.8335,0.9985,0.3186,0.0418,14.0968
HDRIHaven,4x4,hdr-rgb-canarywharf.hdr,46.0086,0.9957,0.3274,0.0476,12.3915
HDRIHaven,4x4,hdr-rgb-eveningroad.hdr,49.8342,0.9957,0.3234,0.0435,13.5576
HDRIHaven,4x4,hdr-rgb-riverwalk.hdr,31.4770,0.9756,0.3391,0.0584,10.1056
HDRIHaven,5x5,hdr-rgb-arboretum.hdr,37.7818,0.9874,0.3215,0.0411,14.3541
HDRIHaven,5x5,hdr-rgb-bellparkpier.hdr,50.1733,0.9971,0.3240,0.0428,13.7906
HDRIHaven,5x5,hdr-rgb-canarywharf.hdr,42.4882,0.9911,0.3253,0.0465,12.6806
HDRIHaven,5x5,hdr-rgb-eveningroad.hdr,46.7628,0.9919,0.3232,0.0424,13.9046
HDRIHaven,5x5,hdr-rgb-riverwalk.hdr,27.4874,0.9407,0.3323,0.0525,11.2333
HDRIHaven,6x6,hdr-rgb-arboretum.hdr,33.8900,0.9723,0.3234,0.0437,13.5085
HDRIHaven,6x6,hdr-rgb-bellparkpier.hdr,47.4019,0.9950,0.3211,0.0435,13.5689
HDRIHaven,6x6,hdr-rgb-canarywharf.hdr,39.7608,0.9840,0.3248,0.0475,12.4291
HDRIHaven,6x6,hdr-rgb-eveningroad.hdr,44.2119,0.9857,0.3215,0.0445,13.2566
HDRIHaven,6x6,hdr-rgb-riverwalk.hdr,25.1981,0.9030,0.3311,0.0519,11.3675
HDRIHaven,8x8,hdr-rgb-arboretum.hdr,29.3765,0.9250,0.3359,0.0588,10.0296
HDRIHaven,8x8,hdr-rgb-bellparkpier.hdr,43.6345,0.9893,0.3367,0.0554,10.6444
HDRIHaven,8x8,hdr-rgb-canarywharf.hdr,36.3013,0.9650,0.3373,0.0609,9.6789
HDRIHaven,8x8,hdr-rgb-eveningroad.hdr,40.9622,0.9695,0.3356,0.0561,10.5073
HDRIHaven,8x8,hdr-rgb-riverwalk.hdr,22.6140,0.8284,0.3437,0.0624,9.4558
HDRIHaven,12x12,hdr-rgb-arboretum.hdr,25.7452,0.8289,0.3528,0.0733,8.0489
HDRIHaven,12x12,hdr-rgb-bellparkpier.hdr,39.3828,0.9751,0.3463,0.0672,8.7831
HDRIHaven,12x12,hdr-rgb-canarywharf.hdr,33.1104,0.9277,0.3571,0.0753,7.8287
HDRIHaven,12x12,hdr-rgb-eveningroad.hdr,37.9923,0.9387,0.3513,0.0713,8.2710
HDRIHaven,12x12,hdr-rgb-riverwalk.hdr,20.3313,0.7079,0.3578,0.0749,7.8752
Loading
Loading