Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions .github/workflows/tuning.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
name: RD tuning sweep

# Parallel parameter sweep for known-flaws A3/A4. Dispatch builds the codec
# with 8 different ladder/divisor combinations simultaneously. Each job
# with 9 different ladder/divisor combinations simultaneously. Each job
# produces an RD table in the step summary. Compare all tables to pick
# winning constants, then bake them into v2_core.cpp as the new defaults.
# Stock values: scale=1.5, divisor=16.0 (retuned after the photo-pattern RD
# sweep; divisor 16 adds intermediate lossy ladder steps with no regressions).
# Stock values: scale=2.5, divisor=16.0 (retuned after the photo-pattern RD
# sweeps; divisor 16 adds intermediate lossy ladder steps and scale 2.5 yields
# the finest usable ladder - nine lossy steps on natural content).

on:
workflow_dispatch:
Expand All @@ -21,6 +22,9 @@ jobs:
matrix:
include:
- label: stock
scale: "2.5"
divisor: "16.0"
- label: legacy-ladder
scale: "1.5"
divisor: "16.0"
- label: steep-ladder
Expand All @@ -41,8 +45,8 @@ jobs:
- label: gentle-divisor
scale: "1.5"
divisor: "4.0"
- label: steep-divisor
scale: "2.5"
- label: steepest
scale: "3.5"
divisor: "16.0"
steps:
- uses: actions/checkout@v6
Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@ All notable WIMF changes are recorded here. The project follows semantic version

## 2.2.0 - Unreleased

- Retuned the default quality ladder from scale 1.5 to 2.5 (divisor stays 16)
based on the photo/natural RD sweeps: the lossy ladder gains a ninth usable
step on natural content and every quality index produces smaller files at
smoothly increasing PSNR. Same-quality-index output is more compressed and
slightly softer than 2.1; lossless output is unchanged. Old files decode
identically - the container format is untouched.
- Fixed the codec benchmark summary printing "Native speedup 0x": the ratio
was computed inverted and is now guarded against zero reference rates.
- Upgraded the scalar CRC-32 to slice-by-8: all eight slice tables are derived
from the polynomial at compile time, and the bulk loop consumes eight bytes
per iteration instead of one (typically 4-6x table throughput on every
Expand Down
8 changes: 5 additions & 3 deletions benchmarks/codec_summary.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,11 @@ def main() -> int:
)
print(native_row)
print(reference_row)
if native["encode_mpx_s"] > 0 and native["decode_mpx_s"] > 0:
encode_speedup = reference["encode_mpx_s"] / native["encode_mpx_s"]
decode_speedup = reference["decode_mpx_s"] / native["decode_mpx_s"]
if reference["encode_mpx_s"] > 0 and reference["decode_mpx_s"] > 0:
# Speedup is native relative to the Python reference; skip the row
# when a reference rate rounds to zero so the ratio stays finite.
encode_speedup = native["encode_mpx_s"] / reference["encode_mpx_s"]
decode_speedup = native["decode_mpx_s"] / reference["decode_mpx_s"]
print(f"| Native speedup | {encode_speedup:.0f}x | {decode_speedup:.0f}x | - |")
print()
print("Each runner is different hardware: compare within a table, not across tables or runs.")
Expand Down
7 changes: 4 additions & 3 deletions docs/known-flaws.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,10 @@ determinism, corruption rejection, and memory guards all audited clean
streams. Still missing: context modeling of residuals/subbands - the
structural advantage modern image codecs exploit.
- **[P1] A4 Coarse quantizer dead zone.** Largely fixed: monotonic scoring
(A4), the divisor retune 8.0→16.0, and the 0.9x quantizer sub-step in tile
scoring removed the non-monotonicity and the one-ladder-notch gap that showed
as a 34-43 dB jump on the photo pattern. Remaining coarseness is minor; the
(A4), the divisor retune 8.0→16.0, the 0.9x quantizer sub-step in tile
scoring, and the ladder scale retune 1.5→2.5 leave natural-content ladders
with nine smooth lossy steps. The photo pattern retains one Q5→Q6 jump from
per-tile energy bimodality (sharp edges next to flat regions); the
structural win left is the context-modeled entropy stage (A3).
- **[P2] A5 Per-tile framing overhead.** Default 128 px tiles give ~2.8k
independent Zstd frames per 45 MP encode (per scored mode), with no
Expand Down
12 changes: 7 additions & 5 deletions src/v2_core.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,14 @@
#include "zstd.h"
#include "v2_simd.hpp"

// Tunable codec constants. The ladder scale reproduces the historical quality
// ladder; the scoring divisor default was retuned from 8.0 to 16.0 after an
// RD sweep showed it adds intermediate lossy ladder steps with no regressions.
// The tuning workflow overrides these via -D flags to sweep candidate curves.
// Tunable codec constants. The scoring divisor and ladder scale were retuned
// from the historical 8.0/1.5 after photo-pattern RD sweeps: divisor 16 adds
// intermediate lossy ladder steps, and scale 2.5 yields the finest usable
// ladder (nine lossy steps on natural content, smaller files at every quality
// index). The tuning workflow overrides these via -D flags to sweep candidate
// curves.
#ifndef WIMF_LADDER_SCALE
#define WIMF_LADDER_SCALE 1.5f
#define WIMF_LADDER_SCALE 2.5f
#endif
#ifndef WIMF_SCORING_DIVISOR
#define WIMF_SCORING_DIVISOR 16.0
Expand Down
2 changes: 1 addition & 1 deletion tools/wimf_rd_sweep.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
#include <vector>

#ifndef WIMF_LADDER_SCALE
#define WIMF_LADDER_SCALE 1.5f
#define WIMF_LADDER_SCALE 2.5f
#endif
#ifndef WIMF_SCORING_DIVISOR
#define WIMF_SCORING_DIVISOR 16.0
Expand Down
7 changes: 4 additions & 3 deletions wiki/Known-Flaws.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,10 @@ determinism, corruption rejection, and memory guards all audited clean
streams. Still missing: context modeling of residuals/subbands - the
structural advantage modern image codecs exploit.
- **[P1] A4 Coarse quantizer dead zone.** Largely fixed: monotonic scoring
(A4), the divisor retune 8.0→16.0, and the 0.9x quantizer sub-step in tile
scoring removed the non-monotonicity and the one-ladder-notch gap that showed
as a 34-43 dB jump on the photo pattern. Remaining coarseness is minor; the
(A4), the divisor retune 8.0→16.0, the 0.9x quantizer sub-step in tile
scoring, and the ladder scale retune 1.5→2.5 leave natural-content ladders
with nine smooth lossy steps. The photo pattern retains one Q5→Q6 jump from
per-tile energy bimodality (sharp edges next to flat regions); the
structural win left is the context-modeled entropy stage (A3).
- **[P2] A5 Per-tile framing overhead.** Default 128 px tiles give ~2.8k
independent Zstd frames per 45 MP encode (per scored mode), with no
Expand Down
Loading