Truncate transform diverges from Iceberg Java's TruncateUtil for widths above 2^30, and panics in debug builds at the integer minimum

[andygrove]

Apache Iceberg Rust version

main @ 3d84c81353b1b23b6e4ae8eea8f8a021cc6927a7

Describe the bug

Truncate::transform in crates/iceberg/src/transform/truncate.rs does not reproduce Iceberg Java's TruncateUtil arithmetic. There are two distinct problems.

1. truncate_i32 diverges from Java for widths above 2^30

fn truncate_i32(v: i32, width: i32) -> i32 {
    v - v.rem_euclid(width)
}

Java computes the positive remainder with an expression that is allowed to overflow:

// org.apache.iceberg.util.TruncateUtil
public static int truncateInt(int width, int value) {
  return value - (((value % width) + width) % width);
}

(value % width) + width overflows int whenever value % width > Integer.MAX_VALUE - width, which is reachable for any width > 2^30. rem_euclid never overflows, so the two produce different partition values. Only positive values with a large width are affected; negative values and widths at or below 2^30 agree.

Truncate.get(int width) in Iceberg Java only validates width > 0, so these widths are accepted by the spec. Whether Java's overflow is desirable is beside the point — it is the value iceberg-java writes into the partition tuple, so a Rust writer that computes something else produces a table the Java reader partitions differently.

2. Overflow panics in debug builds at the integer minimum

fn truncate_i64(v: i64, width: i64) -> i64 {
    v - (((v % width) + width) % width)
}

For v = i64::MIN, width = 1000 the remainder is 192 and v - 192 underflows. With debug_assertions on this panics (attempt to subtract with overflow); in a release build it wraps and matches Java exactly. truncate_i32 has the same shape at i32::MIN, and truncate_decimal_i128 uses the same unchecked subtraction (unreachable for a 38-digit Iceberg decimal, but the same pattern).

A panic here aborts the write task, so a debug or test build of any downstream writer fails on data that Java handles.

To Reproduce

Values below were produced by running TruncateUtil from apache-iceberg 1.11.0 on JDK 17 against the Rust expressions:

width	value	Java TruncateUtil.truncateInt	iceberg-rust truncate_i32
2000000000	1500000000	-2000000000	0
2147483647	2147483646	-2147483647	0
1073741824	1500000000	1073741824	1073741824
1000	2147483647	2147483000	2147483000

For problem 2, in a debug build:

let t = create_transform_function(&Transform::Truncate(1000)).unwrap();
t.transform(Arc::new(Int64Array::from(vec![i64::MIN]))).unwrap(); // panics

Expected behavior

Both kernels bit-identical to Java in debug and release. Wrapping arithmetic gives that:

fn truncate_i32(v: i32, w: i32) -> i32 {
    v.wrapping_sub((v % w).wrapping_add(w) % w)
}

fn truncate_i64(v: i64, w: i64) -> i64 {
    v.wrapping_sub((v % w).wrapping_add(w) % w)
}

That fixes both problems at once: it reproduces Java's overflow for large widths and removes the debug panic. The alternative for problem 1 is to reject width > 2^30 at construction, which overlaps with #2473, but that would reject specs iceberg-java accepts.

Willingness to contribute

I would be willing to contribute a fix for this bug with guidance from the Iceberg community.

---

How this was found. Apache DataFusion Comet is adding native kernels for Iceberg's Spark system functions (apache/datafusion-comet#5638). On a partitioned write the sort key comes from those kernels (Java semantics) while the partition values the clustered writer groups by come from iceberg-rust's transforms, and the clustered writer fails at runtime with The input is not sorted! Cannot write to partition that was previously closed when the two disagree. Cross-checking the two implementations over boundary inputs is what surfaced this.

This report was drafted with LLM assistance (Claude Code); the divergences were verified against Iceberg Java on a JVM.