Describe the bug
Native make_interval delegates to DataFusion SparkMakeInterval. Its seconds argument is coerced from Spark Decimal(18,6) to Float64, and its time components are accumulated as i64 nanoseconds. This creates two Spark compatibility gaps:
Float64 coercion can lose microsecond precision.
- Nanosecond accumulation gives hours, minutes, and seconds a 1000x smaller range than Spark's microsecond accumulation, so valid Spark intervals can overflow natively.
Steps to reproduce
SELECT make_interval(0, 0, 0, 0, 2562048);
SELECT make_interval(0, 0, 0, 0, 0, 0, 999999999.000001);
SELECT make_interval(1, 2, 3, 4, 0, 0, 123456789012.123456);
The hours query is valid in Spark but overflows the native nanosecond calculation. The second query loses one microsecond natively. The third returns NULL in non-ANSI mode or throws in ANSI mode natively, while Spark returns a valid interval.
The native overflow cutoffs include hours >= 2,562,048 and minutes >= 153,722,868; large seconds values have the same root cause.
Expected behavior
Native make_interval should preserve Decimal(18,6) microsecond precision and support every hours, minutes, and seconds value Spark can represent.
Additional context
Found while reviewing #5039. Until this is fixed, CometMakeInterval is marked Incompatible: the compatible default uses JVM codegen dispatch, and the native path requires explicit opt-in.
Review context: #5039 (comment)
Describe the bug
Native
make_intervaldelegates to DataFusionSparkMakeInterval. Its seconds argument is coerced from SparkDecimal(18,6)toFloat64, and its time components are accumulated as i64 nanoseconds. This creates two Spark compatibility gaps:Float64coercion can lose microsecond precision.Steps to reproduce
The hours query is valid in Spark but overflows the native nanosecond calculation. The second query loses one microsecond natively. The third returns NULL in non-ANSI mode or throws in ANSI mode natively, while Spark returns a valid interval.
The native overflow cutoffs include hours >= 2,562,048 and minutes >= 153,722,868; large seconds values have the same root cause.
Expected behavior
Native
make_intervalshould preserveDecimal(18,6)microsecond precision and support every hours, minutes, and seconds value Spark can represent.Additional context
Found while reviewing #5039. Until this is fixed,
CometMakeIntervalis marked Incompatible: the compatible default uses JVM codegen dispatch, and the native path requires explicit opt-in.Review context: #5039 (comment)