Describe the bug
In ANSI mode, Comet's wide-decimal arithmetic overflow reports a different value parameter from Spark. PR #5169 fixes the exception type and propagates the error class, SQLSTATE, and query context, but Comet still formats the rescaled i256 result while Spark formats its pre-toPrecision Decimal.
This is a follow-up to #5072.
Steps to reproduce
Using Spark 4.1.2, run the query once with Comet disabled and once with Comet enabled:
SET spark.sql.ansi.enabled = true;
CREATE TABLE tbl (_1 DECIMAL(20, 0)) USING PARQUET;
INSERT INTO tbl VALUES (11000000000000000000);
SELECT _1 * _1 FROM tbl;
Spark reports:
[NUMERIC_VALUE_OUT_OF_RANGE.WITH_SUGGESTION] 121000000000000000000000000000000000000 cannot be represented as Decimal(38, 6). If necessary set "spark.sql.ansi.enabled" to "false" to bypass this error, and return NULL instead. SQLSTATE: 22003
Comet reports:
[NUMERIC_VALUE_OUT_OF_RANGE.WITH_SUGGESTION] 121000000000000000000000000000000000000.000000 cannot be represented as Decimal(38, 6). If necessary set "spark.sql.ansi.enabled" to "false" to bypass this error, and return NULL instead. SQLSTATE: 22003
Expected behavior
Comet should match Spark's value parameter and report 121000000000000000000000000000000000000 without the output-scale suffix.
Additional context
The regression in spark/src/test/scala/org/apache/comet/CometExpressionSuite.scala added by #5169 documents the current difference: Spark formats its pre-toPrecision Decimal, while Comet formats the rescaled i256 value. The test intentionally compares the structured error fields and query context without comparing this differing message parameter.
Describe the bug
In ANSI mode, Comet's wide-decimal arithmetic overflow reports a different
valueparameter from Spark. PR #5169 fixes the exception type and propagates the error class, SQLSTATE, and query context, but Comet still formats the rescaled i256 result while Spark formats its pre-toPrecisionDecimal.This is a follow-up to #5072.
Steps to reproduce
Using Spark 4.1.2, run the query once with Comet disabled and once with Comet enabled:
Spark reports:
Comet reports:
Expected behavior
Comet should match Spark's
valueparameter and report121000000000000000000000000000000000000without the output-scale suffix.Additional context
The regression in
spark/src/test/scala/org/apache/comet/CometExpressionSuite.scalaadded by #5169 documents the current difference: Spark formats its pre-toPrecisionDecimal, while Comet formats the rescaled i256 value. The test intentionally compares the structured error fields and query context without comparing this differing message parameter.