Repository navigation
[C++][Substrait] ProjectRel conversion aborts on computed output types #55493
Description
Describe the bug, including details regarding any error messages, version, and platform.
Converting a (presumably) valid Substrait ProjectRel aborts in Debug mode when a projected expression uses a kernel with a computed output type.
Calling DeserializePlan aborts in arrow::compute::OutputType::type()
at cpp/src/arrow/compute/kernel.cc:459:
Check failed: (FIXED) == (kind_)
Adding the (AI generated) test below to cpp/src/arrow/engine/substrait/serde_test.cc reproduces the abort on macOS arm64 with a Debug build of main at 7f1f38d, with ARROW_COMPUTE, ARROW_ACERO, and ARROW_SUBSTRAIT enabled.
This reproduces it with boolean columns A and B, an int32 column C, and a projected IfThen expression equivalent to:
CASE WHEN A THEN C WHEN B THEN 0 ELSE -1 ENDProjectRel conversion currently reads the kernel signature's output type through out_type().type() which only supports fixed output types.
I did not test Release behavior, but it would presumably return a null type rather than abort.
TEST(SubstraitRoundTrip, ProjectRelWithComputedOutputType) {
auto input_schema =
schema({field("A", boolean()), field("B", boolean()), field("C", int32())});
auto input_table = TableFromJSON(input_schema, {R"([
[true, false, 10],
[false, true, 20],
[false, false, 30],
[null, null, 40]
])"});
std::string substrait_json = R"({
"version": {"major_number": 9999, "minor_number": 9999, "patch_number": 9999},
"relations": [{"rel": {"project": {
"expressions": [{"if_then": {
"ifs": [
{
"if": {"selection": {
"directReference": {"structField": {"field": 0}}, "rootReference": {}
}},
"then": {"selection": {
"directReference": {"structField": {"field": 2}}, "rootReference": {}
}}
},
{
"if": {"selection": {
"directReference": {"structField": {"field": 1}}, "rootReference": {}
}},
"then": {"literal": {"i32": 0}}
}
],
"else": {"literal": {"i32": -1}}
}}],
"input": {"read": {
"base_schema": {
"names": ["A", "B", "C"],
"struct": {"types": [{"bool": {}}, {"bool": {}}, {"i32": {}}]}
},
"namedTable": {"names": ["A"]}
}}
}}}]
})";
ASSERT_OK_AND_ASSIGN(auto buf,
internal::SubstraitFromJSON("Plan", substrait_json,
/*ignore_unknown_fields=*/false));
ConversionOptions conversion_options;
conversion_options.named_table_provider = AlwaysProvideSameTable(input_table);
// case_when resolves its output type when the expression is bound.
ASSERT_OK_AND_ASSIGN(auto plan_info,
DeserializePlan(*buf, /*registry=*/nullptr,
/*ext_set_out=*/nullptr, conversion_options));
const auto& output_schema = plan_info.root.output_schema;
ASSERT_EQ(4, output_schema->num_fields());
ASSERT_NE(nullptr, output_schema->field(3)->type());
AssertTypeEqual(*int32(), *output_schema->field(3)->type());
auto expected_schema = schema({field("A", boolean()), field("B", boolean()),
field("C", int32()), field("case_when", int32())});
auto expected_table = TableFromJSON(expected_schema, {R"([
[true, false, 10, 10],
[false, true, 20, 0],
[false, false, 30, -1],
[null, null, 40, -1]
])"});
CheckRoundTripResult(std::move(expected_table), buf, /*include_columns=*/{},
conversion_options);
}Component(s)
C++
Activity