Core: Update ExpressionParser to match new expressions spec - #17343
Draft
singhpk234 wants to merge 2 commits into
Draft
Core: Update ExpressionParser to match new expressions spec#17343singhpk234 wants to merge 2 commits into
singhpk234 wants to merge 2 commits into
Conversation
singhpk234
changed the title
sfc-gh-prsingh
force-pushed
the
feature/expression-parser-update
branch
2 times, most recently
from
July 24, 2026 00:06
2f2a6ab to
e681062
Compare
Adds UnboundApply and FunctionReference to represent the APPLY value expression from the Iceberg expressions spec, along with factory methods on Expressions. A function reference identifies a function by a multi-part identifier and an optional catalog, as defined by the spec. Known Iceberg partition transforms are recognized by name and can be converted to UnboundTransform so that existing transform handling continues to work. Per Appendix A, partition transforms are functions other than void, so void is not recognized. Transform names are resolved through Transforms rather than a separate list of names. Construction goes through Expressions.function and Expressions.apply, following the pattern used for UnboundPredicate, UnboundTransform, and NamedReference: a public type with a package-private constructor. Argument and identifier lists are stored in mutable lists so that Kryo can deserialize them, matching UnboundPredicate.
Updates ExpressionParser to read and write the predicate and value expression forms defined in Appendix B of format/expressions-spec.md, while continuing to parse the deprecated term-based forms sent by older clients. Predicates are written with 'child' for unary and set operations and 'left'/'right' for comparisons. References are written as objects, and transforms are written as apply expressions with the transform parameter as the first argument. Details worth noting: - Bound references are written and read as field IDs, which are resolved through the schema. Parsing an ID reference without a schema fails rather than guessing a name. - The 'data-type' on literal and literals objects is used to convert values, so typed literals do not decay to strings. - In value expression position a bare string is a literal, not a reference. Bare strings remain references in the deprecated term position. A literal cannot be a predicate operand, so bare strings in 'left' and 'child' are rejected. - IN and NOT_IN always write 'values', including when the list is empty. - Writing a literal value of an unsupported type fails instead of emitting a field name with no value.
sfc-gh-prsingh
force-pushed
the
feature/expression-parser-update
branch
from
July 28, 2026 17:54
e681062 to
134e03b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Description
Summary
Updates the Java
ExpressionParserto serialize predicates in the newform defined in
format/expressions-spec.md,while continuing to accept the deprecated term-based form for backward
compatibility.
This is the Java-side counterpart to #17138, which updates the REST
OpenAPI spec to the new
Predicate/ValueExpressionmodel.Changes
New model classes (
apimodule)FunctionReference— value class for function references, supportingthe four spec forms: bare name, name array,
{identifier}, and{catalog, identifier}.UnboundApply<T>— implementsUnboundTerm<T>to represent generalfunction applications.
bind()delegates toUnboundTransformforknown Iceberg transforms (identity, year, month, day, hour, bucket,
truncate, void); other functions currently throw at bind time.
Parser changes (
coremodule)child/left/rightinstead ofterm/value{"type": "reference", "name": ...}forunbound and
{"type": "reference", "id": N}for boundapplywith thearguments (e.g.
bucket[100](id)→{"type": "apply", "function": "bucket", "arguments": [100, {"type": "reference", "name": "id"}]})termfield, the deprecated path parses it(existing behavior)
child/left/right,references (
nameor deprecatedterm), andapplyexpressionsapplyare converted back toUnboundTransformso downstream binding paths are unchangedTests
TestExpressionParser,TestPlanTableScanRequestParser,TestFileScanTaskParser,TestPlanTableScanResponseParser,TestFetchPlanningResultResponseParser,TestFetchScanTasksResponseParser,TestAllManifestsTableTaskParser, andTestFilesTableTaskParsertothe new writer output.
predicate form (
testReadDeprecatedUnaryPredicate,testReadDeprecatedComparisonPredicate,testReadDeprecatedSetPredicate,testReadDeprecatedTransformPredicate,testReadDeprecatedObjectReference).testNewFormatRoundTrip,testApplySimpleFunctionName,testApplyWithCatalogFunction,testNewFormatNamedReference,testNewFormatTypedLiteral.Test plan
./gradlew :iceberg-api:test./gradlew :iceberg-core:test --tests "org.apache.iceberg.expressions.*"./gradlew :iceberg-core:test --tests "org.apache.iceberg.rest.*"./gradlew :iceberg-core:test --tests "org.apache.iceberg.metrics.*"./gradlew :iceberg-core:test --tests "org.apache.iceberg.TestFileScanTaskParser"./gradlew spotlessApply