Skip to content

Raise SQL identifier limit to 253 to match K8s name limit - #258

Merged
jogrogan merged 2 commits into
mainfrom
jogrogan/pipelineLength
Oct 8, 2026
Merged

jogrogan merged 2 commits into
mainfrom
jogrogan/pipelineLength

Conversation

@jogrogan

@jogrogan jogrogan commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator

Generated pipeline/view/job identifiers can exceed Calcite's default identifier length of 128, producing "Length of identifier ... must be less than or equal to 128 characters" parse errors. These identifiers ultimately become Kubernetes resource names, whose ceiling is the DNS-subdomain limit of 253 characters.

  • Add shared IdentifierUtils.MAX_IDENTIFIER_LENGTH (253) and apply it to both parser paths: IdentifierUtils (quoted bare-string parsing used by graph resolution, CLI !resolve/!graph, and PipelineGraphBuilder) and HoptimatorDriver (DDL/query parser).
  • Fix K8sUtils.checkK8sName, which incorrectly capped names at the 63-char DNS-label limit; that limit applies to label values and label-typed resources (e.g. Services), not to these DNS-subdomain resource names. Now capped at 253 via the shared constant.
  • Add unit tests for both parser paths and the name guard (accept at 253, reject at 254), and a real-k8s integration test asserting the apiserver's 422 "must be no more than 253 characters" response.

Realistically we should be able to support larger identifiers but we'd need to add some sort of deterministic hashing logic for the k8s CR names which can make observability difficult, so we'll punt this for now.

jogrogan and others added 2 commits October 8, 2026 17:24
Generated pipeline/view/job identifiers routinely exceed Calcite's
default identifier length of 128, producing cryptic
"Length of identifier ... must be less than or equal to 128 characters"
parse errors. These identifiers ultimately become Kubernetes resource
names, whose ceiling is the DNS-subdomain limit of 253 characters.

- Add shared IdentifierUtils.MAX_IDENTIFIER_LENGTH (253) and apply it to
  both parser paths: IdentifierUtils (quoted bare-string parsing used by
  graph resolution, CLI !resolve/!graph, and PipelineGraphBuilder) and
  HoptimatorDriver (DDL/query parser).
- Fix K8sUtils.checkK8sName, which incorrectly capped names at the
  63-char DNS-label limit; that limit applies to label values and
  label-typed resources (e.g. Services), not to these DNS-subdomain
  resource names. Now capped at 253 via the shared constant.
- Add unit tests for both parser paths and the name guard (accept at
  253, reject at 254), and a real-k8s integration test asserting the
  apiserver's 422 "must be no more than 253 characters" response.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Code Coverage

Overall Project 84.98% 🟢
Files changed 100% 🟢

File Coverage
IdentifierUtils.java 100% 🟢
K8sUtils.java 100% 🟢
HoptimatorDriver.java 91.63% 🟢

@jogrogan
jogrogan enabled auto-merge (squash) October 8, 2026 22:27

@srnand srnand left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@jogrogan
jogrogan merged commit 05e5dd6 into main Oct 8, 2026
1 check passed
@jogrogan
jogrogan deleted the jogrogan/pipelineLength branch October 8, 2026 22:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants