Skip to content

build: add reproducible lookahead auditing - #2829

Draft
fudianchn wants to merge 3 commits into
JSQLParser:masterfrom
fudianchn:perf/lookahead-audit
Draft

fudianchn wants to merge 3 commits into
JSQLParser:masterfrom
fudianchn:perf/lookahead-audit

Conversation

@fudianchn

@fudianchn fudianchn commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

AI disclosure: this change was prepared with AI coding agents, reviewed and revised line by line by me.

What

Add reproducible lookahead auditing, context-bound review evidence and a CI report, with one DBMSDestination cleanup as a worked case.

Depends on #2827 at 68c035bc058d5236ec8d30ca7909816a0a56e091. Its two commits remain unchanged in this branch and appear in the current diff against master. This phase adds only 2700c0546b4347b55aa40e1dc5621cf09cbd6f9b; review the incremental diff.

Why

Reuse evidence while relevant inputs remain unchanged, and flag it for review when they change. This reduces repeated review without silently retaining stale conclusions.

How

  1. Run ./gradlew lookaheadAudit -PlookaheadBase=<commit> to calculate fresh JavaCC FIRST/nullable/FOLLOW facts, structural anchors and transitive production dependencies for both versions.
  2. Bind investigated decisions to grammar context, production Java/resources/build inputs, analyzer sources and JavaCC JARs. Report changed decisions, dependency roots and reusable or invalidated evidence.
  3. Upload the report in CI. Historical findings remain informational; -PlookaheadFailOnFindings=true enables a strict gate. The analyzer uses existing test dependencies and stays outside the library JAR.

Root cause

An unchanged local FIRST set can hide changed callees or FOLLOW context. Java actions, predicates, recovery code, unknown continuations and ambiguous anchors require conservative review. The report does not prove general semantic equivalence or automatically rewrite grammar.

Testing

  • Twelve analyzer tests and thirteen cleanup cases cover evidence invalidation, stable anchors, conservative handling, AST fields, EOF consumption, malformed neighbors and EXASOL round trips. The real entry reports 1,256 unchanged sites without new findings; a watched Java edit invalidates evidence, and exact restoration reuses it without changing the record. All 43 ambiguous groups remain unbound.

  • Full Gradle checks: 10,429 discovered, 10,404 passed, 25 skipped. Full Maven verify: 10,411 discovered, 10,386 passed, 25 skipped. Both have zero failures/errors; no Maven gates were skipped. Maven excludes HowToUseSample, whose eighteen cases passed under Gradle.

  • Project JMH through ssh dev compared perf: skip redundant lookahead simulations in expression dispatch #2827's head with 626548d: unchanged 54-statement corpus, four interleaved windows, 120 samples. Latency changes were +0.27% and -0.97%, with overlapping score confidence intervals and no clear speedup. Deparsed SQL, semantic-field projections and serialized AST bytes matched. This tooling refactor preserves all 996 library class files byte for byte; JMH was not rerun for 2700c05.

  • The three targeted PMD 6.55 rules reproduce all thirteen original Codacy findings and pass after refactoring; maximum method NPath is 128 (threshold 200). Expanded PMD 7.17 checks and Codacy on 2700c05 also pass.

Behavior notes

The worked cleanup removes one redundant LOOKAHEAD(3) before TABLE, while retaining ordinary/cloud connection disambiguation. EXA AT conn TABLE remains invalid, but the current token at failure moves from conn to TABLE; the test checks this boundary.

The report step ran successfully on 2700c05 in GitHub Actions and uploaded lookahead-review. The stated tests and corpus do not establish equivalence for every SQL input or runtime configuration.

Verification of the original issue

No issue is linked. This follows the maintainer's request for a systematic cleanup approach, with #2827's head as the second-phase review and validation baseline.

…expression lists

The main branch of Condition/SQLCondition is the last alternative of its
choice: entering it and letting the productions reject is equivalent to the
deep jj_2 phase simulation that ran for every plain condition first, so gate
it with a cheap !interrupted check instead. The named-parameter alternatives
of ComplexExpressionList only ever match a single-token name directly
followed by their assignment token, so a first check on the second token is
equivalent to their 2-token lookahead scan.

parseSQLStatements: 21.3-21.4 vs 24.6-24.8 ms/op against master in 3
interleaved JMH pairs (-13% to -14%, error intervals not overlapping);
behavior verified identical by 26 EXISTS/named-parameter probes, the full
10,344-test suite (same 35 pre-existing environment failures as master) and
a byte-identical AST dump of performance.sql.

Signed-off-by: 付典 <fudianchn@gmail.com>
Use exact token predicates for PRIOR, OVERLAPS, time keywords, ALL and boolean literals while retaining interruption checks. Add 52 boundary cases for expression dispatch, named arguments, malformed input and cancellation.

Signed-off-by: 付典 <fudianchn@gmail.com>
Report fresh JavaCC decision contexts and transitive dependencies through
an existing-dependency Gradle entry and CI artifact. Bind investigated
cases to watched inputs and retain unknown effects conservatively.

Separate lexical facts, expansion traversal, evidence loading and report
comparison. Keep audit classes outside the library JAR.

Remove the redundant DBMSDestination probe with regression coverage for
ASTs and consumption boundaries. Missing TABLE names now fail after TABLE.

Signed-off-by: 付典 <fudianchn@gmail.com>
@fudianchn
fudianchn force-pushed the perf/lookahead-audit branch from 626548d to 2700c05 Compare October 16, 2026 08:24

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant