Skip to content

Docs: Assert exact parser errors in some negative tests - #2713

Open
s5dsn-eqee wants to merge 1 commit into
apache:mainfrom
s5dsn-eqee:agents-exact-error-tests
Open

s5dsn-eqee wants to merge 1 commit into
apache:mainfrom
s5dsn-eqee:agents-exact-error-tests

Conversation

@s5dsn-eqee

Copy link
Copy Markdown
Contributor

Docs: Assert exact parser errors in some negative tests

Motivation: when a negative test asserts the exact error, a change to the
parser branch that rejects the input shows up as a failing test. That helps
a developer or an agent see which branches a PR changed, straight from the
test diff. A bare is_err() passes no matter which branch rejected the input.

The usual objection is maintenance: when error messages change, updating
many exact assertions by hand is slow and tedious. With an LLM, updating
them is now a few minutes of work, so the extra signal is worth the cost.

@codecov-commenter

codecov-commenter commented Oct 24, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.97%. Comparing base (7ab6bfa) to head (6a04207).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2713      +/-   ##
==========================================
+ Coverage   81.95%   81.97%   +0.02%     
==========================================
  Files          42       42              
  Lines       34702    35011     +309     
  Branches    34702    35011     +309     
==========================================
+ Hits        28440    28702     +262     
- Misses       2735     2744       +9     
- Partials     3527     3565      +38     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread AGENTS.md Outdated
- New unit tests should be added to the `tests` module in the corresponding dialect file (e.g., `tests/sqlparser_redshift.rs` for Redshift), and should be placed at the end of the file.
- If the new functionality is gated using a dialect function, and the SQL is likely relevant in most dialects, tests should be placed under `tests/sqlparser_common.rs`.
- Cover both positive and negative cases: valid input parses and round-trips, invalid input fails with the expected error.
- Cover both positive and negative cases: valid input parses and round-trips, invalid input fails with the expected error. For some of them, assert the exact error text on a full statement, not just `is_err()`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought this was already blocked by the clippy lint now?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not fully, as far as I can see. The lint only fires on a stored result like
assert!(res.is_err()). A direct call still passes clippy, for example:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@s5dsn-eqee
s5dsn-eqee force-pushed the agents-exact-error-tests branch from 9813534 to 6a04207 Compare October 24, 2026 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants