You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 4ba800d
Browse filesBrowse the repository at this point in the historyBrowse files
refactor: restructure token handling with NonReservedWord() sentinels
Replace the manual updateKeywords task and hardcoded keyword whitelist
with a NonReservedWord() BNF production bracketed by MIN/MAX sentinel
tokens. Non-reserved keywords are now determined by O(1) range check
in isIdentifierAhead() instead of enumerating them in
RelObjectNameWithoutValue().
- Refactor ParserKeywordsUtils: derive keywords dynamically from
CCJSqlParserConstants + grammar file, remove hardcoded
ALL_RESERVED_KEYWORDS array and RESTRICTED_* flags
- Consolidate ConditionalKeywordsTest into KeywordsTest
- Simplify updateKeywords Gradle task (generates RST doc only)
- Update contribution.rst and usage.rst documentation
- allow nested comments
Fixes#1175
Signed-off-by: Andreas Reichel <andreas@manticore-projects.com>
Signed-off-by: manticore-projects <andreas@manticore-projects.com>
Copy file name to clipboardExpand all lines: src/site/sphinx/contribution.rst
+11-33Lines changed: 11 additions & 33 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -97,49 +97,27 @@ The JSQLParser is generated by ``JavaCC`` based on the provided Grammar. The Gra
97
97
Manage Reserved Keywords
98
98
------------------------------
99
99
100
-
Since JSQLParser is built by JavaCC from a Token based Grammar, ``Reserved Keywords`` need a special treatment. All Tokens of the Grammar would become ``Reserved Keywords`` -- unless explicitly allowed and white-listened.
100
+
Since JSQLParser is built by JavaCC from a Token based Grammar, ``Reserved Keywords`` need a special treatment. All Tokens of the Grammar would become ``Reserved Keywords`` -- unless explicitly allowed as identifiers.
101
+
102
+
The Grammar uses a ``NonReservedWord()`` BNF production with inline token declarations, bracketed by ``MIN_NON_RESERVED_WORD`` and ``MAX_NON_RESERVED_WORD`` sentinel tokens. JavaCC assigns consecutive token kind values to the inline declarations, which enables an efficient O(1) range check in ``isIdentifierAhead()`` to determine whether a token can be used as an unquoted identifier.
101
103
102
104
.. code-block:: sql
103
-
:caption: White-list Keyword example
105
+
:caption: Non-reserved keyword example
104
106
105
-
-- <K_OVERLAPS:"OVERLAPS"> is a Token, recently defined in the Grammar
106
-
-- Although it is not restricted by the SQL Standard and could be used for Column, Table and Alias names
107
-
-- Explicitly white-listing OVERLAPS by adding it to the RelObjectNameWithoutValue() Production will allow for parsing the following statement
107
+
-- <K_OVERLAPS:"OVERLAPS"> is defined as a non-reserved keyword inside the NonReservedWord() production
108
+
-- It can be used for Column, Table and Alias names without quoting
108
109
109
110
SELECT Overlaps( overlaps ) AS overlaps
110
111
FROM overlaps.overlaps overlaps
111
112
WHERE overlaps = 'overlaps'
112
113
AND (CURRENT_TIME, INTERVAL '1' HOUR) OVERLAPS (CURRENT_TIME, INTERVAL -'1' HOUR)
113
114
;
114
115
115
-
So we will need to define and white-list any Keywords which may be allowed for Object Names (such as `Schema`, `Table`, `Column`, `Function`, `Alias`). This White-List must be updated whenever the Tokens of the Grammar change (e. |_| g. when adding a new Token or Production).
116
-
117
-
There is a task ``updateKeywords`` for Gradle and Maven, which will:
118
-
119
-
1) Parse the Grammar in order to find all Token definitions
120
-
2) Read the list of explicitly ``Reserved Keywords`` from ``net/sf/jsqlparser/parser/ParserKeywordsUtils.java``
121
-
3) Derive the list of ``White-Listed Keywords`` as difference between ``All Tokens`` and ``Reserved Keywords``
122
-
4) Modifies the Grammar Productions ``RelObjectNameWithoutValue...`` adding all Tokens according to ``White-Listed Keywords``
123
-
5) Run two special Unit Tests to verify parsing of all ``White-Listed Keywords`` (as `Schema`, `Table`, `Column`, `Function` or `Alias`)
124
-
6) Update the web page about the Reserved Keywords
125
-
126
-
127
-
.. tab:: Gradle
128
-
129
-
.. code-block:: shell
130
-
:caption: Gradle `updateKeywords` Task
131
-
132
-
gradle updateKeywords
133
-
134
-
.. tab:: Maven
135
-
136
-
.. code-block:: shell
137
-
:caption: Maven `updateKeywords` Task
138
-
139
-
mvn exec:java
140
-
116
+
When adding a new keyword token to the Grammar:
141
117
142
-
Without this Gradle Task, any new Token or Production will become a ``Reserved Keyword`` automatically and can't be used for Object Names without quoting.
118
+
1) If the keyword should be usable as an unquoted identifier (the common case), add its inline token declaration to the ``NonReservedWord()`` production. It will automatically be placed between the sentinel tokens and recognised by the range check.
119
+
2) If the keyword must be reserved (e. |_| g. core SQL syntax like ``SELECT``, ``FROM``, ``WHERE``), add it to the ``Reserved SQL Keywords`` TOKEN block **after** the ``MAX_NON_RESERVED_WORD`` sentinel.
120
+
3) Verify that existing tests pass and that the keyword can be used as a ``Schema``, ``Table``, ``Column``, ``Function`` or ``Alias`` name where expected.
143
121
144
122
145
123
Commit a Pull Request
@@ -196,4 +174,4 @@ Please consider using `Conventional Commits` and structure your commit message a
196
174
* - **revert**
197
175
- reverts one or many previous commits
198
176
199
-
Please visit `Better Programming <https://betterprogramming.pub/write-better-git-commit-messages-to-increase-your-productivity-89fa773e8375>`_ for more information and guidance.
177
+
Please visit `Better Programming <https://betterprogramming.pub/write-better-git-commit-messages-to-increase-your-productivity-89fa773e8375>`_ for more information and guidance.
0 commit comments