Repository navigation
chore(vendor): vendor the VB6/VBA tree-sitter grammar - #2969
Merged
Merged
Conversation
Vendors harumiWeb/tree-sitter-vba (MIT, (c) 2026 harumiWeb) at 83c5c3c3c721 (tag v0.14.5, upstream HEAD), its vba dialect, as a self-maintained fork in tools/tree-sitter-vba/. This is the precondition for VB6/VBA extraction (#2564, ported after #1179); it registers no language yet. Why generated here: upstream does not track its generated parser (its ADR 0002), so the fork carries upstream's grammar source in upstream's layout with no local patch, and parser.c is generated in this repository with tree-sitter-cli 0.25.10 (ABI 15), the generator every fork here uses. Upstream's own copy is generated by 0.26.9. Why this pin: v0.13.0 (the pin #2564 carried) exposes the same procedures but misreads statements without an ERROR node. On the three corpora below it yields 1,276 expression_statement nodes where v0.14.5 yields 56 (`Err.Raise a, b, c` after a label loses its callee), and it leaves an ERROR at `Case obj Is Nothing`. Why the vba dialect rather than upstream's vb6 one: measured with the CLI on kellyethridge/VBCorLib @ 100acbbae0c0 (655 VB6 files), Kr00l/VBCCR @ 4b86c5c2c075 (161 VB6 files, 70 .ctl) and upstream's 481 VBA examples, procedures outside ERROR nodes are 14,084/14,084, 12,385/1,552 and 8,104/6,573 (vba/vb6). vb6 rejects the PtrSafe/LongPtr forms dual-target VB6 keeps inside #If VBA7 blocks and then loses whole files; vba never sees fewer procedures in any file. vb6's parser.c is also 75.8 MB against 48.3 MB. Provenance by hash: common/define-grammar.js is upstream's blob 5e1f7d7; vba/grammar.js minus its 40-line provenance header is upstream's blob 9247dbbfe166. Two independent generations are byte-identical, and identical to a generation from upstream's own layout and tree-sitter.json. The generated node-types.json is upstream's checked-in file byte-identical (500e635), and the parse-table counts equal upstream's own generated parser. Review: 0 non-ASCII bytes, 0 unmatched characters on full tokenization, one comment (the generator stamp), no string literal with a space, no external scanner, no call outside the tree-sitter macros. No upstream script, package hook or binary was run or vendored. - internal/cbm/vendored/grammars/vba/: the generated parser.c, the tree_sitter/ headers (byte-identical to copies already vendored) and upstream's MIT LICENSE byte-identical. - internal/cbm/grammar_vba.c: compiles and links tree_sitter_vba. No CBMLanguage, extension or alias routes to it yet; extraction adds them under the vb6/vba names (vb and visualbasic mean VB.NET). - MANIFEST.md (fork note with the measurements and known gaps, forks table, 164 grammars, ABI recount), THIRD_PARTY.md (fork list and a vba section), scripts/vendored-checksums.txt, and the FORKS entry in scripts/audit-license-provenance.py. - README.md and the Chocolatey nuspec say 164 languages: the language-count contract defines the count as the vendored grammar directories. - tests/test_parse_coverage.c: a dual-target VB6 class module (IDE class header, Declare in both #If VBA7 branches, Property Get/Let/Set, Implements, events, a label, bare calls) parses with no ERROR node and every bare call keeps its callee; a CRLF VB6 form parses clean; broken source is still flagged. The class case fails on v0.13.0 and on the vb6 dialect. Refs #721, #2564 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings in #2957 (sql became a self-maintained fork), #2959 and #2962. Both sides added a self-maintained grammar fork, so every conflict keeps both additions: - MANIFEST.md summary: 164 grammars = 139 vendored-from-upstream + 20 self-maintained (sql moved over on main, vba added here) + 5 registry-disagreement; the ABI tally stays 9/78/77 (sql stayed ABI 15). The notes and the forks table carry both the sql and the vba rows. - THIRD_PARTY.md: thirteen further self-maintained forks including sql and vba; main's sql paragraph, then this branch's tree-sitter-vba section. - tests/test_parse_coverage.c: both test blocks and both sets of RUN_TEST registrations. - scripts/vendored-checksums.txt: regenerated with security-vendored.sh --update; against main only the MANIFEST.md line and the five vba files differ. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings in #2974 (incl. the jsonnet scanner patch) and #2970. One conflict: scripts/vendored-checksums.txt, where #2974 and the vba vendoring both rewrite the MANIFEST.md line. Resolved by taking main's file and regenerating it with scripts/security-vendored.sh --update; against main it differs only in the MANIFEST.md line and the five vba lines, and the integrity verify passes. THIRD_PARTY.md, MANIFEST.md and tests/test_parse_coverage.c merged cleanly; each equals main plus this branch's own change (identical patch-ids), and test_parse_coverage.c has 76 TEST and 76 RUN_TEST entries (71 at the merge base, +2 from main, +3 from this branch). The MANIFEST summary (164 grammars) and ABI line (9/78/77) match a recount from the tree. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
DeusData
added a commit
to ankitsusheel/codebase-memory-mcp
that referenced
this pull request
Nov 14, 2026
Brings the branch up to main 08ab4a0 (DeusData#2966, DeusData#2969, DeusData#2970, DeusData#2977, DeusData#2978, DeusData#2981). scripts/vendored-checksums.txt: main (DeusData#2969) vendored the VB6/VBA grammar while this branch edits MANIFEST.md. Regenerated with scripts/security-vendored.sh --update (1101 files); only the MANIFEST.md line differs from main. Everything else merged without conflict and was checked by hand: - MANIFEST.md keeps main's vba, sql and jsonnet text; only this branch's vbnet registration sentence and custom-handling row differ. - The language count stays 164 (vendored grammar directories); the language-count contract passes. VB.NET adds a language name to the README list, not a grammar. - DeusData#2977's receiver gate is an allowlist (gate_scope); VB.NET is not on it, so it stays GATE_OFF, and receiver_chain_admits is unchanged. - The VB.NET case-insensitive retry still hooks cbm_registry_resolve_lang, cbm_registry_resolve_ref and resolve_call_lang; the Field-call USAGE policy and the enum-member gate are intact. - DeusData#2981's linear-growth rewrites in extract_defs.c touch no VB.NET function; find_class_body still decides VB.NET first. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
DeusData
added a commit
that referenced
this pull request
Nov 14, 2026
Main brought in #2969 (vendored VB6/VBA grammar), #2981 (linear-growth rewrites in extraction and eight LSP passes, a node for C# operators, C increments recorded as writes), #2977 (receiver gate in the registry), #2978 (possible override targets in trace_path), #2966 (CONTRIBUTING test reference) and #2985 (freebsd-vm action bump). Two files conflicted: - internal/cbm/vendored/grammars/MANIFEST.md: the summary header. Took main's counts (164 grammars, 20 first-party, ABI 9x/78x/77x), which a recount of the merged tree confirms (164 grammar directories, LANGUAGE_VERSION 13/14/15 = 9/78/77). Kept this branch's 2026-11-13 ObjectScript recount line above main's 2026-11-12 vba line; main had folded the 2026-11-11 sql line into its vba line, so that line goes. Main's vba prose and table rows and this branch's ObjectScript v1.11.0 rows are both present. - scripts/vendored-checksums.txt: regenerated with scripts/security-vendored.sh --update. Only the MANIFEST.md hash moved; the result differs from this branch only by main's vba entries and from main only by the ObjectScript entries. Every other file merged cleanly. For each file both sides changed (THIRD_PARTY.md, extract_calls.c, extract_defs.c, extract_usages.c, helpers.c, lang_specs.c, tests/test_parse_coverage.c) the merged file differs from each parent by exactly the other parent's change. #2981 rewrote objectscript_routine_preceding_tag onto a memoized sibling search with the same probe; this branch does not touch that function. ObjectScript stays outside #2977's receiver gate allowlist. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this vendors
A Visual Basic for Applications / Visual Basic 6 tree-sitter grammar, as a
self-maintained fork in
tools/tree-sitter-vba/, generated intointernal/cbm/vendored/grammars/vba/:@
83c5c3c3c7210468fb6745b585b5fb6532aa2376(tagv0.14.5, upstreamHEAD), its
vbadialect. MIT, (c) 2026 harumiWeb.ADR 0002), so the fork carries upstream's grammar source in upstream's own
layout and the parser is generated here with tree-sitter-cli 0.25.10
(ABI 15), the generator every fork in this repository uses.
This is the precondition for VB6/VBA extraction (#2564, which we port onto
this grammar after #1179 merges). It registers no language.
Why this pin (v0.14.5, not v0.13.0)
#2564 carried v0.13.0 (
63b2f8d0). Both pins were regenerated with the sameCLI and run over the same three corpora (below):
expression_statementnodesThe procedure count is the same, but v0.13.0 misreads statements without
leaving an ERROR node: a bare call such as
Err.Raise a, b, cafter a labelbecomes an expression statement and loses its callee. That is a wrong CALLS
edge nobody would see in a parse-error count. Upstream fixed these in
v0.14.0 to v0.14.3, and fixed
Case obj Is Nothing(an ERROR in v0.13.0) inv0.14.1.
Why the vba dialect (not upstream's vb6 one)
Since v0.14.0 upstream generates a second,
vb6dialect from the same core.Measured with the tree-sitter CLI on two pinned MIT VB6 code bases and
upstream's VBA examples (vba / vb6):
100acbbae0c0(pure VB6)4b86c5c2c075(dual-target VB6, 70.ctl)examples/(VBA)Real VB6 libraries keep a VBA7 branch next to the VB6 one
(
#If VBA7 Then ... Declare PtrSafe ... LongPtr ... #Else ... #End If).A tree-sitter grammar parses both branches, and the vb6 dialect rejects the
VBA7 forms; its error recovery then covers 83 of the 161 VBCCR files almost
entirely. In no file of any corpus does the vba dialect see fewer procedures
than vb6. The vb6 dialect is cleaner only on VB6 component reference lines
(see Known gaps), and its
parser.cis 75.8 MB, above the 50 MB line.Provenance, by hash
tools/tree-sitter-vba/common/define-grammar.jsis upstream's filebyte-identical (git blob
5e1f7d758ce1).tools/tree-sitter-vba/vba/grammar.jswithout its 40-line provenanceheader is upstream's
vba/grammar.js(blob9247dbbfe166).parser.cis generated with tree-sitter-cli 0.25.10 using upstream's owncommand (
generate --abi 15 --output vba/src vba/grammar.js). Twoindependent generations are byte-identical (SHA-256
bd2a7ecde688...), andidentical to a generation from upstream's own layout and
tree-sitter.json.node-types.jsonis upstream's checked-invba/src/node-types.jsonbyte-identical (blob
500e63561011), and the parse-table counts (16,029states, 417 symbols, 180 tokens, 65 fields, 362 productions) equal those of
upstream's 0.26.9-generated
bindings/go/parser.c.LICENSEis upstream's MIT text byte-identical (blobefec104b6dc4).script, workflow, Wasm/playground tooling, agent instruction file or test
fixture. Generation ran with network access denied and writes confined
to a scratch directory.
Generated-parser review
48,349,333 bytes, 1,119,783 lines, ABI 15. 0 non-ASCII bytes and 0 control
bytes other than LF; full tokenization leaves 0 unmatched characters; one
comment (the generator stamp); 490 string literals, none containing a space;
729 hex literals, every one a lexer code-point comparison against
lookahead; no call outside the tree-sitter macros. No external scanner(
EXTERNAL_TOKEN_COUNT 0, no.external_scannerin the language struct).tree_sitter/*.hare byte-identical to copies already vendored here.License
MIT, (c) 2026 harumiWeb. The shared grammar core's vb6 layer came upstream
through harumiWeb/tree-sitter-vba#56 under the same license. Upstream's
THIRD_PARTY_LICENSE.mdcovers only itsexamples/third_partytestfixtures (16 MIT projects and one CC BY 3.0); none of them is carried here.
No copyleft code is involved.
Sizes
parser.cis 48,349,333 bytes, under the 50 MB line, so no LFS questionarises. The copy in
tools/tree-sitter-vba/vba/src/and the vendored copyare the same git blob.
Known gaps (for the extraction PR)
On real VB6 the vba dialect leaves three localized error shapes:
.frm/.ctlcomponent reference line,Object = "{GUID}#1.2#0"; "COMDLG32.OCX": a 16-character ERROR on thatline;
ToolboxBitmap = "X.ctx":003B: a 1-character ERROR;#If VBA7/#Elsewith anAttribute Name.VB_Descriptionline inside each branch: error recoveryruns to the end of the file (2 of the 161 VBCCR files).
All three are recorded in
internal/cbm/vendored/grammars/MANIFEST.md.What it enables next
VB6/VBA extraction through #2564, which we port onto this grammar with
maintainer commits after #1179 merges (Tokarzewski's commits stay). This PR
registers no language:
internal/cbm/grammar_vba.ccompiles and linkstree_sitter_vba, but noCBMLanguage, file extension or name alias routesto it yet. The extraction PR adds them under the
vb6andvbanames;vbandvisualbasicmean VB.NET.The README and Chocolatey counts move from 163 to 164 languages, because
the language-count contract counts vendored grammar directories.
Verification
tests/test_parse_coverage.c: a dual-target VB6 class module (the IDEclass header, a Declare in both
#If VBA7branches, theLongPtrshimenum, Implements, an Event, Property Get/Let/Set, a label, bare calls)
parses through
tree_sitter_vba()with no ERROR node, every procedure isa declaration node, and every bare call keeps its callee; a CRLF VB6 form
parses clean; broken source still reports an ERROR. With the v0.13.0
parser swapped in, the class case fails (65 passed, 1 failed); the vb6
dialect also errors on it (at
PtrSafe).scripts/security-vendored.sh(1,101 files, no dangerous calls),tests/test_vendored_integrity_contract.sh,tests/test_language_count_contract.sh(164) andscripts/audit-grammar-security.shpass. The notice bundle carries thevba
LICENSE, and the SBOM generator counts 164 grammars. The licenseprovenance audit (which routes
vbato upstream through itsFORKSmap)runs in the hosted security workflow.
lang_contract, registry, repro_language_registry, grammar_regression,
grammar_labels and extraction (972 passed), with no sanitizer report.
Refs #721, #2564