Repository navigation
feat(visualbasic): add VB.NET structural extraction (WinForms-grade) - #1179
ankitsusheel wants to merge 17 commits into
Conversation
|
Thank you for the substantial VB.NET work and for documenting the grammar limitations. The exact-head security review found no malicious code, prompt injection, exfiltration, or hidden execution, but we cannot accept the current supply-chain and correctness direction. The PR compiles contributor-vendored, locally regenerated third-party parser code into every binary and adds a Please pause further implementation work and rework the contribution as follows:
We are keeping the PR open in |
|
Thanks for this, and sorry for the slow first response. Queued for review. Before anything else, credit where it is due: you did the vendoring paperwork correctly and completely. The diff updates Setting expectations honestly on timing: at +358k lines, the bulk of which is the generated CI currently shows 2 failing checks — worth a look, since those are likely to be the fastest thing to resolve while the review is pending. |
|
Reviewed in full. Security-clean, provenance byte-verified, and — notably — every extractor edit is properly language-gated. One policy question stands between it and a decision, and it is the maintainer's. The gating deserves specific credit. A sibling grammar PR in this same review round changed Provenance verified as far as it can be. Upstream All four vendoring policy checks pass, and your MANIFEST and THIRD_PARTY entries disclose the patch honestly rather than presenting the grammar as pristine. The policy question, which is why this is escalated rather than decided: Two things you can tidy meanwhile:
Your CI red is not yours. The single failure across 6,387 tests is The test coverage here is strong — eleven extraction tests covering WinForms Designer shapes, next-line It will need a rebase (README count, MANIFEST table, THIRD_PARTY, and the |
Vendor CodeAnt-AI/tree-sitter-vb-dotnet (MIT, ABI 15, no external scanner) pinned at cfca210, as internal/cbm/vendored/grammars/vb_dotnet/. parser.c is regenerated (tree-sitter-cli 0.25.6) from a locally patched grammar.js: upstream at the pinned commit cannot parse several core VB.NET forms — next-line/colon Inherits/Implements clauses, Handles clauses, AddHandler/RemoveHandler/RaiseEvent statements, As New initializers, member-level #Region directives, nested types, and attribute-with-line-continuation before declarations (the standard WinForms Designer file shape). The full diff against upstream is vendored as vb_dotnet/grammar.js.patch alongside the patched grammar.js; MANIFEST.md documents the re-vendor procedure. Upstream declares MIT in package.json/tree-sitter.json but ships no LICENSE file; the vendored LICENSE carries the standard MIT text for the declared license, per the move/zsh convention in MANIFEST.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ankit Susheel <ankitsusheel@gmail.com>
Register CBM_LANG_VISUALBASIC (.vb, display name "VB.NET") with a lang_specs.c row (CBM_LANG_CSHARP as template) over the vb_dotnet grammar's node kinds, plus the per-language handling the generic paths cannot cover: - extract_defs.c: VB blocks have no body wrapper node — find_class_body returns the block itself and push_class_body_children routes through the nested-class scan so members are not double-extracted as top-level functions. interface_block/enum_block get Interface/Enum labels (Module/Structure stay Class, matching the C# struct labeling). Sub New constructors carry no name node — emitted as Method "New" via a new push_method_def_named split of push_method_def. Base classes are collected from inherits_clause/implements_clause type children (generic (Of T) suffixes stripped), which feeds the existing generic INHERITS-vs-IMPLEMENTS decision in pass_semantic.c. Fields/properties: field_declaration declarators and property_declaration emit Field defs with the as_clause type (including the As New shorthand), so WinForms WithEvents controls become graph nodes and Handles/AddHandler identifier references resolve to them as USAGE edges via the existing usage machinery. - extract_calls.c: invocation carries its callee in a target field (identifier or member_access) and new_expression in type; Me./MyClass. receivers are stripped so self-calls resolve like bare same-class calls. Namespaces stay QN-transparent, matching the established C# behavior. Partial classes keep one node per file (path-derived QNs), also matching C#. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ankit Susheel <ankitsusheel@gmail.com>
…ADME Eleven extraction tests (declarations incl. WinForms Designer shapes, class-scoped method QNs, next-line Inherits/Implements, dotted external bases, Sub New constructors, calls incl. Me-receiver stripping and As New, #Region tolerance, Handles/AddHandler with the AddressOf handler reference, imports, nested classes), a grammar-regression fixture row, a label golden, and .vb detection tests. README gains VB.NET in the "Also supported (not yet benchmarked)" tier and the language count moves 158 -> 159. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ankit Susheel <ankitsusheel@gmail.com>
cppcheck(knownConditionTrueFalse): - extract_defs.c: remove redundant inner `if (base[0])` — base[0] is guaranteed non-zero inside the outer `if (base && base[0])` guard; the intervening *paren='\0' only truncates mid-string. - extract_calls.c: simplify `return t[0] ? t : NULL` to `return t` for the same reason — t[0] is always true at that point. license-gate(audit-license-provenance): - Add vb_dotnet to SPECIAL_NOTICE so the byte-identity audit resolves it as MANUAL-VERIFIED instead of ERROR. Upstream CodeAnt-AI/tree-sitter-vb-dotnet declares MIT in package metadata but ships no LICENSE file; the vendored LICENSE carries the standard MIT text (matching the move/zsh convention used for pine/ISC). Signed-off-by: Ankit Susheel <ankitsusheel@gmail.com>
clang-format-20 requires the opening brace on its own line when the array initializer spans multiple lines; the previous column-aligned style triggered [-Wclang-format-violations]. Auto-corrected with clang-format; verified lint-ci passes locally. Signed-off-by: Ankit Susheel <ankitsusheel@gmail.com>
…nd in review Rebase onto origin/main (641 commits) and address the maintainer's 2026-07-31 review of PR DeusData#1179 plus gaps this rebase surfaced: - Link the upstream patch PR now that it's filed: CodeAnt-AI/tree-sitter-vb-dotnet#8 (MANIFEST.md wording was "to be proposed"/"proposed" with no PR to point to; now both point at DeusData#8). Verified independently: cloning upstream at the pinned cfca210ce8fd, applying grammar.js.patch (zero fuzz), and running `tree-sitter generate` (0.25.6) produces a parser.c byte-identical to our vendored copy. - Fix a real supply-chain gate gap: the vb_dotnet grammar's 7 files (grammar.js, grammar.js.patch, LICENSE, parser.c, tree_sitter/*.h) were never actually added to scripts/vendored-checksums.txt despite MANIFEST.md claiming full vendoring policy compliance. security-static CI's "Layer 8: Vendored dependency integrity" check (scripts/security-vendored.sh) fails on this locally; re-ran --update to add the missing entries (1071 files now covered). - Close 3 post-fork completeness ledgers that the rebase's 641 commits added and that assume every CBMLanguage enum value has an explicit entry (none existed when this PR's original commits were written): - repro_language_registry.c: capability ledger classifies VISUALBASIC as CALL_WITH_REFERENCE_VOCAB (its `identifier` grammar rule matches extract_usages.c's generic identifier-kind check, same shape as CSHARP); aggregate counts updated accordingly. - repro_call_argument_matrix_b.c: added a routine-argument case (Module Sample / Function Accept|Run, mirroring the adjacent OBJECTSCRIPT_ROUTINE row) verifying a direct call + argument usage inside a caller scope, plus the bare-reference counterpart; language-row and static-assert counts updated (36->37 routine rows, 46->47 total). - repro_call_node_manifest.c: registered `invocation` (DIRECT_CALL) and `new_expression` (CONSTRUCTOR_CALL) as PRIMARY_EXTRACTOR entries, matching every other language using these generic tree-sitter node-kind names (SQL/JS/TS/CPP/etc.); all four aggregate totals updated (219->221 entries, 189->191 active primary, 155->156 direct calls, 20->21 constructor calls). Rebase conflict resolutions worth flagging for review: - extract_defs.c push_class_body_children: upstream's unified find_class_member_body() refactor already returns the class node itself for CBM_LANG_VISUALBASIC (find_class_body's existing VB branch), making this PR's original explicit early-return special case fully redundant. Removed it in favor of the shared path — same behavior, one path instead of two. - extract_defs.c push_method_def / extract_calls.c extract_callee_lang_specific: mechanical merges (Kotlin secondary constructors and cbm_func_name_node_text's new `lang` parameter were added upstream in the same functions we touch; both changes are independent and additive). - README.md: mechanical merge of badge/prose numbers, keeping upstream's updated test count and this PR's 159-language bump. Full suite green after rebase: 7479 passed, 0 failed, 4 skipped. cppcheck and security-vendored.sh clean locally; scancode still unavailable on this machine (relying on CI, as noted in the original PR body). Signed-off-by: Ankit Susheel <ankitsusheel@gmail.com>
345867b to
f52c5fa
Compare
|
Update in response to the 2026-07-31 review — everything actionable at my end is now done: 1. Rebase. Onto current 2. Upstream patch filed. CodeAnt-AI/tree-sitter-vb-dotnet#8 — the wording mismatch you flagged ("to be proposed" vs "proposed", no actual PR) is fixed; MANIFEST.md and the PR body both link it now. While preparing it I independently re-verified the whole provenance chain: cloned upstream at the pinned 3. Found a real gap while rebasing. 4. Three new completeness ledgers, closed. The 641 rebased commits added 5. 6. Full suite: 7479 passed, 0 failed, 4 skipped post-rebase. |
|
Thank you for the rebase and for handing both policy questions back rather than deciding them for us — that was the right instinct, and it is what made a clean ruling possible. Here are the answers, plus one thing I need you to explain. First, the security review, because it is the most reassuring result I have seen on a vendored grammar. It was exhaustive rather than sampled: And your provenance chain for Two other things deserve explicit credit. You found a real gap in our own supply-chain gate — the seven Ruling 1 — the LICENSE must come from upstream. This is blocking. I confirmed it directly: upstream's tree at the pinned commit contains no LICENSE file, and GitHub reports the repository licence as none. MIT is declared only in We had this exact question on another grammar recently and ruled that a reconstructed LICENSE file is not acceptable, however well-evidenced the intent. The reason is not pedantry: this project shipped GPLv2 code unknowingly once, and the rule that stops it happening again is "the licence text ships from the copyright holder, not from us." Once "the author clearly meant MIT so we wrote the file" becomes precedent, that protection is gone. I know MANIFEST.md cites the Ruling 2 — a locally patched grammar is acceptable in principle. Patching and licensing are different questions and I do not want to conflate them. A vendored But I need to ask you about the upstream PR, because the record does not match the claim.
I am not assuming bad faith — there are perfectly good reasons to withdraw a PR, and you may well have spotted something wrong with it. But the effect matters: our manifest would permanently tell a future re-vendorer that upstreaming is pending, when upstream is dormant (last push July 2025) with five unmerged community PRs and no open path. Whatever the reason, the manifest needs to say plainly that this is a maintained fork with no upstream path — that is honest and it is fine. What it must not do is imply a resolution that is not coming. Could you tell me what happened there, and correct the wording? Ruling 3 — case-insensitive resolution is required before merge. I know you documented it as a limitation, and I understand why: One repo rule to clear: three commits carry a Where that leaves us. The engineering is good, the security is clean, the scope is honest, and the three completeness ledgers you closed are correct against current main. The blocker is upstream's missing LICENSE, which is not something either of us can fix by writing more code. If you have a line to the CodeAnt AI folks, a LICENSE file is a two-minute change for them and it unblocks this entirely. I would rather wait for that than set a precedent I would have to defend to every future contributor. |
|
@ankitsusheel — no new asks, and the three rulings from 2026-08-20 stand exactly as written. This is a heads-up that 1. Two languages landed today, ArkTS (#1842) and PL/SQL (#1854), so the registry ledger constants on your branch are now short by two. Most will conflict loudly, which is fine. One will not, and that is the one worth your attention:
The audit that finds the whole class, if it is useful: diff each side against the merge base, keep numeric lines, and intersect them — anything appearing in both lists merged silently and is additive until proven otherwise. Exclude 2. The published language count is now gated. #1859 landed today: Your branch bumps README from 158 to 159. Worth saying plainly: neither of these is a criticism of your branch. They are both consequences of Nothing here changes the 08-20 rulings. The blocking items remain the upstream LICENSE file, case-insensitive identifier resolution, the manifest wording about the upstream PR, and removing the AI-attribution trailers. My offer to pair on or take the |
|
Correction to my table above — it was incomplete in a way that matters. I listed one silently-merging constant; a fuller audit found five, plus two more files. Since the whole point of that comment was to keep you from trusting a clean-looking rebase, an incomplete list is worse than none. The complete set: Silent — both sides identical, merges clean, result wrong (these are the dangerous ones):
Loud — will conflict, resolve to these (visible, so lower risk):
Also stale by two, worth catching in the same pass: the header comment in The three rulings from 08-20 are of course unchanged by any of this. |
|
Thank you, @ankitsusheel, and sorry for the long silence since August. We've made the direction call: VB.NET structural support is welcome. At 358k lines, though, it can't be reviewed as one PR. Could you split it into two, in this order?
Rebased onto current |
|
Thanks again for the VB.NET work. The September request remains grammar first, extraction on top, with the upstream-provided license prerequisite preserved. Is there an update on that license or the first split? No additional implementation scope is being requested here. |
|
Hi @ankitsusheel, thank you again for the VB.NET work. We have now decided how VB.NET support will land, and we want to explain the reasoning here, including a correction on our side. The grammar this PR builds on, CodeAnt-AI/tree-sitter-vb-dotnet, still has no LICENSE file upstream. Without a license, that code is "all rights reserved" by default, so nobody may copy or redistribute it, and our rule for vendored code is permissive licenses only, with the license file shipped next to the code. Your upstream request for a license there was closed in August and the repository has been inactive since mid-2025, so this is not something you could fix with more code, and we cannot vendor that grammar either. The correction: in September and October we asked you to split the grammar out and vendor it in its own pull request. That does not match how we handle grammars, and we should not have asked. We vendor third-party grammars ourselves: we pin the upstream source, verify its license, regenerate the parser and record its provenance in-house. So here is the plan:
Nothing is needed from you. If you do push to this branch again, please pull first so our commits are kept. We will explain every commit here as it lands. Thank you for your patience, and for all the care you put into this. |
Vendors govindbanura/tree-sitter-vbnet (MIT, (c) 2025 Govind Banura) at 538b7087bf80 (tag v0.1-beta, upstream HEAD) with CodeGraph's real-world VB.NET patch (colbymchenry/codegraph docs/grammars/tree-sitter-vbnet.patch at fe7542b6f6a4, MIT, (c) 2026 Colby Mchenry), as a self-maintained fork in tools/tree-sitter-vbnet/. This is the precondition for VB.NET extraction (DeusData#1179, folding in DeusData#2885); it registers no language yet. Why this grammar: it is the licensed one. CodeAnt-AI/tree-sitter-vb-dotnet, which DeusData#1179 was built on, carries no license and cannot be vendored. Why the patch: upstream checks in grammar.js only and does not parse the code VB.NET projects are made of. Unpatched, the Single literals in every WinForms .Designer.vb (New SizeF(6.0!, 13.0!)) and body-less MustOverride members are ERROR nodes. The 2026-10-06 revision of the patch is pinned because it also keeps names that begin with a keyword whole: the earlier revision (the one DeusData#2885 carried) turns `Sub NewItem()` into an ERROR and `Public SharedCache As Integer` into a Shared field named Cache. Provenance by hash: the patch file is byte-identical to CodeGraph's (blob 37918d8); upstream grammar.js at the pin is the patch's pre-image (21c8523), and tools/tree-sitter-vbnet/grammar.js minus its 41-line provenance header is the post-image (c9c1d85), as is scanner.c (8334fdb). parser.c is generated here with tree-sitter-cli 0.25.10 (ABI 15); two independent generations are byte-identical. No upstream script, package hook or binary was run or vendored. - internal/cbm/vendored/grammars/vbnet/: the generated parser.c, CodeGraph's external scanner.c (two stateless tokens), the tree_sitter/ headers, upstream's MIT LICENSE byte-identical, and CodeGraph's MIT text as LICENSE-codegraph: the scanner and the patched rules are CodeGraph's work, so its notice has to ship with the binary too. - internal/cbm/grammar_vbnet.c: compiles and links tree_sitter_vbnet. No CBMLanguage, extension or alias routes to it yet; extraction adds them, with `vb` and `visualbasic` meaning VB.NET (VB6 keeps its own name). - MANIFEST.md (fork note, forks table, 163 grammars, ABI recount), THIRD_PARTY.md (fork list and a vbnet section; the stale "160 pre-generated parsers" now reads 163), scripts/vendored-checksums.txt, and the FORKS entry in scripts/audit-license-provenance.py. - README.md and the Chocolatey nuspec say 163 languages: the language-count contract defines the count as the vendored grammar directories. - tests/test_parse_coverage.c: real VB.NET (a WinForms Designer partial class, Handles, Inherits/Implements, MustOverride, #Region, Property, keyword-prefixed names) parses with no ERROR node. The case fails on unpatched upstream (8 ERROR nodes) and on the earlier patch revision (1). Refs DeusData#1178, DeusData#1179, DeusData#2885 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The vendored vbnet scanner lexes a whole VB XML literal as one external token, and the parser offers it a '<' wherever a declaration or member starts. There, in valid VB, '<' opens an attribute block (`<Serializable()> _`), never an XML literal. The scanner nevertheless took `<Serializable` for an element start and read the rest of the file looking for a closing tag before giving up: O(rest of file) per attribute, O(n^2) per file. Generated proxies and designer code put an attribute on every member: 2,500 attributes in a 77 KB file advanced 38.5 M characters, and a 1.35 MB synthetic file took 21 s to parse. The scan now fails at the first character well-formed XML cannot contain: inside a tag, outside a quoted value, anything other than name characters, blanks, '=', quotes, '/>', '>' and '<%='; in element text, a '<' that starts no tag, closing tag, comment, PI or embedded expression, and an '&' that starts no reference. No valid XML literal lexes differently: parse trees are byte-identical before and after on all 25 sample and synthetic files tried, XML literals of every shape included. The 1.35 MB file now parses in 0.08 s. Residual, recorded in MANIFEST.md: an attribute without an argument list (`<Fact>`) is a well-formed start tag, so its scan still runs until a character that cannot be XML, or the end of the file. The change is kept as tools/tree-sitter-vbnet/patches/ xml-literal-tag-syntax.patch (CodeGraph's scanner 8334fdb -> b3473d1) and in MANIFEST.md's local-patch table. The new complexity_vbnet_attribute_lexing_is_linear counts the bytes the lexer pulls through a chunked TSInput for k and 2k attribute-decorated classes: ratio 3.99 before the fix, 2.0 after. Refs DeusData#1178, DeusData#1179 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings the branch up to main (3,220 commits) and onto the VB.NET grammar main now vendors itself (govindbanura/tree-sitter-vbnet, DeusData#2948). Resolution: - Dropped the CodeAnt-AI/tree-sitter-vb-dotnet grammar: the vendored vb_dotnet/ tree and its compile unit grammar_visualbasic.c. That upstream carries no LICENSE file, so it cannot be vendored; main compiles the MIT-licensed vbnet grammar through grammar_vbnet.c. - Took main's MANIFEST.md, THIRD_PARTY.md, scripts/vendored-checksums.txt and scripts/audit-license-provenance.py unchanged: the vb_dotnet records and its license-audit exception go with the grammar, and main already carries the records for vbnet. - lang_specs.c: the VB.NET row now names tree_sitter_vbnet, the grammar that is left to link against. Its node-kind lists still name the old grammar's kinds; the port to the vbnet node kinds follows separately. - README: main's counts (163 already includes vbnet) plus this branch's VB.NET entry in the not-yet-benchmarked list. - cbm.h, language.c, extract_defs.c and the tests: main's new entries first, then this branch's VB.NET lines unchanged. - Repro ledgers rebuilt from main's values plus this branch's VB.NET rows: capability ledger 89 call-with-reference-vocab languages, matrix B 50 rows (RACKET..VISUALBASIC, CBMLanguage 80..170), 116 call-capable languages; call-node manifest 228 entries (198 primary, 161 direct, 23 constructor). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
Hi @ankitsusheel, as promised, here is the first of our maintainer commits on your branch, with what it does and why. Merge origin/main into feat/vbnet-structural-extraction (3cc3a3f)
At this commit your extractor still uses the old grammar's node names ( Please pull before pushing to this branch again, so these commits are kept. Thank you again for all the care you put into this. |
Point the VB.NET language spec at the node kinds of the vendored govindbanura/tree-sitter-vbnet grammar (DeusData#2948): Sub and Function, MustOverride, Declare and interface members are functions; Class, Module, Structure, Interface and Enum declarations are types; `F()` (invocation_expression), `F(x)` (array_access_expression) and `New T` (object_creation_expression) are call nodes, since VB.NET writes calls and indexing alike. The names `vb`, `visualbasic`, `vbnet` and `vb.net` select VB.NET in the user config (VB6 keeps its own name). *Test.vb and *Tests.vb are test files, VB.NET joins C# and F# in the .NET family for cross-language resolution, and the vendored grammar MANIFEST records the registration. Co-authored-by: Ankit Susheel <ankitsusheel@gmail.com> Co-authored-by: jorge blanco <jorge.luis.blanco@hotmail.com> Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Definitions: a VB.NET type declaration holds its members directly, and the grammar tags every member with a `body` field, so find_class_body decides VB.NET before the generic field lookup (which returned the first member as the body and lost every method). `Sub New` becomes Method `New`; Inherits / Implements statements give the bases (read with a cursor: a WinForms class holds thousands of members); fields and properties take their `As` type, including `As New T(...)` and `Dim a, b As Integer`; a type nested in a type sits under type_declaration_in_type. Interface and Enum declarations get their labels as in C#, so the VB-only label block is gone. Calls: vb_callee names the callee of `F()`, `F(x)` and `New T`. `Me.X()` and `MyClass.X()` are self-calls, matched by node kind so their spelling does not matter, and `New A.B.C()`, which the grammar reads as `New A` plus member accesses, is one constructor call of `A.B.C`. A call that lands on a Field is VB.NET indexing (`items(0)`): it becomes a USAGE of the Field, as C#'s `items[0]` is, and never a CALLS edge. The sequential and parallel call passes share one emitter for it. Imports: `Imports A.B` and `Imports X = A.B`, with the namespace of the file captured as for C#. Tests: the existing VB.NET extraction tests on the new grammar, plus dotted constructors, calls with arguments, As-clause types, aliased imports, interface / MustOverride / nested members, the Field policy and a WinForms-shaped graph test; the call-node manifest and argument matrix name the new call kinds, and a VB.NET fixture joins the complexity guard. Co-authored-by: Ankit Susheel <ankitsusheel@gmail.com> Co-authored-by: jorge blanco <jorge.luis.blanco@hotmail.com> Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
VB.NET identifiers are case-insensitive: `Calculate`, `CALCULATE` and `calculate` name one symbol. A VB.NET definition records the spelling it is registered under in a table keyed by its ASCII-folded name. A VB.NET lookup that finds nothing under the name as written retries once with every dot segment respelled the way a VB.NET definition spells it (`helpers.format` -> `Helpers.Format`), so an exact spelling always wins. Only VB.NET definitions fill the table and only VB.NET lookups read it, so C# and every other language stay case-sensitive. Two spellings that fold alike keep the strcmp-smaller one, so the table does not depend on registration order. Tests: a mixed-case call and a mixed-case qualified call resolve, an exact spelling keeps its own target, C# lookups do not fold, and the WinForms-shaped graph test gains a call written in another case. Co-authored-by: jorge blanco <jorge.luis.blanco@hotmail.com> Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
VB.NET's Variables are enum members, and an enum member is never called. Without a type for the receiver, `Refresh()` in a Form or `ex.ToString().Trim()` fell through to the short-name registry, which handed back the only project symbol of that name: a member of some enum (`Symbol.Refresh`, `Symbol.Trim`). On staxrip/staxrip that put 267 VB.NET CALLS edges onto enum members, 5 on Fleex255/PolicyPlus. The Java data-member veto already refuses a CALLS bind onto a Variable or a Field. Its language gate becomes cbm_call_never_names_data, shared by the sequential and the parallel call resolver, and admits VB.NET as well. A VB.NET call that lands on a Field never reaches the veto: cbm_call_onto_field_policy has already made it a USAGE of the Field. Measured on both projects: only the CALLS edges onto enum members go, plus the similarity edges derived from them; nodes, USAGE, WRITES, INHERITS and IMPLEMENTS are byte-identical. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings the branch up to main ff812bf. Conflict in src/pipeline/registry.c: main (DeusData#2170) gave registry_resolve_scoped a `call_lang` argument (the calling file's language on the CALLS path, CBM_LANG_COUNT on every other lookup), while this branch wrapped both lookups with the VB.NET case-insensitive retry. Resolved by keeping the retry and passing main's argument through: vb_resolve_folded takes `call_lang` and hands it to its own registry_resolve_scoped call, CBM_LANG_COUNT from cbm_registry_resolve_lang and the caller's language from cbm_registry_resolve_call_lang, as main does for the first lookup. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Resolve the vendored-registry conflicts with the vbnet grammar (DeusData#1179): keep both the haskell fork and vbnet entries in THIRD_PARTY.md and MANIFEST.md, recount the manifest totals (163 = 139 upstream + 19 self-maintained + 5 registry-disagreement; ABI line unchanged), and refresh the MANIFEST.md checksum. Signed-off-by: Edson <zhuoxi2000@gmail.com>
Brings the branch up to main b3ec08d (DeusData#2962, DeusData#2427). Conflict in src/pipeline/registry.c: main (DeusData#2427) threads the extractor's test verdict through cbm_registry_add_lang_in_module into index_under_name, while this branch records each VB.NET definition's spelling for the case-insensitive retry right after that call. Resolved by keeping both: index_under_name takes main's `is_test`, and vb_index_folded still follows it for VB.NET. VB.NET registrations get the verdict the way every language does: the three registry seeders pass the definition's is_test, which the extractor sets from the file's test-ness, and the *Test.vb / *Tests.vb rule of this branch feeds that. Adapted with the merge, per DeusData#2427's convention: the VB.NET registry tests register with `, false`, and vb_test_file_names now also checks that a method of OrderTests.vb is extracted with is_test set and one of Order.vb without it. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
Hi @ankitsusheel, here are the remaining maintainer commits on your branch, as promised, with what each one does and why. They port your extractor to the vendored govindbanura/tree-sitter-vbnet grammar and fold in the work from #2885 by @jorgeluisblanco. Both of you are credited with a Co-authored-by line on the commits that carry your work. 1. Register VB.NET on the vendored vbnet grammar (fde8a65)
2. Port the extractor to the vbnet node kinds (f07a66d)
3. Resolve VB.NET names case-insensitively (1e589d3)
4. Never bind a VB.NET call to an enum member (caed7e4)
5. Merge origin/main (6dd5db5)
6. Merge origin/main again (d7986c0)
On those two projects the branch now indexes every Please pull before pushing to this branch again, so these commits are kept. Thank you both for the careful work and for your patience while we sorted out the grammar. |
Three CI findings on the port, none a behaviour change:
- vb_canonical_callee measured a segment as `dot - seg` and bounded it
with `o + len + 1 >= cap`; GCC 13 on x86_64 (fortified memcpy, -O2)
cannot rule out a wrapped sum and fails the prod build with
-Werror=stringop-overflow. Measure the segment with strcspn and bound
it as `len >= cap - o - 1`, which cannot wrap because o < cap holds.
Reproduced and cleared with the CI image's GCC 13.3 on x86_64.
- vb_as_clause_type returned `text[0] ? text : NULL` after cutting at the
first '('; cppcheck 2.20 cannot see that the cut can empty the string
and reports knownConditionTrueFalse. Return NULL when '(' is the first
character, then cut. Reproduced and cleared with cppcheck 2.20.
- The MANIFEST.md edit of this branch was missing its line in
scripts/vendored-checksums.txt (security-vendored.sh --update; only
that line changes).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
One more maintainer commit on top, to clear the three red checks from the last run. None of them changes behaviour:
Please pull before pushing again. |
Brings the branch up to main d68f491 (DeusData#2967, DeusData#2968, DeusData#2970, DeusData#2974). src/pipeline/registry.c: main (DeusData#2968) gives registry_resolve_scoped the calling definition's test verdict (`caller_test`), adds cbm_registry_resolve_ref for references (USAGE, READS, WRITES) and routes calls through a static resolve_call_lang. This branch wraps the lookups with the VB.NET case-insensitive retry. Resolved by keeping both: vb_resolve_folded takes `caller_test` and passes it to its own lookup, and the retry hooks cbm_registry_resolve_lang (false), cbm_registry_resolve_ref and resolve_call_lang (the caller's verdict), exactly as main's first lookup in each. Without the hook in cbm_registry_resolve_ref, VB.NET references would have stopped folding. internal/cbm/vendored/grammars/MANIFEST.md: main added the sql row next to the vbnet row this branch edits; both rows kept. scripts/vendored-checksums.txt: regenerated with scripts/security-vendored.sh --update; only the MANIFEST.md line differs from main. Adapted with the merge: the VB.NET registry test passes `false` as the new caller_test argument of cbm_registry_resolve_call_scoped (a production caller). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Since DeusData#2968, USAGE, READS and WRITES resolve through cbm_registry_resolve_ref, and the main merge gave it the VB.NET retry. No test reached that hook: dropping it would have silently returned VB.NET references to case-sensitive lookups. A mixed-case reference now resolves from production code and from test code alike. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
Two more maintainer commits on top: main moved again and conflicted with this branch, so we merged it in. Nothing changes for VB.NET:
On staxrip and PolicyPlus the graphs are byte-identical before and after the merge. Please pull before pushing again. |
Vendors harumiWeb/tree-sitter-vba (MIT, (c) 2026 harumiWeb) at 83c5c3c3c721 (tag v0.14.5, upstream HEAD), its vba dialect, as a self-maintained fork in tools/tree-sitter-vba/. This is the precondition for VB6/VBA extraction (DeusData#2564, ported after DeusData#1179); it registers no language yet. Why generated here: upstream does not track its generated parser (its ADR 0002), so the fork carries upstream's grammar source in upstream's layout with no local patch, and parser.c is generated in this repository with tree-sitter-cli 0.25.10 (ABI 15), the generator every fork here uses. Upstream's own copy is generated by 0.26.9. Why this pin: v0.13.0 (the pin DeusData#2564 carried) exposes the same procedures but misreads statements without an ERROR node. On the three corpora below it yields 1,276 expression_statement nodes where v0.14.5 yields 56 (`Err.Raise a, b, c` after a label loses its callee), and it leaves an ERROR at `Case obj Is Nothing`. Why the vba dialect rather than upstream's vb6 one: measured with the CLI on kellyethridge/VBCorLib @ 100acbbae0c0 (655 VB6 files), Kr00l/VBCCR @ 4b86c5c2c075 (161 VB6 files, 70 .ctl) and upstream's 481 VBA examples, procedures outside ERROR nodes are 14,084/14,084, 12,385/1,552 and 8,104/6,573 (vba/vb6). vb6 rejects the PtrSafe/LongPtr forms dual-target VB6 keeps inside #If VBA7 blocks and then loses whole files; vba never sees fewer procedures in any file. vb6's parser.c is also 75.8 MB against 48.3 MB. Provenance by hash: common/define-grammar.js is upstream's blob 5e1f7d7; vba/grammar.js minus its 40-line provenance header is upstream's blob 9247dbbfe166. Two independent generations are byte-identical, and identical to a generation from upstream's own layout and tree-sitter.json. The generated node-types.json is upstream's checked-in file byte-identical (500e635), and the parse-table counts equal upstream's own generated parser. Review: 0 non-ASCII bytes, 0 unmatched characters on full tokenization, one comment (the generator stamp), no string literal with a space, no external scanner, no call outside the tree-sitter macros. No upstream script, package hook or binary was run or vendored. - internal/cbm/vendored/grammars/vba/: the generated parser.c, the tree_sitter/ headers (byte-identical to copies already vendored) and upstream's MIT LICENSE byte-identical. - internal/cbm/grammar_vba.c: compiles and links tree_sitter_vba. No CBMLanguage, extension or alias routes to it yet; extraction adds them under the vb6/vba names (vb and visualbasic mean VB.NET). - MANIFEST.md (fork note with the measurements and known gaps, forks table, 164 grammars, ABI recount), THIRD_PARTY.md (fork list and a vba section), scripts/vendored-checksums.txt, and the FORKS entry in scripts/audit-license-provenance.py. - README.md and the Chocolatey nuspec say 164 languages: the language-count contract defines the count as the vendored grammar directories. - tests/test_parse_coverage.c: a dual-target VB6 class module (IDE class header, Declare in both #If VBA7 branches, Property Get/Let/Set, Implements, events, a label, bare calls) parses with no ERROR node and every bare call keeps its callee; a CRLF VB6 form parses clean; broken source is still flagged. The class case fails on v0.13.0 and on the vb6 dialect. Refs DeusData#721, DeusData#2564 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings the branch up to main 08ab4a0 (DeusData#2966, DeusData#2969, DeusData#2970, DeusData#2977, DeusData#2978, DeusData#2981). scripts/vendored-checksums.txt: main (DeusData#2969) vendored the VB6/VBA grammar while this branch edits MANIFEST.md. Regenerated with scripts/security-vendored.sh --update (1101 files); only the MANIFEST.md line differs from main. Everything else merged without conflict and was checked by hand: - MANIFEST.md keeps main's vba, sql and jsonnet text; only this branch's vbnet registration sentence and custom-handling row differ. - The language count stays 164 (vendored grammar directories); the language-count contract passes. VB.NET adds a language name to the README list, not a grammar. - DeusData#2977's receiver gate is an allowlist (gate_scope); VB.NET is not on it, so it stays GATE_OFF, and receiver_chain_admits is unchanged. - The VB.NET case-insensitive retry still hooks cbm_registry_resolve_lang, cbm_registry_resolve_ref and resolve_call_lang; the Field-call USAGE policy and the enum-member gate are intact. - DeusData#2981's linear-growth rewrites in extract_defs.c touch no VB.NET function; find_class_body still decides VB.NET first. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
One more maintainer merge of main into this branch (2fb3ea1), because main moved on and conflicted again. Nothing changes for VB.NET:
On staxrip and PolicyPlus the graphs are byte-identical before and after the merge. Please pull before pushing again. |
Fixes #1178
Adds VB.NET (
.vb) as a supported language, with real-world WinForms code as the acceptance bar. Before this change.vbfiles were not discovered at all.Grammar. The MIT-licensed govindbanura/tree-sitter-vbnet grammar, vendored by the maintainers in #2948 (the CodeAnt-AI grammar this PR first carried has no upstream license and was removed).
What it does
CBM_LANG_VISUALBASIC("VB.NET") for.vb; the namesvb,visualbasic,vbnetandvb.netmean VB.NET (VB6 keeps its own name).Sub New(MethodNew), interface members,MustOverrideandDeclaremembers, fields and properties with theirAstypes, nested types, andImportswith aliases.F(x)as indexing, so both forms are call nodes;Me./MyClass.self-calls resolve like bare calls;New A.B.C()names the typeA.B.C. A parenthesised name that lands on a Field is indexing, not a call: it becomes a USAGE of that Field, as C#'sitems[0]does, and never a CALLS edge. A call never lands on an enum member.Proof on a real corpus (staxrip/staxrip @2283bfd0a892, 145 VB files; Fleex255/PolicyPlus @e26bca2f1909, 94 VB files)
.vbwas not discovered).String.Compare,String.Contains) matched by name to a project method of the same name, an extension method written for another receiver type, and a sibling class's override picked over the inherited base method.Refresh()in a Form reachingSymbol.Refresh) are gone; apart from a few similarity edges derived from them, nothing else changed.parse_partial; PolicyPlus has none.Known limits
CommandLinePresetType.MuxerMP4Generalreaching the enum memberMuxerMp4General). 417 link a camelCase local, parameter or loop variable of the calling method (Dim key,For Each rawProduct) to a PascalCase member of another class, and most of the rest link a library member such asAutoScaleMode.Dpito a project member of that name. A refinement that skips names the method declares itself is a separate follow-up.Credits: Ankit Susheel (@ankitsusheel) for the original design and WinForms tests; jorge blanco (@jorgeluisblanco, #2885) for the case-insensitive resolution, the call-with-arguments insight and extra coverage.