Skip to content

feat(visualbasic): add VB.NET structural extraction (WinForms-grade) - #1179

Open
ankitsusheel wants to merge 17 commits into
DeusData:mainfrom
ankitsusheel:feat/vbnet-structural-extraction
Open

ankitsusheel wants to merge 17 commits into
DeusData:mainfrom
ankitsusheel:feat/vbnet-structural-extraction

Conversation

@ankitsusheel

@ankitsusheel ankitsusheel commented Jul 19, 2026 •

Copy link
Copy Markdown

Fixes #1178

Adds VB.NET (.vb) as a supported language, with real-world WinForms code as the acceptance bar. Before this change .vb files were not discovered at all.

Grammar. The MIT-licensed govindbanura/tree-sitter-vbnet grammar, vendored by the maintainers in #2948 (the CodeAnt-AI grammar this PR first carried has no upstream license and was removed).

What it does

  • Registers CBM_LANG_VISUALBASIC ("VB.NET") for .vb; the names vb, visualbasic, vbnet and vb.net mean VB.NET (VB6 keeps its own name).
  • Extracts Class/Module/Structure (label Class), Interface, Enum (members as Variable), methods including Sub New (Method New), interface members, MustOverride and Declare members, fields and properties with their As types, nested types, and Imports with aliases.
  • Calls: VB writes calls and indexing alike, and the grammar reads F(x) as indexing, so both forms are call nodes; Me. / MyClass. self-calls resolve like bare calls; New A.B.C() names the type A.B.C. A parenthesised name that lands on a Field is indexing, not a call: it becomes a USAGE of that Field, as C#'s items[0] does, and never a CALLS edge. A call never lands on an enum member.
  • Names resolve case-insensitively, as VB.NET does: a lookup that finds nothing under its own spelling retries once with the spelling a VB.NET definition uses. Only VB.NET lookups read that table, so other languages stay case-sensitive.
  • VB.NET joins C# and F# in the .NET language family for cross-language resolution.

Proof on a real corpus (staxrip/staxrip @2283bfd0a892, 145 VB files; Fleex255/PolicyPlus @e26bca2f1909, 94 VB files)

  • Before: 0 nodes from VB files on either project (.vb was not discovered).
  • After, staxrip: 145 files, 448 Class (incl. Module/Structure), 27 Interface, 107 Enum, 3,427 Method, 5,234 Field, 2,662 enum members; from VB code 6,206 CALLS, 118 INHERITS, 20 IMPLEMENTS, 26,858 USAGE.
  • After, PolicyPlus: 94 files, 137 Class, 1 Interface, 9 Enum, 461 Method, 630 Field, 35 enum members; 442 CALLS, 14 INHERITS, 3 IMPLEMENTS, 2,155 USAGE.
  • Call sites inside methods whose callee names a definition of the project: 66.5% linked on staxrip (11,032 of 16,579), 78.2% on PolicyPlus (687 of 879).
  • Judged sample of 24 edges (seed 1179): 20/24 correct. INHERITS 4/4, IMPLEMENTS 4/4, CALLS 12/16 (the same seed on the branch before the enum-member fix drew a CALLS sample that scored 8/16). The wrong calls are library methods on receivers of library types (String.Compare, String.Contains) matched by name to a project method of the same name, an extension method written for another receiver type, and a sibling class's override picked over the inherited base method.
  • Enum members: 272 CALLS edges onto enum members (Refresh() in a Form reaching Symbol.Refresh) are gone; apart from a few similarity edges derived from them, nothing else changed.
  • Parsing: 4 of staxrip's 145 files keep a localized ERROR from the grammar's known gaps and are flagged parse_partial; PolicyPlus has none.
  • Deterministic: two runs byte-identical on both projects; a non-VB control project (helm @9b88efd6) indexes byte-identical to main.
  • Complexity guard green with a VB.NET fixture (linear scaling).

Known limits

  • A method call on a receiver of a library type is matched by name until receiver types are tracked (typed property chains, a planned follow-up).
  • The case-insensitive retry does not know local scopes yet. On the two projects, 498 CALLS and USAGE edges link only because of it, and 4 of them are correct (CommandLinePresetType.MuxerMP4General reaching the enum member MuxerMp4General). 417 link a camelCase local, parameter or loop variable of the calling method (Dim key, For Each rawProduct) to a PascalCase member of another class, and most of the rest link a library member such as AutoScaleMode.Dpi to a project member of that name. A refinement that skips names the method declares itself is a separate follow-up.

Credits: Ankit Susheel (@ankitsusheel) for the original design and WinForms tests; jorge blanco (@jorgeluisblanco, #2885) for the case-insensitive resolution, the call-with-arguments insight and extra coverage.

@ankitsusheel
ankitsusheel requested a review from DeusData as a code owner July 19, 2026 13:02
@DeusData DeusData added the enhancement New feature or request label Jul 19, 2026
@DeusData DeusData added this to the 0.9.2-rc milestone Jul 19, 2026
@DeusData DeusData added parsing/quality Graph extraction bugs, false positives, missing edges language-request Request for new language support priority/backlog Valuable contribution, lower scheduling urgency; review when maintainer capacity opens. security Security vulnerabilities, hardening labels Jul 19, 2026
@DeusData

Copy link
Copy Markdown
Owner

Thank you for the substantial VB.NET work and for documenting the grammar limitations. The exact-head security review found no malicious code, prompt injection, exfiltration, or hidden execution, but we cannot accept the current supply-chain and correctness direction.

The PR compiles contributor-vendored, locally regenerated third-party parser code into every binary and adds a SPECIAL_NOTICE exception that makes the license audit accept it without an upstream license comparison. The pinned upstream has no LICENSE file; adding standard MIT text locally does not provide the provenance guarantee we require, and the move/zsh precedent is not equivalent because those notices came from alternative upstream repositories with actual license text.

Please pause further implementation work and rework the contribution as follows:

  1. Remove the vendored grammar files, locally created LICENSE, audit exception, generated parser, and unrelated checksum reordering. State the required upstream URL, exact commit, and claimed license in the PR instead. Maintainers will independently audit and vendor/regenerate any accepted grammar.
  2. Obtain an upstream license file or explicit upstream license grant before we vendor it.
  3. Treat VB.NET identifier resolution as case-insensitive before documenting the language as supported.
  4. Add durable multi-file partial-class/Designer coverage plus graph-level assertions for INHERITS versus IMPLEMENTS, Handles resolution, field types, mixed-case calls, and trace behavior.

We are keeping the PR open in 0.9.2-rc so the useful extraction work and review history are not lost. This review is for head 345867b764beac6c0776cde67adbde47d558b113.

@DeusData

Copy link
Copy Markdown
Owner

Thanks for this, and sorry for the slow first response. Queued for review.

Before anything else, credit where it is due: you did the vendoring paperwork correctly and completely. The diff updates internal/cbm/vendored/grammars/MANIFEST.md, vendors vb_dotnet/LICENSE alongside the code, updates THIRD_PARTY.md, and refreshes scripts/vendored-checksums.txt. That is the full policy, met without being asked, and it is genuinely rare in a first contribution. Thank you.

Setting expectations honestly on timing: at +358k lines, the bulk of which is the generated parser.c, this needs a careful review rather than a fast one — license provenance, grammar origin, and the generated-versus-handwritten boundary all get checked on vendored trees. The size is inherent to a tree-sitter grammar and is not a criticism; it just means this will not be a same-day merge.

CI currently shows 2 failing checks — worth a look, since those are likely to be the fastest thing to resolve while the review is pending.

@DeusData

Copy link
Copy Markdown
Owner

Reviewed in full. Security-clean, provenance byte-verified, and — notably — every extractor edit is properly language-gated. One policy question stands between it and a decision, and it is the maintainer's.

The gating deserves specific credit. A sibling grammar PR in this same review round changed type_declaration labelling and component_body traversal globally, silently affecting eight other languages. Yours does not: base-class collection, interface/enum labelling, find_class_body, constructor handling, field/property extraction, body-children routing and both callee paths are each behind an explicit == CBM_LANG_VISUALBASIC guard. The only shared-path change is the mechanical split of push_method_def into push_method_def_named, which is behaviour-equivalent for existing languages. That is the difference between a language addition and a cross-language regression, and you got it right.

Provenance verified as far as it can be. Upstream grammar.js at your pinned cfca210ce8fd, plus your grammar.js.patch, produces the vendored grammar.js byte-identically — the patch applies with zero fuzz. The three tree_sitter/ headers are byte-identical to upstream. MIT is declared in both package.json and tree-sitter.json at that commit, which I confirmed independently. There is no scanner.c (EXTERNAL_TOKEN_COUNT 0, matching upstream), so the hand-written-C attack surface of the vendored tree is zero. A marker grep across the full 12.7 MB diff came back clean — no curl, wget, base64, eval, system(, popen(, getenv, /etc/, .ssh, and zero raw fopen( additions.

All four vendoring policy checks pass, and your MANIFEST and THIRD_PARTY entries disclose the patch honestly rather than presenting the grammar as pristine.

The policy question, which is why this is escalated rather than decided: parser.c is regenerated from a locally patched grammar, so unlike a pristine vendoring it cannot be byte-compared against upstream. Its generation input is fully verified and its structure is canonical tree-sitter v0.25.6 output, but the artifact itself is a fork-by-patch. Whether we accept a patched vendored grammar with a documented re-vendor procedure — or require pristine upstream and wait for your 107-line patch to land there first — is the maintainer's call. I have put it to him with the verification results attached.

Two things you can tidy meanwhile:

  1. The "patches proposed upstream" wording is not yet true. Your PR body says "to be proposed" while MANIFEST says "proposed", and I could find no PR or issue from you on CodeAnt-AI/tree-sitter-vb-dotnet as of today. Either align the wording or actually file them — the second would materially strengthen the case, since it turns a permanent fork into a temporary one.
  2. You added a SPECIAL_NOTICE entry to audit-license-provenance.py for your own grammar. The claim it encodes is true — I verified upstream ships no LICENSE while declaring MIT in metadata — and there is precedent (pine). But a contributor editing the licence gate so their own contribution passes it is something the maintainer should re-issue rather than inherit.

Your CI red is not yours. The single failure across 6,387 tests is test_py_lsp_bench.c:264 ASSERT(ms < 150.0) — a Python LSP timing budget on a shared runner, entirely unrelated to VB.NET. I have flagged that test separately; an absolute-millisecond bound on shared hardware is exactly the flaky shape our own doctrine forbids.

The test coverage here is strong — eleven extraction tests covering WinForms Designer shapes, next-line Inherits/Implements, Sub New, Me.-stripping, #Region, Handles/AddHandler and nested classes, plus a grammar-regression fixture and a label golden. And the design mirrors the existing C# worldview rather than inventing a new one.

It will need a rebase (README count, MANIFEST table, THIRD_PARTY, and the lang_specs/cbm.h tails), but those are mechanical.

ankitsusheel and others added 6 commits August 14, 2026 22:55
Vendor CodeAnt-AI/tree-sitter-vb-dotnet (MIT, ABI 15, no external
scanner) pinned at cfca210, as internal/cbm/vendored/grammars/vb_dotnet/.

parser.c is regenerated (tree-sitter-cli 0.25.6) from a locally patched
grammar.js: upstream at the pinned commit cannot parse several core
VB.NET forms — next-line/colon Inherits/Implements clauses, Handles
clauses, AddHandler/RemoveHandler/RaiseEvent statements, As New
initializers, member-level #Region directives, nested types, and
attribute-with-line-continuation before declarations (the standard
WinForms Designer file shape). The full diff against upstream is
vendored as vb_dotnet/grammar.js.patch alongside the patched grammar.js;
MANIFEST.md documents the re-vendor procedure.

Upstream declares MIT in package.json/tree-sitter.json but ships no
LICENSE file; the vendored LICENSE carries the standard MIT text for
the declared license, per the move/zsh convention in MANIFEST.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Ankit Susheel <ankitsusheel@gmail.com>
Register CBM_LANG_VISUALBASIC (.vb, display name "VB.NET") with a
lang_specs.c row (CBM_LANG_CSHARP as template) over the vb_dotnet
grammar's node kinds, plus the per-language handling the generic
paths cannot cover:

- extract_defs.c: VB blocks have no body wrapper node — find_class_body
  returns the block itself and push_class_body_children routes through
  the nested-class scan so members are not double-extracted as top-level
  functions. interface_block/enum_block get Interface/Enum labels
  (Module/Structure stay Class, matching the C# struct labeling).
  Sub New constructors carry no name node — emitted as Method "New"
  via a new push_method_def_named split of push_method_def.
  Base classes are collected from inherits_clause/implements_clause
  type children (generic (Of T) suffixes stripped), which feeds the
  existing generic INHERITS-vs-IMPLEMENTS decision in pass_semantic.c.
  Fields/properties: field_declaration declarators and
  property_declaration emit Field defs with the as_clause type
  (including the As New shorthand), so WinForms WithEvents controls
  become graph nodes and Handles/AddHandler identifier references
  resolve to them as USAGE edges via the existing usage machinery.

- extract_calls.c: invocation carries its callee in a target field
  (identifier or member_access) and new_expression in type;
  Me./MyClass. receivers are stripped so self-calls resolve like bare
  same-class calls.

Namespaces stay QN-transparent, matching the established C# behavior.
Partial classes keep one node per file (path-derived QNs), also
matching C#.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Ankit Susheel <ankitsusheel@gmail.com>
…ADME

Eleven extraction tests (declarations incl. WinForms Designer shapes,
class-scoped method QNs, next-line Inherits/Implements, dotted external
bases, Sub New constructors, calls incl. Me-receiver stripping and
As New, #Region tolerance, Handles/AddHandler with the AddressOf
handler reference, imports, nested classes), a grammar-regression
fixture row, a label golden, and .vb detection tests. README gains
VB.NET in the "Also supported (not yet benchmarked)" tier and the
language count moves 158 -> 159.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Ankit Susheel <ankitsusheel@gmail.com>
cppcheck(knownConditionTrueFalse):
- extract_defs.c: remove redundant inner `if (base[0])` — base[0] is
  guaranteed non-zero inside the outer `if (base && base[0])` guard;
  the intervening *paren='\0' only truncates mid-string.
- extract_calls.c: simplify `return t[0] ? t : NULL` to `return t`
  for the same reason — t[0] is always true at that point.

license-gate(audit-license-provenance):
- Add vb_dotnet to SPECIAL_NOTICE so the byte-identity audit resolves
  it as MANUAL-VERIFIED instead of ERROR.  Upstream
  CodeAnt-AI/tree-sitter-vb-dotnet declares MIT in package metadata
  but ships no LICENSE file; the vendored LICENSE carries the standard
  MIT text (matching the move/zsh convention used for pine/ISC).

Signed-off-by: Ankit Susheel <ankitsusheel@gmail.com>
clang-format-20 requires the opening brace on its own line when the
array initializer spans multiple lines; the previous column-aligned
style triggered [-Wclang-format-violations].  Auto-corrected with
clang-format; verified lint-ci passes locally.

Signed-off-by: Ankit Susheel <ankitsusheel@gmail.com>
…nd in review

Rebase onto origin/main (641 commits) and address the maintainer's
2026-07-31 review of PR DeusData#1179 plus gaps this rebase surfaced:

- Link the upstream patch PR now that it's filed:
  CodeAnt-AI/tree-sitter-vb-dotnet#8 (MANIFEST.md wording was "to be
  proposed"/"proposed" with no PR to point to; now both point at DeusData#8).
  Verified independently: cloning upstream at the pinned cfca210ce8fd,
  applying grammar.js.patch (zero fuzz), and running `tree-sitter
  generate` (0.25.6) produces a parser.c byte-identical to our vendored
  copy.

- Fix a real supply-chain gate gap: the vb_dotnet grammar's 7 files
  (grammar.js, grammar.js.patch, LICENSE, parser.c, tree_sitter/*.h)
  were never actually added to scripts/vendored-checksums.txt despite
  MANIFEST.md claiming full vendoring policy compliance. security-static
  CI's "Layer 8: Vendored dependency integrity" check
  (scripts/security-vendored.sh) fails on this locally; re-ran
  --update to add the missing entries (1071 files now covered).

- Close 3 post-fork completeness ledgers that the rebase's 641 commits
  added and that assume every CBMLanguage enum value has an explicit
  entry (none existed when this PR's original commits were written):
    - repro_language_registry.c: capability ledger classifies
      VISUALBASIC as CALL_WITH_REFERENCE_VOCAB (its `identifier` grammar
      rule matches extract_usages.c's generic identifier-kind check,
      same shape as CSHARP); aggregate counts updated accordingly.
    - repro_call_argument_matrix_b.c: added a routine-argument case
      (Module Sample / Function Accept|Run, mirroring the adjacent
      OBJECTSCRIPT_ROUTINE row) verifying a direct call + argument
      usage inside a caller scope, plus the bare-reference counterpart;
      language-row and static-assert counts updated (36->37 routine
      rows, 46->47 total).
    - repro_call_node_manifest.c: registered `invocation` (DIRECT_CALL)
      and `new_expression` (CONSTRUCTOR_CALL) as PRIMARY_EXTRACTOR
      entries, matching every other language using these generic
      tree-sitter node-kind names (SQL/JS/TS/CPP/etc.); all four
      aggregate totals updated (219->221 entries, 189->191 active
      primary, 155->156 direct calls, 20->21 constructor calls).

Rebase conflict resolutions worth flagging for review:
- extract_defs.c push_class_body_children: upstream's unified
  find_class_member_body() refactor already returns the class node
  itself for CBM_LANG_VISUALBASIC (find_class_body's existing VB
  branch), making this PR's original explicit early-return special
  case fully redundant. Removed it in favor of the shared path — same
  behavior, one path instead of two.
- extract_defs.c push_method_def / extract_calls.c
  extract_callee_lang_specific: mechanical merges (Kotlin secondary
  constructors and cbm_func_name_node_text's new `lang` parameter were
  added upstream in the same functions we touch; both changes are
  independent and additive).
- README.md: mechanical merge of badge/prose numbers, keeping upstream's
  updated test count and this PR's 159-language bump.

Full suite green after rebase: 7479 passed, 0 failed, 4 skipped.
cppcheck and security-vendored.sh clean locally; scancode still
unavailable on this machine (relying on CI, as noted in the original
PR body).

Signed-off-by: Ankit Susheel <ankitsusheel@gmail.com>
@ankitsusheel
ankitsusheel force-pushed the feat/vbnet-structural-extraction branch from 345867b to f52c5fa Compare August 14, 2026 14:12
@ankitsusheel

Copy link
Copy Markdown
Author

Update in response to the 2026-07-31 review — everything actionable at my end is now done:

1. Rebase. Onto current main (was 641 commits behind). Resolved conflicts in extract_defs.c/extract_calls.c. One worth a look: upstream's push_class_body_children refactor (unified find_class_member_body()) turned out to already handle VB.NET correctly via find_class_body's existing VB branch (which returns the class node itself) — so this PR's original explicit VB early-return was redundant and I removed it in favor of the shared path. Same behavior, less code.

2. Upstream patch filed. CodeAnt-AI/tree-sitter-vb-dotnet#8 — the wording mismatch you flagged ("to be proposed" vs "proposed", no actual PR) is fixed; MANIFEST.md and the PR body both link it now. While preparing it I independently re-verified the whole provenance chain: cloned upstream at the pinned cfca210ce8fd, applied grammar.js.patch (zero fuzz, confirming your check), ran tree-sitter generate 0.25.6, and the output parser.c is byte-identical to what's vendored here.

3. Found a real gap while rebasing. scripts/vendored-checksums.txt never actually got the vb_dotnet grammar's 7 files added, despite MANIFEST.md/THIRD_PARTY.md claiming full policy compliance — security-vendored.sh (Layer 8, the same one security-static CI runs) fails on it locally. Fixed via --update; 1071 files now covered, re-verified clean.

4. Three new completeness ledgers, closed. The 641 rebased commits added repro_language_registry.c, repro_call_argument_matrix_b.c, and repro_call_node_manifest.c — each requires every CBMLanguage enum value to carry an explicit entry (a repro-test pattern that didn't exist when this PR's original commits were written). Added VB.NET's rows to all three, modeled on the closest sibling entries (CSHARP for the capability ledger's reference-vocab classification, OBJECTSCRIPT_ROUTINE for the argument-matrix fixture shape, and the generic invocation/new_expression node-kind pattern shared by SQL/JS/TS/CPP/etc. for the call-node manifest). All three verified passing against the live grammar/extraction, not just asserted.

5. SPECIAL_NOTICE policy question — still standing as you flagged it (I added the entry for my own grammar; re-issuing it is a maintainer call, noted in the PR body).

6. parser.c provenance policy question — also still standing, unchanged by anything above; noted in the PR body for whichever way you/the maintainer decide.

Full suite: 7479 passed, 0 failed, 4 skipped post-rebase. security-vendored.sh and the provenance audit both clean locally. mergeable: MERGEABLE as of this push.

@DeusData

Copy link
Copy Markdown
Owner

Thank you for the rebase and for handing both policy questions back rather than deciding them for us — that was the right instinct, and it is what made a clean ruling possible. Here are the answers, plus one thing I need you to explain.

First, the security review, because it is the most reassuring result I have seen on a vendored grammar. It was exhaustive rather than sampled: parser.c was reconstructed from the diff, verified against the blob hashes, and fully tokenised with zero unmatched characters. All 12,254,747 bytes fall inside {LF} ∪ printable ASCII — so hidden Unicode is impossible, not merely unobserved. 321 distinct string literals, not one containing a space; one comment (the generator banner); two function definitions; no libc call of any kind; no external scanner, confirmed by walking all 32 fields of the TSLanguage initialiser rather than by looking for a filename. Your three support headers are byte-identical to copies already vendored here. Nothing in the payload is a concern.

And your provenance chain for grammar.js is genuinely proven, not asserted. Fetching upstream at cfca210c and applying your grammar.js.patch applies clean with zero fuzz and produces blob 9cb8282d — exactly the vendored file. The symbol delta in parser.c versus upstream is +10/−0, and the five new named symbols are precisely the five rules your patch adds. Every identifier in the parser traces back to the patched grammar with no orphans. That is a much stronger answer than "trust me, I regenerated it."

Two other things deserve explicit credit. You found a real gap in our own supply-chain gate — the seven vb_dotnet files were never actually in scripts/vendored-checksums.txt despite the paperwork claiming compliance — and fixed it. And you flagged your own SPECIAL_NOTICE addition and asked us to re-issue it rather than inherit it. We will re-issue it; that was the correct call to hand over.


Ruling 1 — the LICENSE must come from upstream. This is blocking.

I confirmed it directly: upstream's tree at the pinned commit contains no LICENSE file, and GitHub reports the repository licence as none. MIT is declared only in package.json, tree-sitter.json and the grammar.js header.

We had this exact question on another grammar recently and ruled that a reconstructed LICENSE file is not acceptable, however well-evidenced the intent. The reason is not pedantry: this project shipped GPLv2 code unknowingly once, and the rule that stops it happening again is "the licence text ships from the copyright holder, not from us." Once "the author clearly meant MIT so we wrote the file" becomes precedent, that protection is gone.

I know MANIFEST.md cites the move/zsh convention. The difference is that those notices came from upstream repositories that carry actual licence text; here there is none to carry. So: upstream needs to publish a LICENSE file, and then we re-pin and vendor it.

Ruling 2 — a locally patched grammar is acceptable in principle. Patching and licensing are different questions and I do not want to conflate them. A vendored .patch plus a pinned tree-sitter-cli version is a reproducible chain, and you demonstrated that it reproduces. So we are not going to demand your grammar changes merge upstream before we will take them.


But I need to ask you about the upstream PR, because the record does not match the claim.

CodeAnt-AI/tree-sitter-vb-dotnet#8 was opened on 14 August at 13:21 and closed by you at 14:21, one hour later, unmerged. Your comment here, and both MANIFEST.md and THIRD_PARTY.md, describe the patches as "proposed upstream" with that PR as the reference.

I am not assuming bad faith — there are perfectly good reasons to withdraw a PR, and you may well have spotted something wrong with it. But the effect matters: our manifest would permanently tell a future re-vendorer that upstreaming is pending, when upstream is dormant (last push July 2025) with five unmerged community PRs and no open path. Whatever the reason, the manifest needs to say plainly that this is a maintained fork with no upstream path — that is honest and it is fine. What it must not do is imply a resolution that is not coming. Could you tell me what happened there, and correct the wording?

Ruling 3 — case-insensitive resolution is required before merge. I know you documented it as a limitation, and I understand why: registry.c is shared cross-language surface and touching it is riskier than the isolated extractor work. But VB.NET is case-insensitive by design, so Foo and foo are one symbol, and without folding we would silently drop real edges. A missing edge that nobody can see is the worst failure mode this graph has, and calling the language "supported" while that is true would be a claim we cannot honour. If the registry.c change is where you would rather not go alone, say so and I will pair on it or take that piece.

One repo rule to clear: three commits carry a Co-Authored-By: Claude Fable 5 trailer and the PR body ends with a "Generated with Claude Code" footer. We do not carry AI attribution in the public history, so those need stripping before merge — it is not a judgement on how you worked, just a house rule about what lands on main.


Where that leaves us. The engineering is good, the security is clean, the scope is honest, and the three completeness ledgers you closed are correct against current main. The blocker is upstream's missing LICENSE, which is not something either of us can fix by writing more code. If you have a line to the CodeAnt AI folks, a LICENSE file is a two-minute change for them and it unblocks this entirely. I would rather wait for that than set a precedent I would have to defend to every future contributor.

@DeusData

Copy link
Copy Markdown
Owner

@ankitsusheel — no new asks, and the three rulings from 2026-08-20 stand exactly as written. This is a heads-up that main moved under you today in two ways that will affect your next rebase, so you find out from me rather than from a red CI run.

1. Two languages landed today, ArkTS (#1842) and PL/SQL (#1854), so the registry ledger constants on your branch are now short by two. Most will conflict loudly, which is fine. One will not, and that is the one worth your attention:

constant your branch main today correct with VB.NET
EXPECTED_MATRIX_B_ROWS 47 47 48
EXPECTED_CALL_CAPABLE_LANGUAGES 112 113 114
EXPECTED_HISTORICAL_CALL_NODE_TOTAL 221 222 223
EXPECTED_ACTIVE_PRIMARY_TOTAL / EXPECTED_PRIMARY_OWNERS 191 192 193
EXPECTED_DIRECT_CALLS 156 157 158

EXPECTED_MATRIX_B_ROWS is the trap: your branch and main both independently wrote 47, so git will merge it cleanly with no conflict — and the result is wrong, because the truth after both changes is 48. Identical text, additive meaning. I hit this exact failure myself this morning rebasing PL/SQL across ArkTS: ten files conflicted loudly and were easy, and the single one that merged silently was the one that shipped a wrong number. repro_language_registry catches it, but only after you have already convinced yourself the rebase went well.

The audit that finds the whole class, if it is useful: diff each side against the merge base, keep numeric lines, and intersect them — anything appearing in both lists merged silently and is additive until proven otherwise. Exclude */vendored/* or the generated parser.c tables drown the signal.

2. The published language count is now gated. #1859 landed today: tests/test_language_count_contract.sh derives the count from the vendored grammar directories and asserts that every published surface states it — README, docs/index.html, docs/llms.txt, pkg/npm/README.md, the chocolatey nuspec, scripts/package-release.sh and server.json. It also fails if an unregistered file states a count.

Your branch bumps README from 158 to 159. main is now at 161, so with vb_dotnet the correct value is 162, and it has to be updated in all seven places rather than just the README. The gate runs as Step 0w in scripts/test.sh, so you can check it locally before pushing.

Worth saying plainly: neither of these is a criticism of your branch. They are both consequences of main moving while a carefully-reviewed PR waited, and the second one exists precisely because the count had silently drifted across seven surfaces — the same class of problem as the counter above.

Nothing here changes the 08-20 rulings. The blocking items remain the upstream LICENSE file, case-insensitive identifier resolution, the manifest wording about the upstream PR, and removing the AI-attribution trailers. My offer to pair on or take the registry.c case-folding piece also still stands.

@DeusData

Copy link
Copy Markdown
Owner

Correction to my table above — it was incomplete in a way that matters. I listed one silently-merging constant; a fuller audit found five, plus two more files. Since the whole point of that comment was to keep you from trusting a clean-looking rebase, an incomplete list is worse than none. The complete set:

Silent — both sides identical, merges clean, result wrong (these are the dangerous ones):

constant both sides say correct with VB.NET
EXPECTED_MATRIX_B_ROWS (repro_language_registry.c) 47 48
counts[CAP_CALL_WITH_REFERENCE_VOCAB] (same file) 87 88
EXPECTED_CONSTRUCTOR_CALLS (repro_call_node_manifest.c) 21 22
ROUTINE_ARGUMENT_LANGUAGE_COUNT (repro_call_argument_matrix_b.c) 37 38
MATRIX_LANGUAGE_COUNT assert (same file) 47 48, range label becomes RACKET..VISUALBASIC

Loud — will conflict, resolve to these (visible, so lower risk):

constant yours main correct
EXPECTED_CALL_CAPABLE_LANGUAGES 112 113 114
EXPECTED_HISTORICAL_CALL_NODE_TOTAL (+ its "exactly N entries" assert string) 221 222 223 → with your row, 224
EXPECTED_ACTIVE_PRIMARY_TOTAL / EXPECTED_PRIMARY_OWNERS 191 192 194
EXPECTED_DIRECT_CALLS 156 157 158
EXPECTED_MATRIX_A_ROWS / CAP_CALL_WITHOUT_REFERENCE_VOCAB untouched 68 / 26 take main's

Also stale by two, worth catching in the same pass: the header comment in repro_call_argument_matrix_b.c (CBMLanguage 80..N — after your rebase, 80..165) and the README shields badge (languages-N-orange). The badge is newly relevant: the count contract from #1859 originally matched only the prose form, which is exactly how the badge sat at 158 while the prose said 161 — that hole is closed in #1863, so after it lands the gate will fail on a stale badge too. With vb_dotnet vendored, every counted surface (prose + badge + MANIFEST + the six package surfaces) should say 162.

The three rulings from 08-20 are of course unchanged by any of this.

@DeusData

Copy link
Copy Markdown
Owner

Thank you, @ankitsusheel, and sorry for the long silence since August. We've made the direction call: VB.NET structural support is welcome. At 358k lines, though, it can't be reviewed as one PR. Could you split it into two, in this order?

  1. The grammar, vendored on its own.
    • The grammar's license must be permissive; if it's GPL, check with us first.
    • Vendor its LICENSE file with the code, and update THIRD_PARTY.md, the SBOM and internal/cbm/vendored/grammars/MANIFEST.md in the same PR.
    • Update every language-count surface; the version-metadata contract test will name any you miss.
    • The generated parser.c is the bulk of it, and that is fine.
  2. VB.NET extraction on top, with a small real-world proof: node and edge counts, plus index time, on one public VB.NET or WinForms project. That's how we judge graph quality before building on it.

Rebased onto current main, DCO-signed, and green before review, please. Thank you for the substantial work here.

@DeusData

Copy link
Copy Markdown
Owner

Thanks again for the VB.NET work. The September request remains grammar first, extraction on top, with the upstream-provided license prerequisite preserved. Is there an update on that license or the first split? No additional implementation scope is being requested here.

@DeusData

Copy link
Copy Markdown
Owner

Hi @ankitsusheel, thank you again for the VB.NET work. We have now decided how VB.NET support will land, and we want to explain the reasoning here, including a correction on our side.

The grammar this PR builds on, CodeAnt-AI/tree-sitter-vb-dotnet, still has no LICENSE file upstream. Without a license, that code is "all rights reserved" by default, so nobody may copy or redistribute it, and our rule for vendored code is permissive licenses only, with the license file shipped next to the code. Your upstream request for a license there was closed in August and the repository has been inactive since mid-2025, so this is not something you could fix with more code, and we cannot vendor that grammar either.

The correction: in September and October we asked you to split the grammar out and vendor it in its own pull request. That does not match how we handle grammars, and we should not have asked. We vendor third-party grammars ourselves: we pin the upstream source, verify its license, regenerate the parser and record its provenance in-house.

So here is the plan:

  1. We vendor the MIT-licensed govindbanura/tree-sitter-vbnet grammar ourselves, in our own pull request. That one lands first; it is the precondition for the rest.
  2. Then we port this PR ourselves with maintainer commits on your branch (thank you for allowing maintainer edits): we remove the CodeAnt-AI grammar files from it and adapt your extractor to the vendored grammar's node names, keeping your design and your WinForms test shapes (partial classes, next-line Inherits/Implements, Sub New, Handles/AddHandler, #Region, nested types). Your commits and authorship stay on the branch.
  3. Two behaviour decisions come with it: vb and visualbasic will mean VB.NET (VB6 keeps its own name), and VB.NET names will resolve case-insensitively, as the language does. We will also fold in the case-insensitive resolution and any extra coverage from feat: add Visual Basic .NET support #2885, with credit to its author.

Nothing is needed from you. If you do push to this branch again, please pull first so our commits are kept. We will explain every commit here as it lands. Thank you for your patience, and for all the care you put into this.

DeusData added a commit to AmirF194/codebase-memory-mcp that referenced this pull request Nov 10, 2026
Vendors govindbanura/tree-sitter-vbnet (MIT, (c) 2025 Govind Banura) at
538b7087bf80 (tag v0.1-beta, upstream HEAD) with CodeGraph's real-world
VB.NET patch (colbymchenry/codegraph docs/grammars/tree-sitter-vbnet.patch
at fe7542b6f6a4, MIT, (c) 2026 Colby Mchenry), as a self-maintained fork
in tools/tree-sitter-vbnet/. This is the precondition for VB.NET
extraction (DeusData#1179, folding in DeusData#2885); it registers no language yet.

Why this grammar: it is the licensed one. CodeAnt-AI/tree-sitter-vb-dotnet,
which DeusData#1179 was built on, carries no license and cannot be vendored.

Why the patch: upstream checks in grammar.js only and does not parse the
code VB.NET projects are made of. Unpatched, the Single literals in every
WinForms .Designer.vb (New SizeF(6.0!, 13.0!)) and body-less MustOverride
members are ERROR nodes. The 2026-10-06 revision of the patch is pinned
because it also keeps names that begin with a keyword whole: the earlier
revision (the one DeusData#2885 carried) turns `Sub NewItem()` into an ERROR and
`Public SharedCache As Integer` into a Shared field named Cache.

Provenance by hash: the patch file is byte-identical to CodeGraph's (blob
37918d8); upstream grammar.js at the pin is the patch's pre-image
(21c8523), and tools/tree-sitter-vbnet/grammar.js minus its 41-line
provenance header is the post-image (c9c1d85), as is scanner.c (8334fdb).
parser.c is generated here with tree-sitter-cli 0.25.10 (ABI 15); two
independent generations are byte-identical. No upstream script, package
hook or binary was run or vendored.

- internal/cbm/vendored/grammars/vbnet/: the generated parser.c,
  CodeGraph's external scanner.c (two stateless tokens), the tree_sitter/
  headers, upstream's MIT LICENSE byte-identical, and CodeGraph's MIT text
  as LICENSE-codegraph: the scanner and the patched rules are CodeGraph's
  work, so its notice has to ship with the binary too.
- internal/cbm/grammar_vbnet.c: compiles and links tree_sitter_vbnet. No
  CBMLanguage, extension or alias routes to it yet; extraction adds them,
  with `vb` and `visualbasic` meaning VB.NET (VB6 keeps its own name).
- MANIFEST.md (fork note, forks table, 163 grammars, ABI recount),
  THIRD_PARTY.md (fork list and a vbnet section; the stale "160
  pre-generated parsers" now reads 163), scripts/vendored-checksums.txt,
  and the FORKS entry in scripts/audit-license-provenance.py.
- README.md and the Chocolatey nuspec say 163 languages: the
  language-count contract defines the count as the vendored grammar
  directories.
- tests/test_parse_coverage.c: real VB.NET (a WinForms Designer partial
  class, Handles, Inherits/Implements, MustOverride, #Region, Property,
  keyword-prefixed names) parses with no ERROR node. The case fails on
  unpatched upstream (8 ERROR nodes) and on the earlier patch revision (1).

Refs DeusData#1178, DeusData#1179, DeusData#2885

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
DeusData added a commit to AmirF194/codebase-memory-mcp that referenced this pull request Nov 10, 2026
The vendored vbnet scanner lexes a whole VB XML literal as one external
token, and the parser offers it a '<' wherever a declaration or member
starts. There, in valid VB, '<' opens an attribute block
(`<Serializable()> _`), never an XML literal. The scanner nevertheless
took `<Serializable` for an element start and read the rest of the file
looking for a closing tag before giving up: O(rest of file) per
attribute, O(n^2) per file. Generated proxies and designer code put an
attribute on every member: 2,500 attributes in a 77 KB file advanced
38.5 M characters, and a 1.35 MB synthetic file took 21 s to parse.

The scan now fails at the first character well-formed XML cannot
contain: inside a tag, outside a quoted value, anything other than name
characters, blanks, '=', quotes, '/>', '>' and '<%='; in element text, a
'<' that starts no tag, closing tag, comment, PI or embedded expression,
and an '&' that starts no reference. No valid XML literal lexes
differently: parse trees are byte-identical before and after on all 25
sample and synthetic files tried, XML literals of every shape included.
The 1.35 MB file now parses in 0.08 s.

Residual, recorded in MANIFEST.md: an attribute without an argument list
(`<Fact>`) is a well-formed start tag, so its scan still runs until a
character that cannot be XML, or the end of the file.

The change is kept as tools/tree-sitter-vbnet/patches/
xml-literal-tag-syntax.patch (CodeGraph's scanner 8334fdb -> b3473d1)
and in MANIFEST.md's local-patch table. The new
complexity_vbnet_attribute_lexing_is_linear counts the bytes the lexer
pulls through a chunked TSInput for k and 2k attribute-decorated
classes: ratio 3.99 before the fix, 2.0 after.

Refs DeusData#1178, DeusData#1179

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings the branch up to main (3,220 commits) and onto the VB.NET grammar
main now vendors itself (govindbanura/tree-sitter-vbnet, DeusData#2948).

Resolution:
- Dropped the CodeAnt-AI/tree-sitter-vb-dotnet grammar: the vendored
  vb_dotnet/ tree and its compile unit grammar_visualbasic.c. That
  upstream carries no LICENSE file, so it cannot be vendored; main
  compiles the MIT-licensed vbnet grammar through grammar_vbnet.c.
- Took main's MANIFEST.md, THIRD_PARTY.md, scripts/vendored-checksums.txt
  and scripts/audit-license-provenance.py unchanged: the vb_dotnet
  records and its license-audit exception go with the grammar, and main
  already carries the records for vbnet.
- lang_specs.c: the VB.NET row now names tree_sitter_vbnet, the grammar
  that is left to link against. Its node-kind lists still name the old
  grammar's kinds; the port to the vbnet node kinds follows separately.
- README: main's counts (163 already includes vbnet) plus this branch's
  VB.NET entry in the not-yet-benchmarked list.
- cbm.h, language.c, extract_defs.c and the tests: main's new entries
  first, then this branch's VB.NET lines unchanged.
- Repro ledgers rebuilt from main's values plus this branch's VB.NET
  rows: capability ledger 89 call-with-reference-vocab languages, matrix
  B 50 rows (RACKET..VISUALBASIC, CBMLanguage 80..170), 116
  call-capable languages; call-node manifest 228 entries (198 primary,
  161 direct, 23 constructor).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData

Copy link
Copy Markdown
Owner

Hi @ankitsusheel, as promised, here is the first of our maintainer commits on your branch, with what it does and why.

Merge origin/main into feat/vbnet-structural-extraction (3cc3a3f)

  1. It brings the branch up to current main (about 3,200 commits) with a regular merge commit, so your six commits stay exactly as you wrote them.
  2. It removes the CodeAnt-AI grammar: the vendored vb_dotnet directory and grammar_visualbasic.c. Main now carries its own vendored copy of the MIT-licensed govindbanura/tree-sitter-vbnet grammar (chore(vendor): vendor the VB.NET tree-sitter grammar #2948), and the VB.NET row in lang_specs.c now points at it.
  3. It drops the branch's license-audit exception, checksum entries and MANIFEST/THIRD_PARTY rows for vb_dotnet. They describe the grammar that is gone, and main already records the vendored vbnet grammar.
  4. Everything else of yours is unchanged: the CBM_LANG_VISUALBASIC registration, the .vb mapping, your extractor code and all eleven WinForms-shaped tests. The README keeps main's count (163, which already includes vbnet) and gains VB.NET in the language list, and the repro ledgers carry your VB.NET rows on top of main's current numbers.

At this commit your extractor still uses the old grammar's node names (class_block, invocation, inherits_clause and so on), so the VB.NET tests are expected to fail until the next commits land. Those port the extractor to the new grammar's node names while keeping your design (partial classes, next-line Inherits/Implements, Sub New, Handles/AddHandler, #Region, nested types), and add case-insensitive name resolution together with the work from #2885. We will explain each of them here when they land.

Please pull before pushing to this branch again, so these commits are kept. Thank you again for all the care you put into this.

DeusData and others added 5 commits November 11, 2026 20:41
Point the VB.NET language spec at the node kinds of the vendored
govindbanura/tree-sitter-vbnet grammar (DeusData#2948): Sub and Function,
MustOverride, Declare and interface members are functions; Class,
Module, Structure, Interface and Enum declarations are types; `F()`
(invocation_expression), `F(x)` (array_access_expression) and `New T`
(object_creation_expression) are call nodes, since VB.NET writes calls
and indexing alike.

The names `vb`, `visualbasic`, `vbnet` and `vb.net` select VB.NET in
the user config (VB6 keeps its own name). *Test.vb and *Tests.vb are
test files, VB.NET joins C# and F# in the .NET family for
cross-language resolution, and the vendored grammar MANIFEST records
the registration.

Co-authored-by: Ankit Susheel <ankitsusheel@gmail.com>
Co-authored-by: jorge blanco <jorge.luis.blanco@hotmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Definitions: a VB.NET type declaration holds its members directly, and
the grammar tags every member with a `body` field, so find_class_body
decides VB.NET before the generic field lookup (which returned the
first member as the body and lost every method). `Sub New` becomes
Method `New`; Inherits / Implements statements give the bases (read
with a cursor: a WinForms class holds thousands of members); fields and
properties take their `As` type, including `As New T(...)` and
`Dim a, b As Integer`; a type nested in a type sits under
type_declaration_in_type. Interface and Enum declarations get their
labels as in C#, so the VB-only label block is gone.

Calls: vb_callee names the callee of `F()`, `F(x)` and `New T`. `Me.X()`
and `MyClass.X()` are self-calls, matched by node kind so their spelling
does not matter, and `New A.B.C()`, which the grammar reads as `New A`
plus member accesses, is one constructor call of `A.B.C`. A call that
lands on a Field is VB.NET indexing (`items(0)`): it becomes a USAGE of
the Field, as C#'s `items[0]` is, and never a CALLS edge. The
sequential and parallel call passes share one emitter for it.

Imports: `Imports A.B` and `Imports X = A.B`, with the namespace of the
file captured as for C#.

Tests: the existing VB.NET extraction tests on the new grammar, plus
dotted constructors, calls with arguments, As-clause types, aliased
imports, interface / MustOverride / nested members, the Field policy
and a WinForms-shaped graph test; the call-node manifest and argument
matrix name the new call kinds, and a VB.NET fixture joins the
complexity guard.

Co-authored-by: Ankit Susheel <ankitsusheel@gmail.com>
Co-authored-by: jorge blanco <jorge.luis.blanco@hotmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
VB.NET identifiers are case-insensitive: `Calculate`, `CALCULATE` and
`calculate` name one symbol. A VB.NET definition records the spelling
it is registered under in a table keyed by its ASCII-folded name. A
VB.NET lookup that finds nothing under the name as written retries once
with every dot segment respelled the way a VB.NET definition spells it
(`helpers.format` -> `Helpers.Format`), so an exact spelling always
wins. Only VB.NET definitions fill the table and only VB.NET lookups
read it, so C# and every other language stay case-sensitive. Two
spellings that fold alike keep the strcmp-smaller one, so the table
does not depend on registration order.

Tests: a mixed-case call and a mixed-case qualified call resolve, an
exact spelling keeps its own target, C# lookups do not fold, and the
WinForms-shaped graph test gains a call written in another case.

Co-authored-by: jorge blanco <jorge.luis.blanco@hotmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
VB.NET's Variables are enum members, and an enum member is never
called. Without a type for the receiver, `Refresh()` in a Form or
`ex.ToString().Trim()` fell through to the short-name registry, which
handed back the only project symbol of that name: a member of some
enum (`Symbol.Refresh`, `Symbol.Trim`). On staxrip/staxrip that put 267
VB.NET CALLS edges onto enum members, 5 on Fleex255/PolicyPlus.

The Java data-member veto already refuses a CALLS bind onto a Variable
or a Field. Its language gate becomes cbm_call_never_names_data, shared
by the sequential and the parallel call resolver, and admits VB.NET as
well. A VB.NET call that lands on a Field never reaches the veto:
cbm_call_onto_field_policy has already made it a USAGE of the Field.

Measured on both projects: only the CALLS edges onto enum members go,
plus the similarity edges derived from them; nodes, USAGE, WRITES,
INHERITS and IMPLEMENTS are byte-identical.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings the branch up to main ff812bf.

Conflict in src/pipeline/registry.c: main (DeusData#2170) gave
registry_resolve_scoped a `call_lang` argument (the calling file's
language on the CALLS path, CBM_LANG_COUNT on every other lookup),
while this branch wrapped both lookups with the VB.NET case-insensitive
retry. Resolved by keeping the retry and passing main's argument
through: vb_resolve_folded takes `call_lang` and hands it to its own
registry_resolve_scoped call, CBM_LANG_COUNT from
cbm_registry_resolve_lang and the caller's language from
cbm_registry_resolve_call_lang, as main does for the first lookup.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Zhuoxi2000 added a commit to Zhuoxi2000/codebase-memory-mcp that referenced this pull request Nov 12, 2026
Resolve the vendored-registry conflicts with the vbnet grammar (DeusData#1179):
keep both the haskell fork and vbnet entries in THIRD_PARTY.md and
MANIFEST.md, recount the manifest totals (163 = 139 upstream + 19
self-maintained + 5 registry-disagreement; ABI line unchanged), and
refresh the MANIFEST.md checksum.

Signed-off-by: Edson <zhuoxi2000@gmail.com>
Brings the branch up to main b3ec08d (DeusData#2962, DeusData#2427).

Conflict in src/pipeline/registry.c: main (DeusData#2427) threads the
extractor's test verdict through cbm_registry_add_lang_in_module into
index_under_name, while this branch records each VB.NET definition's
spelling for the case-insensitive retry right after that call. Resolved
by keeping both: index_under_name takes main's `is_test`, and
vb_index_folded still follows it for VB.NET.

VB.NET registrations get the verdict the way every language does: the
three registry seeders pass the definition's is_test, which the
extractor sets from the file's test-ness, and the *Test.vb / *Tests.vb
rule of this branch feeds that. Adapted with the merge, per DeusData#2427's
convention: the VB.NET registry tests register with `, false`, and
vb_test_file_names now also checks that a method of OrderTests.vb is
extracted with is_test set and one of Order.vb without it.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData

Copy link
Copy Markdown
Owner

Hi @ankitsusheel, here are the remaining maintainer commits on your branch, as promised, with what each one does and why. They port your extractor to the vendored govindbanura/tree-sitter-vbnet grammar and fold in the work from #2885 by @jorgeluisblanco. Both of you are credited with a Co-authored-by line on the commits that carry your work.

1. Register VB.NET on the vendored vbnet grammar (fde8a65)

  1. The VB.NET language spec now names the new grammar's node kinds: Sub and Function, MustOverride, Declare and interface members are functions; Class, Module, Structure, Interface and Enum declarations are types.
  2. F(), F(x) and New T are all call nodes. VB.NET writes calls and indexing the same way and the grammar reads F(x) as indexing, so without that second kind most calls with arguments would stay invisible. That was jorge's observation in feat: add Visual Basic .NET support #2885.
  3. vb, visualbasic, vbnet and vb.net name VB.NET in the config (VB6 keeps its own name), *Test.vb and *Tests.vb are test files, and VB.NET joins C# and F# in the .NET family for cross-language resolution.

2. Port the extractor to the vbnet node kinds (f07a66d)

  1. Your design carries over as it was: members are routed from the type declaration itself, Sub New is Method New, Inherits / Implements feed the INHERITS-vs-IMPLEMENTS decision, fields and properties keep their As types (also As New and Dim a, b As Integer), Handles / AddHandler reach the WithEvents controls, and nested types stay nested. Your eleven WinForms-shaped tests pass unchanged on the new grammar.
  2. The new grammar tags every member of a class with a body field, so the generic body lookup took the first member for the class body and the methods went missing. VB.NET now decides this before that lookup.
  3. The grammar reads New System.Windows.Forms.Button() as New System plus member accesses. It is now one constructor call of System.Windows.Forms.Button.
  4. items(0) on a field is indexing, not a call: it becomes a USAGE of the field, the way C# records items[0], and never a CALLS edge.
  5. Imports with and without an alias, new tests for each case above, and a VB.NET fixture in the complexity guard, which checks that indexing time grows linearly with the project.

3. Resolve VB.NET names case-insensitively (1e589d3)

  1. jorge's folded lookup key, kept in a table that only VB.NET lookups read: a lookup that finds nothing under the spelling as written retries once with the spelling the definition uses, so helpers.format finds Helpers.Format. An exact spelling always wins, and C# and every other language stay case-sensitive.
  2. We measured it on two real WinForms projects (staxrip and PolicyPlus). The retry does not know local scopes yet, so it also links a camelCase local such as Dim key to a PascalCase member of another class. The numbers are under "Known limits" in the description; a refinement that skips names the method declares itself will be a separate follow-up.

4. Never bind a VB.NET call to an enum member (caed7e4)

  1. Enum members are Variables in the graph, and an enum member is never called. Without a type for the receiver, Refresh() in a Form fell through to the only project symbol of that name, a member of an enum, and became a CALLS edge into it. The check that already keeps Java calls off data now covers VB.NET as well, through one language gate that both call resolvers share.
  2. On the two projects that removes 272 such edges and changes nothing else apart from a few similarity edges derived from them. A new test pins it: a VB.NET call that would land on an enum member gets no CALLS edge, while a real method call keeps its own.

5. Merge origin/main (6dd5db5)

  1. It brings the branch up to current main with a regular merge commit. Main had meanwhile given the registry lookup an extra argument (the calling file's language, from fix(registry): recognize C# lower-case type keywords in receiver_chain_admits #2170), and the VB.NET retry from commit 3 now passes that argument through exactly as main's own lookup does. The graphs of both projects are byte-identical before and after the merge.

6. Merge origin/main again (d7986c0)

  1. Main moved once more while this was being checked: fix(registry): score a candidate on the extractor's test verdict, not its name #2427 now ranks registry candidates by the extractor's test verdict instead of by words in their names. The merge keeps both sides: the registry records that verdict, and the VB.NET case-insensitive index from commit 3 still follows it. VB.NET definitions get the verdict the way every language does, so your *Test.vb / *Tests.vb rule feeds it, and a test now checks that a method of OrderTests.vb is marked as test code and one of Order.vb is not.
  2. A side effect on the two projects, all of it for the better: classes whose names only happen to contain "Test" or "spec" (TestForm, InspectPolicyElements) were ranked like test code before, so a call to their own InitializeComponent or Dispose could land on another form's. They now resolve to their own members.

On those two projects the branch now indexes every .vb file; the numbers are in the updated description.

Please pull before pushing to this branch again, so these commits are kept. Thank you both for the careful work and for your patience while we sorted out the grammar.

Three CI findings on the port, none a behaviour change:

- vb_canonical_callee measured a segment as `dot - seg` and bounded it
  with `o + len + 1 >= cap`; GCC 13 on x86_64 (fortified memcpy, -O2)
  cannot rule out a wrapped sum and fails the prod build with
  -Werror=stringop-overflow. Measure the segment with strcspn and bound
  it as `len >= cap - o - 1`, which cannot wrap because o < cap holds.
  Reproduced and cleared with the CI image's GCC 13.3 on x86_64.
- vb_as_clause_type returned `text[0] ? text : NULL` after cutting at the
  first '('; cppcheck 2.20 cannot see that the cut can empty the string
  and reports knownConditionTrueFalse. Return NULL when '(' is the first
  character, then cut. Reproduced and cleared with cppcheck 2.20.
- The MANIFEST.md edit of this branch was missing its line in
  scripts/vendored-checksums.txt (security-vendored.sh --update; only
  that line changes).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData

Copy link
Copy Markdown
Owner

One more maintainer commit on top, to clear the three red checks from the last run. None of them changes behaviour:

  1. The Ubuntu prod build (and the CodeQL build, which compiles the same way) stopped at a GCC fortify error in our vb_canonical_callee: GCC 13 could not rule out a wrapped length in the bounds check before a memcpy. The segment is now measured with strcspn and bounded as len >= cap - o - 1, which cannot wrap. We reproduced the error and its fix with the CI image's GCC on x86_64.
  2. cppcheck reported a condition in vb_as_clause_type as always true, because it cannot see that cutting the type text at its first ( can empty it. The function now returns early when ( is the first character. Reproduced and cleared with the CI's cppcheck 2.20.
  3. The vendored-integrity check flagged MANIFEST.md: the branch's edit there was missing its line in scripts/vendored-checksums.txt. Only that line changes.

Please pull before pushing again.

Brings the branch up to main d68f491 (DeusData#2967, DeusData#2968, DeusData#2970, DeusData#2974).

src/pipeline/registry.c: main (DeusData#2968) gives registry_resolve_scoped the
calling definition's test verdict (`caller_test`), adds
cbm_registry_resolve_ref for references (USAGE, READS, WRITES) and
routes calls through a static resolve_call_lang. This branch wraps the
lookups with the VB.NET case-insensitive retry. Resolved by keeping
both: vb_resolve_folded takes `caller_test` and passes it to its own
lookup, and the retry hooks cbm_registry_resolve_lang (false),
cbm_registry_resolve_ref and resolve_call_lang (the caller's verdict),
exactly as main's first lookup in each. Without the hook in
cbm_registry_resolve_ref, VB.NET references would have stopped folding.

internal/cbm/vendored/grammars/MANIFEST.md: main added the sql row next
to the vbnet row this branch edits; both rows kept.

scripts/vendored-checksums.txt: regenerated with
scripts/security-vendored.sh --update; only the MANIFEST.md line
differs from main.

Adapted with the merge: the VB.NET registry test passes `false` as the
new caller_test argument of cbm_registry_resolve_call_scoped (a
production caller).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Since DeusData#2968, USAGE, READS and WRITES resolve through
cbm_registry_resolve_ref, and the main merge gave it the VB.NET retry.
No test reached that hook: dropping it would have silently returned
VB.NET references to case-sensitive lookups. A mixed-case reference now
resolves from production code and from test code alike.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData

Copy link
Copy Markdown
Owner

Two more maintainer commits on top: main moved again and conflicted with this branch, so we merged it in. Nothing changes for VB.NET:

  1. Merge origin/main (2f37f6f). Main (fix(registry): rank test-file definitions as test for production callers #2968) now passes the calling code's test verdict through every registry lookup and resolves references (USAGE, READS, WRITES) through a lookup of their own. The VB.NET case-insensitive retry now hooks that new lookup as well and passes the verdict through exactly as main's lookups do, so VB.NET references keep resolving whatever their spelling. The grammar manifest keeps both the vbnet row and main's new sql row, and the vendored checksums were regenerated (only the manifest's line differs from main).
  2. A test for that hook (c6c6a2a). Before this, no test reached the reference lookup's retry, so losing it would have gone unnoticed. A mixed-case reference now has to resolve from production code and from test code alike.

On staxrip and PolicyPlus the graphs are byte-identical before and after the merge.

Please pull before pushing again.

DeusData added a commit to potto007/codebase-memory-mcp that referenced this pull request Nov 14, 2026
Vendors harumiWeb/tree-sitter-vba (MIT, (c) 2026 harumiWeb) at
83c5c3c3c721 (tag v0.14.5, upstream HEAD), its vba dialect, as a
self-maintained fork in tools/tree-sitter-vba/. This is the precondition
for VB6/VBA extraction (DeusData#2564, ported after DeusData#1179); it registers no
language yet.

Why generated here: upstream does not track its generated parser (its
ADR 0002), so the fork carries upstream's grammar source in upstream's
layout with no local patch, and parser.c is generated in this repository
with tree-sitter-cli 0.25.10 (ABI 15), the generator every fork here
uses. Upstream's own copy is generated by 0.26.9.

Why this pin: v0.13.0 (the pin DeusData#2564 carried) exposes the same procedures
but misreads statements without an ERROR node. On the three corpora
below it yields 1,276 expression_statement nodes where v0.14.5 yields 56
(`Err.Raise a, b, c` after a label loses its callee), and it leaves an
ERROR at `Case obj Is Nothing`.

Why the vba dialect rather than upstream's vb6 one: measured with the
CLI on kellyethridge/VBCorLib @ 100acbbae0c0 (655 VB6 files),
Kr00l/VBCCR @ 4b86c5c2c075 (161 VB6 files, 70 .ctl) and upstream's 481
VBA examples, procedures outside ERROR nodes are 14,084/14,084,
12,385/1,552 and 8,104/6,573 (vba/vb6). vb6 rejects the PtrSafe/LongPtr
forms dual-target VB6 keeps inside #If VBA7 blocks and then loses whole
files; vba never sees fewer procedures in any file. vb6's parser.c is
also 75.8 MB against 48.3 MB.

Provenance by hash: common/define-grammar.js is upstream's blob
5e1f7d7; vba/grammar.js minus its 40-line provenance header is
upstream's blob 9247dbbfe166. Two independent generations are
byte-identical, and identical to a generation from upstream's own layout
and tree-sitter.json. The generated node-types.json is upstream's
checked-in file byte-identical (500e635), and the parse-table counts
equal upstream's own generated parser. Review: 0 non-ASCII bytes, 0
unmatched characters on full tokenization, one comment (the generator
stamp), no string literal with a space, no external scanner, no call
outside the tree-sitter macros. No upstream script, package hook or
binary was run or vendored.

- internal/cbm/vendored/grammars/vba/: the generated parser.c, the
  tree_sitter/ headers (byte-identical to copies already vendored) and
  upstream's MIT LICENSE byte-identical.
- internal/cbm/grammar_vba.c: compiles and links tree_sitter_vba. No
  CBMLanguage, extension or alias routes to it yet; extraction adds them
  under the vb6/vba names (vb and visualbasic mean VB.NET).
- MANIFEST.md (fork note with the measurements and known gaps, forks
  table, 164 grammars, ABI recount), THIRD_PARTY.md (fork list and a vba
  section), scripts/vendored-checksums.txt, and the FORKS entry in
  scripts/audit-license-provenance.py.
- README.md and the Chocolatey nuspec say 164 languages: the
  language-count contract defines the count as the vendored grammar
  directories.
- tests/test_parse_coverage.c: a dual-target VB6 class module (IDE class
  header, Declare in both #If VBA7 branches, Property Get/Let/Set,
  Implements, events, a label, bare calls) parses with no ERROR node and
  every bare call keeps its callee; a CRLF VB6 form parses clean; broken
  source is still flagged. The class case fails on v0.13.0 and on the vb6
  dialect.

Refs DeusData#721, DeusData#2564

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Brings the branch up to main 08ab4a0 (DeusData#2966, DeusData#2969, DeusData#2970, DeusData#2977,
DeusData#2978, DeusData#2981).

scripts/vendored-checksums.txt: main (DeusData#2969) vendored the VB6/VBA
grammar while this branch edits MANIFEST.md. Regenerated with
scripts/security-vendored.sh --update (1101 files); only the
MANIFEST.md line differs from main.

Everything else merged without conflict and was checked by hand:
- MANIFEST.md keeps main's vba, sql and jsonnet text; only this
  branch's vbnet registration sentence and custom-handling row differ.
- The language count stays 164 (vendored grammar directories); the
  language-count contract passes. VB.NET adds a language name to the
  README list, not a grammar.
- DeusData#2977's receiver gate is an allowlist (gate_scope); VB.NET is not on
  it, so it stays GATE_OFF, and receiver_chain_admits is unchanged.
- The VB.NET case-insensitive retry still hooks
  cbm_registry_resolve_lang, cbm_registry_resolve_ref and
  resolve_call_lang; the Field-call USAGE policy and the enum-member
  gate are intact.
- DeusData#2981's linear-growth rewrites in extract_defs.c touch no VB.NET
  function; find_class_body still decides VB.NET first.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData

Copy link
Copy Markdown
Owner

One more maintainer merge of main into this branch (2fb3ea1), because main moved on and conflicted again. Nothing changes for VB.NET:

  1. Main vendored the VB6/VBA grammar (chore(vendor): vendor the VB6/VBA tree-sitter grammar #2969) next to the vbnet grammar this branch registers. The grammar manifest keeps main's new rows and this branch's vbnet text, and the vendored checksums were regenerated (only the manifest's line differs from main). The language count stays 164.
  2. Main's new receiver check for member calls (fix(registry): refuse name-only member calls nothing ties to the receiver #2977) applies only to the languages it was measured on, and VB.NET is not one of them, so VB.NET calls resolve as before. The case-insensitive retry, the field-indexing rule and the enum-member rule from the earlier commits are all still in place.
  3. Main's speed-ups in the extractor (fix, perf: C# operator nodes, C increment writes and linear growth in extraction and eight language passes #2981) do not touch the VB.NET code paths.

On staxrip and PolicyPlus the graphs are byte-identical before and after the merge.

Please pull before pushing again.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request language-request Request for new language support parsing/quality Graph extraction bugs, false positives, missing edges priority/backlog Valuable contribution, lower scheduling urgency; review when maintainer capacity opens. security Security vulnerabilities, hardening

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: add VB.NET (.vb) tree-sitter structural extraction

2 participants