feat(lang): TwinCAT / CODESYS (IEC 61131-3 Structured Text) support

CONTRIBUTING.md

@@ -100,6 +100,17 @@ Languages like **Dockerfile**, **docker-compose**, **Kubernetes manifests**, and
 - Register the pass in `pipeline.c`.
 - Add tests in `tests/test_pipeline.c` following the `TEST(infra_is_dockerfile)` and `TEST(k8s_extract_manifest)` patterns.
 
+### Transform-Only Container Languages
+
+Some formats are just another registered language wrapped in a different container — TwinCAT's PLC XML (`.TcPOU`/`.TcDUT`/`.TcGVL`/`.TcIO`) and CODESYS/PLCopen TC6 XML exports are both IEC 61131-3 Structured Text underneath, the same way ObjectScript Studio Export XML is UDL underneath. These need neither a new grammar nor an infra-pass extractor: the container is transcoded to the target language's source text and re-extracted through the normal pipeline.
+
+**When adding a new transform-only container language:**
+- Add the `CBM_LANG_<LANG>` enum value in `internal/cbm/cbm.h`; add **no** row in `lang_specs.c` — a comment there instead (see `CBM_LANG_OBJECTSCRIPT_EXPORT`/`CBM_LANG_PLCOPEN_XML`) so the absence reads as deliberate. `cbm_lang_spec()` must return `NULL` for it.
+- Write a transcoder in `internal/cbm/` that turns the container into the target language's source text (`iris_export_xml.c`, `twincat_xml.c` are the examples).
+- Write an aggregating extractor in `src/pipeline/pass_definitions.c` that runs the transcoder, re-extracts each generated unit as the target language, and composes the results into one `CBMFileResult`.
+- Add the dispatch arm in **both** `pass_definitions.c` and `pass_parallel.c` — the sequential and parallel chains must stay symmetric, or the omission breaks only parallel indexing (>50 files), silently.
+- Add a `TRANSFORM_ONLY(...)` row and update the partition counts in `tests/repro/repro_language_registry.c`, and add the language to the transform-only skip list in `tests/repro/repro_call_node_manifest.c`.
+
 ## Commit Format
 
 Use conventional commits: `type(scope): description`
@@ -118,6 +129,37 @@ Examples: `fix(store): set busy_timeout before WAL`, `feat(cli): add --progress
 
 ## Pull Request Guidelines
 
+### Self-Maintained Grammar Forks
+
+Most vendored grammars are byte-for-byte upstream and never change after
+vendoring. A few are forks we maintain because upstream does not cover the
+dialect our users actually write — `iec_st` is one: the upstream IEC 61131-3
+grammar is standard-only, and standard-only does not parse production TwinCAT.
+
+The fork lives beside the other first-party grammars:
+
+1. Edit `tools/tree-sitter-iec-st/grammar.js`. Keep each dialect rule commented
+   with the real construct that motivated it, and prefer narrowing a rule over
+   widening it (an instance-argument list accepted after *any* type specifier
+   makes `s : STRING(255)` ambiguous between a string length and an argument).
+2. Regenerate in place: `cd tools/tree-sitter-iec-st && tree-sitter generate`
+   (CLI 0.26.x). Check `#define LANGUAGE_VERSION` in the generated
+   `src/parser.c` — the runtime ceiling is ABI 15, and a newer CLI that emits
+   ABI 16 must not be vendored.
+3. Copy `src/parser.c` and `src/scanner.c` into
+   `internal/cbm/vendored/grammars/iec_st/`, then refresh the digest manifest
+   with `scripts/security-vendored.sh --update`.
+4. Note the rules you added in `internal/cbm/vendored/grammars/MANIFEST.md`, so
+   a future re-vendor from upstream re-applies them instead of silently
+   reverting the dialect support.
+
+`Makefile.cbm` declares the vendored parser/scanner as explicit prerequisites of
+the `iec_st` grammar objects. `grammar_*.c` is a one-line wrapper that
+`#include`s them, and make cannot see through an include: without that
+dependency a regenerated grammar links a stale object, and the binary keeps the
+old grammar while the source tree shows the new one — with no error anywhere.
+Any future fork needs the same three lines (prod/test/tsan).
+
 ### Before You Write Code
 
 - **Open an issue first — always.** Every PR must reference a tracking issue (`Fixes #N` or `Closes #N`). Describe what you want to change and why. Wait for maintainer feedback before implementing. PRs without a prior issue discussion will be closed.

Makefile.cbm

@@ -250,6 +250,8 @@ EXTRACTION_SRCS = \
     $(CBM_DIR)/lang_specs.c \
     $(CBM_DIR)/macro_table.c \
     $(CBM_DIR)/iris_export_xml.c \
+    $(CBM_DIR)/twincat_xml.c \
+    $(CBM_DIR)/plcopen_xml.c \
     $(CBM_DIR)/service_patterns.c
 
 # LSP resolvers (compiled as one unit via lsp_all.c)
@@ -356,7 +358,8 @@ PIPELINE_SRCS = \
     src/pipeline/pass_complexity.c \
     src/pipeline/pass_cross_repo.c \
     src/pipeline/artifact.c \
-    src/pipeline/pass_pkgmap.c
+    src/pipeline/pass_pkgmap.c \
+    src/pipeline/pass_tcproj.c
 
 # SimHash / MinHash module
 SIMHASH_SRCS = src/simhash/minhash.c
@@ -990,6 +993,21 @@ PP_OBJ_PROD = $(BUILD_DIR)/prod_preprocessor.o
 $(BUILD_DIR)/prod_%.o: $(CBM_DIR)/%.c | $(BUILD_DIR)
 	$(CC) $(GRAMMAR_CFLAGS) -c -o $@ $<
 
+# grammar_*.c is a one-line wrapper that #includes its vendored parser.c and
+# scanner.c, and make cannot see through that include. For every other grammar
+# the vendored bytes never change, so the pattern rule above suffices. iec_st is
+# a self-maintained fork (tools/tree-sitter-iec-st): regenerating it DOES change
+# the vendored parser, and without this explicit dependency the stale object is
+# silently relinked — the binary keeps the old grammar while the source tree
+# shows the new one. Declared for all three variants (prod/test/tsan).
+IEC_ST_VENDORED_DEPS = \
+    $(CBM_DIR)/vendored/grammars/iec_st/parser.c \
+    $(CBM_DIR)/vendored/grammars/iec_st/scanner.c
+
+$(BUILD_DIR)/prod_grammar_iec_st.o: $(IEC_ST_VENDORED_DEPS)
+$(BUILD_DIR)/grammar_iec_st.o: $(IEC_ST_VENDORED_DEPS)
+$(BUILD_DIR)/tsan_grammar_iec_st.o: $(IEC_ST_VENDORED_DEPS)
+
 $(BUILD_DIR)/prod_ts_runtime.o: $(TS_RUNTIME_DEPS) | $(BUILD_DIR)
 	$(CC) $(GRAMMAR_CFLAGS) -c -o $@ $<
 

README.md

@@ -4,7 +4,7 @@
 [![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)
 [![CI](https://img.shields.io/github/actions/workflow/status/DeusData/codebase-memory-mcp/dry-run.yml?label=CI)](https://github.com/DeusData/codebase-memory-mcp/actions/workflows/dry-run.yml)
 [![Tests](https://img.shields.io/badge/tests-6768_passing-brightgreen)](https://github.com/DeusData/codebase-memory-mcp)
-[![Languages](https://img.shields.io/badge/languages-158-orange)](https://github.com/DeusData/codebase-memory-mcp)
+[![Languages](https://img.shields.io/badge/languages-159-orange)](https://github.com/DeusData/codebase-memory-mcp)
 [![Hybrid LSP](https://img.shields.io/badge/Hybrid_LSP-10_languages-blue)](#hybrid-lsp)
 [![Agents](https://img.shields.io/badge/agent_surfaces-43-purple)](https://github.com/DeusData/codebase-memory-mcp)
 [![Pure C](https://img.shields.io/badge/pure_C-no_language_runtime-blue)](https://github.com/DeusData/codebase-memory-mcp)
@@ -16,7 +16,7 @@
 
 **The fastest and most efficient code intelligence engine for AI coding agents.** Full-indexes an average repository in milliseconds, the Linux kernel (28M LOC, 75K files) in 3 minutes. Answers structural queries in under 1ms. Ships as a native executable with a small verified runtime-asset set for macOS, Linux, and Windows — download, run `install`, done.
 
-High-quality parsing through [tree-sitter](https://tree-sitter.github.io/tree-sitter/) AST analysis across all 158 languages, enhanced with [**Hybrid LSP** semantic type resolution](#hybrid-lsp) for Python, TypeScript / JavaScript / JSX / TSX, PHP, C#, Go, C, C++, Java, Kotlin, Rust, and Perl — producing a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links. 15 MCP tools. No language runtime, hosted service, or API key. Plug and play across 43 supported automatic/conditional client surfaces.
+High-quality parsing through [tree-sitter](https://tree-sitter.github.io/tree-sitter/) AST analysis across all 159 languages, enhanced with [**Hybrid LSP** semantic type resolution](#hybrid-lsp) for Python, TypeScript / JavaScript / JSX / TSX, PHP, C#, Go, C, C++, Java, Kotlin, Rust, and Perl — producing a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links. 15 MCP tools. No language runtime, hosted service, or API key. Plug and play across 43 supported automatic/conditional client surfaces.
 
 > **Research** — The design and benchmarks behind this project are described in the preprint [*Codebase-Memory: Tree-Sitter-Based Knowledge Graphs for LLM Code Exploration via MCP*](https://arxiv.org/abs/2603.27277) (arXiv:2603.27277). Evaluated across 31 real-world repositories: 83% answer quality, 10× fewer tokens, 2.1× fewer tool calls vs. file-by-file exploration.
 
@@ -32,7 +32,7 @@ High-quality parsing through [tree-sitter](https://tree-sitter.github.io/tree-si
 
 - **Extreme indexing speed** — Linux kernel (28M LOC, 75K files) in 3 minutes. RAM-first pipeline: LZ4 compression, in-memory SQLite, fused Aho-Corasick pattern matching. Memory released after indexing.
 - **Plug and play** — native executable plus authenticated release-owned assets for macOS (arm64/amd64), Linux (arm64/amd64), and Windows (amd64). The native install needs no Docker, language runtime, or API keys. Download → `install` → restart agent → done.
-- **158 languages** — vendored tree-sitter grammars compiled into the binary. Nothing to install, nothing that breaks.
+- **159 languages** — vendored tree-sitter grammars compiled into the binary. Nothing to install, nothing that breaks.
 - **120x fewer tokens** — 5 structural queries: ~3,400 tokens vs ~412,000 via file-by-file search. One graph query replaces dozens of grep/read cycles.
 - **43 supported automatic/conditional client surfaces** — `install` configures detected clients and safely activates conditional clients only when their documented platform, marker, or explicit existing config path is present. See [Multi-Agent Support](#multi-agent-support) for the complete matrix and manual/UI-only boundaries.
 - **Built-in graph visualization** — 3D interactive UI at `localhost:9749`, served from the binary itself.
@@ -224,7 +224,7 @@ The install script placed beside the binary is **reported, not deleted** — uni
 - `SEMANTICALLY_RELATED` (vocabulary-mismatch, same-language, score ≥ 0.80)
 
 ### Indexing pipeline
-- **158 vendored tree-sitter grammars** compiled into the binary
+- **159 vendored tree-sitter grammars** compiled into the binary
 - **Generic package / module resolution** — bare specifiers like `@myorg/pkg`, `github.com/foo/bar`, `use my_crate::foo` resolved via manifest scanning (`package.json`, `go.mod`, `Cargo.toml`, `pyproject.toml`, `composer.json`, `pubspec.yaml`, `pom.xml`, `build.gradle`, `mix.exs`, `*.gemspec`)
 - **Infrastructure-as-code indexing** — Dockerfiles, Kubernetes manifests, Kustomize overlays as graph nodes
 - **[Hybrid LSP semantic type resolution](#hybrid-lsp)** for Python, TypeScript / JavaScript / JSX / TSX, PHP, C#, Go, C, C++, Java, Kotlin, Rust, and Perl — a lightweight C implementation of language type-resolution algorithms, structurally inspired by and compatible with major language servers including tsserver / typescript-go, pyright, gopls, Roslyn, Eclipse JDT, and rust-analyzer (parameter binding, return-type inference, generic substitution, JSX component dispatch, JSDoc inference for plain JS files, namespace + trait + late-static-binding resolution for PHP, file-scoped namespaces + records + LINQ method syntax for C#, class-hierarchy + overload + lambda resolution for Java, extension-function + scope-function resolution for Kotlin, trait-method + UFCS resolution for Rust)
@@ -754,22 +754,39 @@ codebase-memory-mcp ships a **lightweight C implementation of language type-reso
 
 **Two-layer architecture:**
 
-1. **Tree-sitter pass** — fast, syntactic, runs for every one of the 158 languages. Extracts definitions, calls, imports.
+1. **Tree-sitter pass** — fast, syntactic, runs for every one of the 159 languages. Extracts definitions, calls, imports.
 2. **Hybrid LSP pass** — type-aware, runs above the tree-sitter pass per-language. Refines call edges using the import graph plus a per-file or pre-built cross-file definition registry. Languages without a Hybrid LSP pass yet fall back to textual resolution, so you always get *some* answer.
 
 The result is a knowledge graph accurate enough to drive `trace_path` across packages, inheritance hierarchies, and stdlib calls — without paying for a language server process per project.
 
 ## Language Support
 
-158 languages, all parsed via vendored tree-sitter grammars compiled into the binary. Benchmarked against 64 real open-source repositories (78 to 49K nodes):
+159 languages, all parsed via vendored tree-sitter grammars compiled into the binary. Benchmarked against 64 real open-source repositories (78 to 49K nodes):
 
 | Tier | Score | Languages |
 |------|-------|-----------|
 | **Excellent** (>= 90%) | | Lua, Kotlin, C++, Perl, Objective-C, Groovy, C, Bash, Zig, Swift, CSS, YAML, TOML, HTML, SCSS, HCL, Dockerfile |
 | **Good** (75-89%) | | Python, TypeScript, TSX, Go, Rust, Java, R, Dart, JavaScript, Erlang, Elixir, Scala, Ruby, PHP, C#, SQL |
 | **Functional** (< 75%) | | OCaml, Haskell |
 
-Also supported (not yet benchmarked): Ada, Agda, Apex, Assembly (NASM), Astro, AWK, Beancount, BibTeX, Bicep, Bitbake, Blade, Cairo, Cap'n Proto, Clojure, CMake, COBOL, Common Lisp, Crystal, CSV, CUDA, D, Devicetree, Diff, .env, Elm, Emacs Lisp, F#, Fennel, Fish, FORM, Fortran, FunC, GDScript, .gitattributes, .gitignore, Gleam, GLSL, GN, Go module, Go template, GraphQL, Hare, HLSL, Hyprlang, INI, ISPC, Janet, Jinja2, JSDoc, JSON, JSON5, Jsonnet, Julia, Just, Kconfig, KDL, Lean 4, Linker Script, Liquid, LLVM IR, Luau, Magma, Makefile, Markdown, MATLAB, Mermaid, Meson, Move, Nickel, Nim, Nix, Odin, Pascal, Pkl, PO (gettext), Pony, PowerShell, Prisma, .properties, Protobuf, Puppet, PureScript, Racket, Regex, requirements.txt, ReScript, RON, reStructuredText, Scheme, Slang, Smali, Smithy, Solidity, SOQL, SOSL, Squirrel, SSH config, Starlark, Svelte, Sway, SystemVerilog, TableGen, Tcl, Teal, Templ, Thrift, TLA+, Typst, Verilog, VHDL, Vim script, Vue, WGSL, WIT, Wolfram, XML, Zsh.
+Also supported (not yet benchmarked): Ada, Agda, Apex, Assembly (NASM), Astro, AWK, Beancount, BibTeX, Bicep, Bitbake, Blade, Cairo, Cap'n Proto, Clojure, CMake, COBOL, Common Lisp, Crystal, CSV, CUDA, D, Devicetree, Diff, .env, Elm, Emacs Lisp, F#, Fennel, Fish, FORM, Fortran, FunC, GDScript, .gitattributes, .gitignore, Gleam, GLSL, GN, Go module, Go template, GraphQL, Hare, HLSL, Hyprlang, IEC 61131-3 Structured Text (TwinCAT / CODESYS), INI, ISPC, Janet, Jinja2, JSDoc, JSON, JSON5, Jsonnet, Julia, Just, Kconfig, KDL, Lean 4, Linker Script, Liquid, LLVM IR, Luau, Magma, Makefile, Markdown, MATLAB, Mermaid, Meson, Move, Nickel, Nim, Nix, Odin, Pascal, Pkl, PO (gettext), Pony, PowerShell, Prisma, .properties, Protobuf, Puppet, PureScript, Racket, Regex, requirements.txt, ReScript, RON, reStructuredText, Scheme, Slang, Smali, Smithy, Solidity, SOQL, SOSL, Squirrel, SSH config, Starlark, Svelte, Sway, SystemVerilog, TableGen, Tcl, Teal, Templ, Thrift, TLA+, Typst, Verilog, VHDL, Vim script, Vue, WGSL, WIT, Wolfram, XML, Zsh.
+
+**TwinCAT / IEC 61131-3 Structured Text.** Plain ST sources (`.st`, `.iecst`) parse directly through the `iec_st` grammar, a self-maintained fork (`tools/tree-sitter-iec-st`) of the standard-only upstream grammar. The fork exists because standard-only does not parse production TwinCAT: it adds the dialect forms real projects use — access modifiers after the POU keyword (`FUNCTION_BLOCK PUBLIC FB_X`), wildcard located addresses (`AT %I*`), bit-in-word access (`wError.0`), a base type after an enumerator list (`(Idle, Run) UINT;`), `REFERENCE TO`, function-block instance argument lists (`fb : FB_T()`), the omitted terminator after a block-shaped `TYPE`, attribute pragmas above members, `__TRY/__CATCH/__FINALLY/__ENDTRY`, and `;`-terminated interface prototypes. Measured on a 130-file production solution, partial parses fell from 49 files to 8. TwinCAT 3 PLC XML containers (`.TcPOU`, `.TcDUT`, `.TcGVL`, `.TcIO`) are transcoded to textual ST by recomposing the `<Declaration>`/`<Implementation><ST>` CDATA before extraction; CODESYS/PLCopen TC6 XML exports (`.xml`, detected by a content sniff for the `plcopen.org/xml` namespace) are transcoded the same way, synthesizing declarations from the export's structured `<inputVars>`/`<outputVars>`/`<variable>` lists. `.plcproj`/`.tsproj` project files are indexed by a dedicated pass instead. POU constructs map onto the graph as:
+
+| IEC 61131-3 construct | Graph node / edge |
+|---|---|
+| `FUNCTION_BLOCK`, `PROGRAM` | `Class` |
+| `FUNCTION` | `Function` |
+| `METHOD`, `PROPERTY` (Get/Set bodies nested) | `Method` (`DEFINES_METHOD` from its owner) |
+| `INTERFACE` | `Interface` |
+| `TYPE` (DUT struct/enum) | `Type` |
+| GVL entry | `Variable` |
+| `EXTENDS` | `INHERITS` |
+| `IMPLEMENTS` | `IMPLEMENTS` |
+
+`ACTION` has no equivalent in the `iec_st` grammar itself — plain `.st`/`.iecst` sources never produce one. TwinCAT and PLCopen containers synthesize it as a parameterless `METHOD` before parsing (call sites use the same `inst.Name()` shape either way), so it reaches the graph as `Method` only through those two container transcoders.
+
+Each `.plcproj` becomes a `Package` node with `DEPENDS_ON` edges to its library references (e.g. `Tc2_Standard`) and `CONTAINS_FILE` edges to its member files; a `.tsproj` gets `DEPENDS_ON` edges to the PLC projects it references and `CONFIGURES` edges to their `.xti` device descriptions. Known limitations: cross-file references to GVL variables and DUT types resolve by name rather than by declared type, so an unqualified global shared by two projects can bind loosely; a function-block instance call (`fbInst()`) resolves on the instance identifier rather than the block's type; inside a TwinCAT XML container the POU's own implementation-body line numbers can land after its methods in the extracted result (declaration lines are exact); and in PLCopen exports, array- and pointer-typed variables are dropped rather than approximated (the grammar rejects bare `ARRAY`/`POINTER` identifiers), and resource/instance-level `<globalVars>` outside a POU's `<interface>` are not transcoded.
 
 ## Architecture
 
@@ -787,7 +804,7 @@ src/
   traces/             Runtime trace ingestion
   ui/                 Local HTTP server + verified external 3D-UI asset pack
   foundation/         Platform abstractions (threads, filesystem, logging, memory)
-internal/cbm/         Vendored tree-sitter grammars (158 languages) + AST extraction engine
+internal/cbm/         Vendored tree-sitter grammars (159 languages) + AST extraction engine
 ```
 
 ## Security

internal/cbm/cbm.h

@@ -174,6 +174,9 @@ typedef enum {
     CBM_LANG_OBJECTSCRIPT_UDL,     // InterSystems ObjectScript UDL (.cls class files)
     CBM_LANG_OBJECTSCRIPT_ROUTINE, // InterSystems ObjectScript routine (.mac/.int/.rtn/.inc)
     CBM_LANG_OBJECTSCRIPT_EXPORT,  // InterSystems Studio Export XML (<Export generator="Cache">)
+    CBM_LANG_IEC_ST,               // IEC 61131-3 Structured Text (.st/.iecst — PLC languages)
+    CBM_LANG_TWINCAT,              // TwinCAT 3 PLC XML container (.TcPOU/.TcDUT/.TcGVL/.TcIO)
+    CBM_LANG_PLCOPEN_XML,          // CODESYS/PLCopen TC6 XML export (<project xmlns=".../tc6_...">)
     CBM_LANG_COUNT
 } CBMLanguage;