Repository navigation
Description
Summary
Indexing a repository whose database/ folder holds Microsoft SQL Server (T-SQL) migration scripts: 11 of 14 .sql files are reported parse_partial and 3 are parse_unusable. The bundled .sql grammar appears to be the PostgreSQL family, so ordinary T-SQL constructs have no production and the files contribute almost nothing to the graph (a single Table node is all that survives).
This is not an encoding or file-size problem: the files are 20–950 bytes of plain ASCII/UTF-8 with no BOM, and skipped: [] (no ignore rules involved).
Environment
- codebase-memory-mcp 0.12.1, Windows 11, PowerShell 7
index_mode: full,auto_watch: true- Files: ASCII/UTF-8, no BOM, 20–950 bytes each
Minimal repro
One construct per file, each indexed in isolation in a throwaway project. Rows 00 and 10 are controls.
| # | File content | Result |
|---|---|---|
| 00 | CREATE TABLE sample (id INT NOT NULL PRIMARY KEY, url VARCHAR(200) NOT NULL); |
clean |
| 10 | CREATE TABLE sample (id SERIAL PRIMARY KEY, name TEXT); + grant select on sample to public; |
clean (PG parses fine) |
| 01 | SELECT 1 / GO / SELECT 2 |
parse_partial — line 2 only |
| 02 | SELECT * FROM [dbo].[sample]; |
parse_unusable |
| 11 | SELECT * FROM dbo.sample; |
clean |
| 03 | USE [app] / GO |
parse_unusable |
| 04 | CREATE DATABASE [app] / GO |
parse_partial — line 1 |
| 09 | DROP DATABASE [app] / GO |
parse_partial — line 1 |
| 05 | CREATE TABLE [dbo].[sample] with id INT NOT NULL IDENTITY PRIMARY KEY, url NVARCHAR(200) NOT NULL, createdUtc DATETIME2 NOT NULL, then GO |
parse_partial — line 3 (the IDENTITY column), line 7 (GO) |
| 07 | ALTER TABLE [dbo].[sample] newline ADD maxConnections INT |
parse_partial — line 1 |
| 13 | ALTER TABLE sample ADD maxConnections INT; |
clean |
| 06 | ALTER TABLE [dbo].[sample] ALTER COLUMN companyId INT NULL; |
parse_unusable |
| 12 | ALTER TABLE sample ALTER COLUMN companyId INT NULL; |
parse_unusable |
| 08 | ALTER TABLE [dbo].[sample] newline ADD CONSTRAINT ck_batch CHECK (NOT (method = 'GET' AND batchSize > 1)); |
parse_unusable |
In the real repository check_index_coverage returns parse_partial with ranges scattered across lines 1–8 of 8-line files, plus parse_unusable for whole files. Probe 09 (2 lines) is also flagged, so this is not a complexity, size or encoding threshold.
Key detail: GO and the brackets are not the whole story
Removing just the batch separators and the bracketed identifiers turns a realistic script completely clean:
USE app
CREATE TABLE dbo.sample
(
id INT NOT NULL IDENTITY PRIMARY KEY,
url NVARCHAR(200) NOT NULL
)
ALTER TABLE dbo.sample ADD maxConnections INT...but ALTER COLUMN fails even without brackets — file 12 above is still parse_unusable. So bracket folding alone would not be sufficient.
Expected
T-SQL .sql files should parse without being flagged, ideally surfacing ALTER TABLE … ADD COLUMN and ALTER TABLE … ALTER COLUMN as column/table lineage the way the PostgreSQL path does (cf. #1755).
Suggested fix
- Pre-normalise before tree-sitter: split on
GOat line start and drop it; map[identifier]to"identifier". - Add a lenient production (or error-recovery rule) for
ALTER TABLE <table> ALTER COLUMN <col> <type> [NULL | NOT NULL]. - Longer term, a real T-SQL dialect — same shape as the request in Add Oracle PL/SQL support #669 for PL/SQL.
Question
Is there any existing knob to select a SQL dialect? config list on 0.12.1 exposes only auto_index, auto_index_limit, auto_watch, doc_candidates, watcher_enabled, watch_non_git, ui-*, index_max_files, index_max_source_mb; the CBM_* environment variables contain nothing parser-, grammar- or encoding-related; and index_repository takes repo_path, mode, target_projects, name, persistence, doc_candidates — no dialect/language argument. If one exists, a pointer would be enough and this would be a docs issue rather than a parsing one.
Activity