diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index ba80844c9..389f38926 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -1289,6 +1289,18 @@ the public arrays at runtime. | `volume` | Dockerfile `VOLUME` paths | Container storage search symbol | | `workdir` | Dockerfile `WORKDIR` paths | Container filesystem search symbol | +For C#, `test.method` ownership is determined from the complete leading attribute +block rather than from the immediately preceding physical line. The bounded scanner +reuses the C# lexical masking pass, so multiline attribute arguments and consecutive +attribute lists are supported while generic attribute type arguments stay within their +own item and brackets or test-like names in comments and normal, verbatim, raw, +interpolated, and character literals are ignored. Explicit +`return:`, `assembly:`, and `module:` targets do not classify a method. A declaration- +context gate tracks expression-initializer braces and rejects bracket-led collection +expressions. Only shaped ordinary method declarations with return types can receive the +test taxonomy; fields, indexers, and constructors (including implicit-return partial +constructors) consume their attribute ownership without being reclassified. + `SymbolKindCatalog.CompatibilityKindFamilies` maps both `typealias` and `type_parameter` to the broad `type` family for consumers that only understand the older coarse taxonomy. The persisted `kind` remains semantic, and `--kind` @@ -5363,6 +5375,17 @@ test も更新してください。公開 array を実行時に変更しては | `volume` | Dockerfile `VOLUME` path | container storage search symbol | | `workdir` | Dockerfile `WORKDIR` path | container filesystem search symbol | +C# の `test.method` 所有権は、直前の物理行だけではなく、先頭に連続する attribute +block 全体から判定します。上限付き scanner は C# の字句 masking pass を再利用するため、 +複数行の attribute 引数と連続する attribute list を扱い、generic attribute の型引数を +同じ item 内に保ちつつ、comment、通常文字列、verbatim 文字列、raw 文字列、interpolated +文字列、文字 literal 内の bracket や test 風の名前を無視します。明示的な `return:`、 +`assembly:`、`module:` target は method を `test.method` に分類しません。declaration +context gate は expression initializer の brace を追跡して行頭 bracket の collection +expression を拒否します。return type を持つ shaped な通常 method declaration だけが test +taxonomy を受け取り、field、indexer、constructor（暗黙 return の partial constructor を含む） +は attribute 所有権を消費しますが、再分類しません。 + 古い粗い taxonomy だけを理解する consumer 向けに、 `SymbolKindCatalog.CompatibilityKindFamilies` は `typealias` と `type_parameter` の両方を広い `type` family へ mapping します。永続化される diff --git a/TESTING_GUIDE.md b/TESTING_GUIDE.md index 10b997c98..5bf45e37e 100644 --- a/TESTING_GUIDE.md +++ b/TESTING_GUIDE.md @@ -114,6 +114,7 @@ Candidate-ordered parallel-index recovery tests must prove that the fatal result Extractor coverage is split by language or feature area with partial test classes, while shared helpers remain on the root `SymbolExtractorTests` / `ReferenceExtractorTests` parts. YAML sequence-hierarchy fixtures must keep sequence items path-only, count empty/scalar/alias items in stable indexes, preserve anchor and block-scalar boundaries, assert logical qualified paths separately from nearest emitted symbol parents, and prove a normal full scan refreshes unchanged YAML rows stamped with the prior extractor contract. C# declaration-boundary regressions should pair a direct extractor fixture with a real-index `symbols --exact-name` query. Keep invocation and parameter continuations beside valid multi-line methods, constructors, delegates, and local functions so both false-positive rejection and declaration ranges remain observable. Scope-continuation fixtures must also place unmatched delimiters in character/string literals and comments before later methods, then assert that the later methods remain definitions rather than unresolved declaration-shaped calls. + C# test-attribute ownership coverage pairs `SymbolExtractorCSharpTests` with `QueryCommandRunnerIssue5192Tests`. Keep multiline and same-line attribute lists, comments, qualified xUnit names, MSTest/NUnit forms, generic attribute type arguments containing test-like names, literal-contained brackets, direct and initializer-brace collection expressions, attributed fields/indexers/ordinary and partial constructors, ignored targets, member boundaries, and fail-closed budgets in the direct fixture; the persisted fixture must retain `test.method` through outline, symbols, inspect, unused, and `--exclude-tests` behavior while non-test methods remain visible (#5192, regression of #1961). C# explicit-interface identity coverage pairs extractor assertions for methods, properties, events, and indexers with persisted exact qualified/unqualified queries, fold rewrite validation, inspect/outline checks, and LSP definition/reference scoping. Include multiple and inherited interfaces, generic arity, a same-named public member, and a qualified return type that must not be mistaken for an explicit-interface qualifier. C# callable-containment fixtures should cover block-bodied test methods, local and nested local functions, named lambdas, expression-bodied members, and nested types together, asserting both symbol parents and call-reference containers. Local-function identity coverage additionally keeps pre/post-declaration calls, nested and sibling blocks, overloads, parameter/local delegate shadowing, method groups, identical enclosing names in unrelated types, incomplete ranges, persisted graph/query/inspect/impact behavior, and LSP definition/reference parity in one coordinated contract (#5188, extending #4840). C# switch-expression returned-lambda coverage pairs a direct `ReferenceExtractorCSharpTests` regression with an `IndexCommandRunnerFullScanTests` `--rebuild` graph fixture. Keep expression/block, parenthesized, multiline, nested-switch, async, qualified/unqualified, and non-lambda arms together; assert raw call rows, enclosing containers, exact locations, callers/callees, complete graph readiness, and negative positional-pattern/enum controls including guarded, nested, and contextual-keyword-designation patterns with a following-line arrow (#5085). @@ -1252,6 +1253,7 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests" extractor のカバレッジは言語または機能領域ごとの partial test class に分割し、共有 helper は root 側の `SymbolExtractorTests` / `ReferenceExtractorTests` に残します。 YAML sequence hierarchy の fixture では、sequence item を path 専用のまま保ち、empty / scalar / alias item を安定した index に数え、anchor と block scalar の境界を維持し、論理 qualified path と最寄りの出力済み symbol parent を分けて検証し、旧 extractor contract が stamp された未変更 YAML row を通常の full scan が更新することも確認してください。 C# の declaration boundary に関する regression では、extractor を直接呼ぶ fixture と、実 index に対する `symbols --exact-name` query を組み合わせてください。呼び出し・parameter の continuation と、正当な複数行 method、constructor、delegate、local function を同居させ、false positive の拒否と宣言 range の両方を観測可能にします。スコープ継続用のテストでは、後続メソッドより前の文字・文字列リテラルとコメントに対応しない区切り記号も置き、後続メソッドが未解決の宣言形式呼び出しではなく定義のままになることも検証してください。 + C# の test attribute 所有権 coverage は `SymbolExtractorCSharpTests` と `QueryCommandRunnerIssue5192Tests` を組み合わせます。直接 fixture では複数行 / 同一行の attribute list、comment、修飾 xUnit 名、MSTest / NUnit 形式、test 風の名前を含む generic attribute 型引数、literal 内の bracket、直接および initializer brace 内の collection expression、attribute 付き field / indexer / 通常 constructor / partial constructor、無視する target、member boundary、fail-closed budget を維持し、永続化 fixture では outline、symbols、inspect、unused、`--exclude-tests` の各動作で `test.method` が保たれつつ非 test method が表示されることを確認してください（#5192、#1961 の回帰）。 C# の明示的 interface identity coverage では、method、property、event、indexer の extractor assertion と、永続化後の修飾 / 非修飾の完全一致 query、fold rewrite 検証、inspect / outline、LSP の definition / reference scope を組み合わせます。複数および継承 interface、generic arity、同名 public member に加え、明示的 interface qualifier と誤認してはならない修飾 return type を含めてください。 C# generic outline 表示の coverage は、generic / 非 generic overload、extension / local / explicit-interface method、nested generic の tuple / nullable / array parameter type、constraint、overload を区別する modifier、修飾型 / 具体型名との衝突、escaped / Unicode identifier、attribute / 既定値 literal 内の区切り文字を、抽出と reader setup を共有する1つの `DbReaderTests` fixture に維持します。line 移動を検証する対の fixture は先頭改行以外を同一にし、canonical name / path が不変であることを assertion に含め、旧 index の `Name@line` fallback 用に不完全 signature を手動永続化した row も1件残してください（#4916）。 C# の位置 record に対する検索帰属 coverage は、extractor の座標 assertion、1つの indexed reader fixture、1つの CLI fixture を組み合わせます。record keyword と型名、attribute と component 型、base 引数、body member、同一行の通常 property、複数行 `record struct` component、正規化された verbatim identifier の列、言語未指定の exact grouping、multi-hit の focus 選択、focus の行・列出力、symbol grouping を共通契約に維持します。component 列を持たない旧 symbol では行ベースの fallback を維持してください。C# の座標維持のために comment padding を変更する場合、Java/Kotlin の record component comment が既存 signature を維持することも assertion してください（#5095）。 diff --git a/changelog.d/unreleased/5192.fixed.md b/changelog.d/unreleased/5192.fixed.md new file mode 100644 index 000000000..84ec39ba8 --- /dev/null +++ b/changelog.d/unreleased/5192.fixed.md @@ -0,0 +1,25 @@ +--- +category: fixed +issues: + - 5192 +affected: + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.Contracts.cs + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.DeclarationMetadata.cs + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractionPhases.cs + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.PatternEmission.cs + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.PatternEmissionFlow.cs + - tests/CodeIndex.Tests/SymbolExtractorCSharpTests.cs + - tests/CodeIndex.Tests/QueryCommandRunnerIssue5192Tests.cs + - tests/CodeIndex.Tests/IndexCommandRunnerFullScanTests.cs + - DEVELOPER_GUIDE.md + - TESTING_GUIDE.md +--- + +## English + +- **C# test methods keep their taxonomy across multiline attribute blocks (#5192, regression of #1961)** — `Fact`, `Theory`, MSTest, and NUnit markers now remain attached through multiline arguments, consecutive attribute lists, and intervening comments, while literals, generic attribute type arguments, direct or initializer-brace collection expressions, fields, indexers, constructors, ignored attribute targets, member boundaries, and bounded fail-closed inputs cannot spoof or leak `test.method` classification. The C# extractor contract advances so unchanged files in existing indexes are re-extracted. + +## 日本語 + +- **複数行の attribute block でも C# test method の taxonomy を維持するようになりました (#5192、#1961 の回帰)** — `Fact`、`Theory`、MSTest、NUnit の marker は、複数行引数、連続する attribute list、途中の comment を越えて正しい method に関連付けられ、literal、generic attribute 型引数、直接または initializer brace 内の collection expression、field、indexer、constructor、無視対象の attribute target、member boundary、上限を超えて fail-closed となる入力が `test.method` 分類を偽装または後続宣言へ漏出することを防ぎます。C# extractor contract も更新し、既存 index の未変更ファイルを再抽出します。 diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs index 51f1dd590..ede8fcc40 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs @@ -811,12 +811,13 @@ private static void NormalizeCSharpImplicitPartialConstructorReturnTypes(List= 0 + && declarationLineIndex < attributedDeclarationLines.Length + && attributedDeclarationLines[declarationLineIndex]; + } + + private static void ConsumeCSharpTestAttributePrefix( + bool[] attributedDeclarationLines, + int declarationLineIndex) + { + if (declarationLineIndex >= 0 + && declarationLineIndex < attributedDeclarationLines.Length + && attributedDeclarationLines[declarationLineIndex]) + { + // An attribute prefix belongs to the first emitted declaration after it. + // Delaying consumption until emission avoids losing ownership to an + // overlapping pattern candidate that was accepted but deduplicated. + attributedDeclarationLines[declarationLineIndex] = false; + } + } + + private sealed class CSharpTestAttributePrefixScanner { - var scannedAttributeLine = false; - for (var lineIndex = declarationLineIndex; lineIndex >= 0; lineIndex--) + private readonly StringBuilder _attributeName = new(64); + private bool _inAttributeSection; + private bool _pendingAttributePrefix; + private bool _blockHasTestAttribute; + private bool _sectionHasTestAttribute; + private bool _sectionTargetIgnored; + private bool _itemNameFinalized; + private bool _budgetExceeded; + private bool _canStartAttributePrefix = true; + private int _bracketDepth; + private int _parenthesisDepth; + private int _genericArgumentDepth; + private bool _inExpressionInitializer; + private int _expressionInitializerBraceDepth; + private int _codeParenthesisDepth; + private int _codeBracketDepth; + private int _blockLineCount; + private int _blockCharacterCount; + private int _blockItemCount; + + public bool ScanLine(string sanitizedLine) + { + var firstNonWhitespace = 0; + while (firstNonWhitespace < sanitizedLine.Length + && char.IsWhiteSpace(sanitizedLine[firstNonWhitespace])) + { + firstNonWhitespace++; + } + + if (!_inAttributeSection && !_pendingAttributePrefix) + { + if (firstNonWhitespace >= sanitizedLine.Length) + { + return false; + } + + if (sanitizedLine[firstNonWhitespace] != '[' + || !_canStartAttributePrefix) + { + UpdateCSharpAttributePrefixContext(sanitizedLine); + return false; + } + + ResetBlock(); + } + + CountLine(); + var cursor = 0; + while (cursor < sanitizedLine.Length) + { + var ch = sanitizedLine[cursor]; + if (!_inAttributeSection) + { + if (char.IsWhiteSpace(ch)) + { + CountCharacter(); + cursor++; + continue; + } + + if (ch == '[') + { + StartSection(); + CountCharacter(); + cursor++; + continue; + } + + var isAttributedDeclaration = !_budgetExceeded + && _pendingAttributePrefix + && _blockHasTestAttribute; + UpdateCSharpAttributePrefixContext(sanitizedLine.AsSpan(cursor)); + ResetBlock(); + return isAttributedDeclaration; + } + + CountCharacter(); + cursor++; + + if (ch == '[') + { + _bracketDepth++; + continue; + } + + if (ch == ']') + { + _bracketDepth--; + if (_bracketDepth == 0) + { + CompleteItem(); + if (!_sectionTargetIgnored) + _blockHasTestAttribute |= _sectionHasTestAttribute; + + _inAttributeSection = false; + _pendingAttributePrefix = true; + _parenthesisDepth = 0; + _genericArgumentDepth = 0; + } + + continue; + } + + if (_bracketDepth != 1) + continue; + + if (ch == '(') + { + if (_parenthesisDepth == 0) + FinalizeItemName(); + _parenthesisDepth++; + continue; + } + + if (ch == ')' && _parenthesisDepth > 0) + { + _parenthesisDepth--; + continue; + } + + if (_parenthesisDepth != 0) + continue; + + if (ch == '<') + { + if (_genericArgumentDepth == 0) + FinalizeItemName(); + _genericArgumentDepth++; + continue; + } + + if (ch == '>' && _genericArgumentDepth > 0) + { + _genericArgumentDepth--; + continue; + } + + if (_genericArgumentDepth != 0) + continue; + + if (ch == ',') + { + CompleteItem(); + continue; + } + + if (ch == ':' && IsTargetSpecifier(_attributeName, out var ignoreTarget)) + { + _sectionTargetIgnored = ignoreTarget; + _attributeName.Clear(); + _itemNameFinalized = false; + continue; + } + + if (!char.IsWhiteSpace(ch) && !_itemNameFinalized) + { + if (_attributeName.Length >= CSharpTestAttributeMaxNameCharacters) + { + _budgetExceeded = true; + } + else + { + _attributeName.Append(ch); + } + } + } + + return false; + } + + private void UpdateCSharpAttributePrefixContext(ReadOnlySpan sanitizedLine) { - var trimmed = lines[lineIndex].TrimStart(); + var trimmed = sanitizedLine.Trim(); if (trimmed.Length == 0) - return false; + return; - if (!trimmed.StartsWith('[')) + for (var cursor = 0; cursor < sanitizedLine.Length; cursor++) { - if (lineIndex == declarationLineIndex && !scannedAttributeLine) + var ch = sanitizedLine[cursor]; + if (ch == '(') + { + _codeParenthesisDepth++; continue; + } - return false; + if (ch == ')' && _codeParenthesisDepth > 0) + { + _codeParenthesisDepth--; + continue; + } + + if (ch == '[') + { + _codeBracketDepth++; + continue; + } + + if (ch == ']' && _codeBracketDepth > 0) + { + _codeBracketDepth--; + continue; + } + + if (!_inExpressionInitializer + && ch == '=' + && _codeParenthesisDepth == 0 + && _codeBracketDepth == 0 + && IsCSharpAssignmentOperator(sanitizedLine, cursor)) + { + _inExpressionInitializer = true; + continue; + } + + if (!_inExpressionInitializer) + continue; + + if (ch == '{') + { + _expressionInitializerBraceDepth++; + } + else if (ch == '}' && _expressionInitializerBraceDepth > 0) + { + _expressionInitializerBraceDepth--; + } + else if (ch == ';' && _expressionInitializerBraceDepth == 0) + { + _inExpressionInitializer = false; + } } - scannedAttributeLine = true; - if (CSharpLineHasTestMethodAttribute(trimmed)) - return true; + // A declaration attribute may begin at the file start or after a completed + // declaration/statement/body. A bracket-led line following an expression + // continuation (`=>`, `=`, `return`, an argument list, and so on), including + // an initializer brace, is instead a collection expression and must not create + // attribute ownership. + // declaration attribute は file 先頭または完了した宣言・statement・body の後に + // 開始できる。initializer brace を含む expression continuation 後の行頭 bracket は + // collection expression なので、attribute 所有権を作らない。 + _canStartAttributePrefix = !_inExpressionInitializer + && (trimmed[0] == '#' + || trimmed[^1] is ';' or '{' or '}'); + } - var remainderIndex = trimmed.LastIndexOf(']'); - if (remainderIndex < 0) - return false; + private static bool IsCSharpAssignmentOperator(ReadOnlySpan line, int index) + { + var previous = index > 0 ? line[index - 1] : '\0'; + var next = index + 1 < line.Length ? line[index + 1] : '\0'; + return next != '=' && previous is not ('=' or '!' or '<' or '>'); + } - var remainder = trimmed[(remainderIndex + 1)..].TrimStart(); - if (remainder.Length > 0) - return false; + private void StartSection() + { + _inAttributeSection = true; + _sectionHasTestAttribute = false; + _sectionTargetIgnored = false; + _itemNameFinalized = false; + _bracketDepth = 1; + _parenthesisDepth = 0; + _genericArgumentDepth = 0; + _attributeName.Clear(); } - return false; - } + private void CompleteItem() + { + FinalizeItemName(); + _blockItemCount++; + if (_blockItemCount > CSharpTestAttributeMaxItems) + _budgetExceeded = true; - private static bool CSharpLineHasTestMethodAttribute(string trimmedLine) - { - var cursor = 0; - while (cursor < trimmedLine.Length && trimmedLine[cursor] == '[') + _attributeName.Clear(); + _itemNameFinalized = false; + } + + private void FinalizeItemName() { - var closeIndex = trimmedLine.IndexOf(']', cursor + 1); - if (closeIndex < 0) - return false; + if (_itemNameFinalized) + return; - var content = trimmedLine[(cursor + 1)..closeIndex]; - if (CSharpTestMethodAttributeRegex.IsMatch(content)) - return true; + if (!_budgetExceeded + && _attributeName.Length > 0 + && CSharpTestMethodAttributeRegex.IsMatch(_attributeName.ToString())) + { + _sectionHasTestAttribute = true; + } - cursor = closeIndex + 1; - while (cursor < trimmedLine.Length && char.IsWhiteSpace(trimmedLine[cursor])) - cursor++; + _itemNameFinalized = true; + } + + private static bool IsTargetSpecifier(StringBuilder value, out bool ignoreTarget) + { + var target = value.ToString(); + var isTarget = target is + "assembly" or "event" or "field" or "method" or "module" or + "param" or "property" or "return" or "type" or "typevar"; + ignoreTarget = target is "assembly" or "module" or "return"; + return isTarget; } - return false; + private void CountLine() + { + _blockLineCount++; + if (_blockLineCount > CSharpTestAttributeMaxLines) + _budgetExceeded = true; + } + + private void CountCharacter() + { + _blockCharacterCount++; + if (_blockCharacterCount > CSharpTestAttributeMaxCharacters) + _budgetExceeded = true; + } + + private void ResetBlock() + { + _inAttributeSection = false; + _pendingAttributePrefix = false; + _blockHasTestAttribute = false; + _sectionHasTestAttribute = false; + _sectionTargetIgnored = false; + _itemNameFinalized = false; + _budgetExceeded = false; + _bracketDepth = 0; + _parenthesisDepth = 0; + _genericArgumentDepth = 0; + _blockLineCount = 0; + _blockCharacterCount = 0; + _blockItemCount = 0; + _attributeName.Clear(); + } } private static void AddCppFriendDeclarationSymbol( diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractionPhases.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractionPhases.cs index 932b948f9..941b47d9a 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractionPhases.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractionPhases.cs @@ -80,6 +80,7 @@ public PatternScanInputs( int[]?[] csharpMatchColumnToRaw = null!; string[]? csharpScopeLines = null; + bool[]? csharpTestMethodAttributedDeclarationLines = null; CSharpMatchLines = lang == "csharp" ? BuildCSharpMatchLines( lines, @@ -89,10 +90,12 @@ public PatternScanInputs( applyCSharpRegexProbeOptimizations, csharpRegexProbeCounts, out csharpMatchColumnToRaw, - out csharpScopeLines) + out csharpScopeLines, + out csharpTestMethodAttributedDeclarationLines) : null; CSharpMatchColumnToRaw = csharpMatchColumnToRaw; CSharpScopeLines = csharpScopeLines; + CSharpTestMethodAttributedDeclarationLines = csharpTestMethodAttributedDeclarationLines; GetCSharpLineStartStates = lang == "csharp" ? BuildCSharpLineStartStates : null; @@ -120,6 +123,7 @@ public PatternScanInputs( public int[]?[] CSharpMatchColumnToRaw { get; } public string[]? CSharpMatchLines { get; } public string[]? CSharpScopeLines { get; } + public bool[]? CSharpTestMethodAttributedDeclarationLines { get; } public Func? GetCSharpLineStartStates { get; } public Func? GetPrivateScopeColumns { get; } public Func? GetCSharpSwitchExpressionLines { get; } diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.PatternEmission.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.PatternEmission.cs index 42d84c96e..cebbc85b0 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.PatternEmission.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.PatternEmission.cs @@ -34,7 +34,8 @@ private readonly record struct PatternSymbolEmissionContext( List Symbols, SymbolExtractionState ExtractionState, HashSet? CssSeenSymbols, - HashSet? DockerfileStageNames); + HashSet? DockerfileStageNames, + bool IsCSharpTestMethod); private static string EmitPatternSymbols(PatternSymbolEmissionContext context) { @@ -54,8 +55,8 @@ private static string EmitPatternSymbols(PatternSymbolEmissionContext context) return kind; if (context.Language == "csharp" - && context.Pattern.Kind == "function" - && IsCSharpTestMethod(context.Lines, context.LineIndex)) + && context.Kind == "function" + && context.IsCSharpTestMethod) { kind = "test.method"; } diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.PatternEmissionFlow.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.PatternEmissionFlow.cs index 21262f117..870459241 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.PatternEmissionFlow.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.PatternEmissionFlow.cs @@ -40,10 +40,19 @@ private static PatternScanResult EmitAcceptedPatternSymbol( var extractionState = extraction.ExtractionState; var cssSeenSymbols = extraction.CssSeenSymbols; var dockerfileStageNames = extraction.DockerfileStageNames; + var isCSharpTestMethod = lang == "csharp" + && scanInputs.CSharpTestMethodAttributedDeclarationLines is { } attributedDeclarationLines + && IsCSharpTestMethod( + attributedDeclarationLines, + i, + shapedSymbol.Kind == "function" + && shapedSymbol.RawReturnType != null + && match.Groups["name"].Value != "this"); ref var scanState = ref extraction.ScanState; ref var pendingRecordPrimaryComponents = ref extraction.PendingRecordPrimaryComponents; ref var recordPrimaryComponentParentIndex = ref extraction.RecordPrimaryComponentParentIndex; emittedKind = kind; + var symbolCountBeforeEmission = symbols.Count; kind = EmitPatternSymbols( new PatternSymbolEmissionContext( fileId, @@ -68,7 +77,16 @@ private static PatternScanResult EmitAcceptedPatternSymbol( symbols, extractionState, cssSeenSymbols, - dockerfileStageNames)); + dockerfileStageNames, + isCSharpTestMethod)); + if (lang == "csharp" + && symbols.Count > symbolCountBeforeEmission + && scanInputs.CSharpTestMethodAttributedDeclarationLines is { } emittedAttributeLines) + { + ConsumeCSharpTestAttributePrefix( + emittedAttributeLines, + i); + } if (lang == "css" && pattern.Kind == "namespace" diff --git a/tests/CodeIndex.Tests/IndexCommandRunnerFullScanTests.cs b/tests/CodeIndex.Tests/IndexCommandRunnerFullScanTests.cs index 016ad644d..19b082d06 100644 --- a/tests/CodeIndex.Tests/IndexCommandRunnerFullScanTests.cs +++ b/tests/CodeIndex.Tests/IndexCommandRunnerFullScanTests.cs @@ -6283,7 +6283,7 @@ FROM symbol_references SymbolExtractor.CSharpContractVersion.ToString( System.Globalization.CultureInfo.InvariantCulture), versionCmd.ExecuteScalar() as string); - Assert.Equal(16, SymbolExtractor.CSharpContractVersion); + Assert.Equal(17, SymbolExtractor.CSharpContractVersion); } finally { @@ -6549,7 +6549,7 @@ ORDER BY symbol_name SymbolExtractor.CSharpContractVersion.ToString( System.Globalization.CultureInfo.InvariantCulture), versionCmd.ExecuteScalar() as string); - Assert.Equal(16, SymbolExtractor.CSharpContractVersion); + Assert.Equal(17, SymbolExtractor.CSharpContractVersion); } finally { diff --git a/tests/CodeIndex.Tests/QueryCommandRunnerIssue5192Tests.cs b/tests/CodeIndex.Tests/QueryCommandRunnerIssue5192Tests.cs new file mode 100644 index 000000000..adeaa20ce --- /dev/null +++ b/tests/CodeIndex.Tests/QueryCommandRunnerIssue5192Tests.cs @@ -0,0 +1,144 @@ +using CodeIndex.Cli; +using CodeIndex.Database; +using static CodeIndex.Tests.QueryCommandTestSupport; + +namespace CodeIndex.Tests; + +public partial class QueryCommandRunnerTests +{ + [Fact] + public void CSharpMultilineTestAttributes_PersistAcrossSymbolCommandsAndTestFilters_Issue5192() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_csharp_multiline_test_attributes_5192"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "tests/CalculatorTests.cs", + "csharp", + """ + namespace Demo; + + public class CalculatorTests + { + [Theory] + [InlineData( + 1, + "closing ] and fake [Fact]")] + private void MultilineTheory(int value, string text) { } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Calculator.cs", + "csharp", + """ + namespace Demo; + + public sealed class Fact { } + public sealed class MetadataAttribute : System.Attribute { } + + public class Calculator + { + private void HelperMethod() { } + + [Metadata< + string, + Fact>] + private void GenericAttributeHelper() { } + + private static int FactValue => 1; + private int[][] NestedCollectionInitializer = + { + [ + FactValue + ] + }; private void InitializerFollowingHelper() { } + } + """); + using (var db = new DbContext(DbOpenIntent.WriteIndex, dbPath)) + { + db.InitializeSchema(); + new DbWriter(db.Connection).MarkGraphReady(); + } + + var (outlineExitCode, outlineStdout, outlineStderr) = CaptureConsole(() => + QueryCommandRunner.RunOutline( + ["tests/CalculatorTests.cs", "--db", dbPath, "--json"], + _jsonOptions)); + using var outlineDocument = ParseJsonOutput(outlineStdout); + var outlineTestMethod = Assert.Single( + outlineDocument.RootElement + .GetProperty("symbols") + .EnumerateArray() + .Where(symbol => symbol.GetProperty("name").GetString() == "MultilineTheory")); + + Assert.Equal(CommandExitCodes.Success, outlineExitCode); + Assert.Equal(string.Empty, outlineStderr); + Assert.Equal("test.method", outlineTestMethod.GetProperty("kind").GetString()); + + var (symbolsExitCode, symbolsStdout, symbolsStderr) = CaptureConsole(() => + QueryCommandRunner.RunSymbols( + ["MultilineTheory", "--db", dbPath, "--json", "--exact-name", "--kind", "test.method"], + _jsonOptions)); + var symbolRows = ParseJsonLines(symbolsStdout) + .Select(document => document.RootElement) + .ToArray(); + + Assert.Equal(CommandExitCodes.Success, symbolsExitCode); + Assert.Equal(string.Empty, symbolsStderr); + Assert.Equal("test.method", Assert.Single(symbolRows).GetProperty("kind").GetString()); + + var (inspectExitCode, inspectStdout, inspectStderr) = CaptureConsole(() => + QueryCommandRunner.RunInspect( + ["MultilineTheory", "--db", dbPath, "--json", "--exact"], + _jsonOptions)); + using var inspectDocument = ParseJsonOutput(inspectStdout); + var inspectedDefinition = Assert.Single( + inspectDocument.RootElement.GetProperty("definitions").EnumerateArray()); + + Assert.Equal(CommandExitCodes.Success, inspectExitCode); + Assert.Equal(string.Empty, inspectStderr); + Assert.Equal("test.method", inspectedDefinition.GetProperty("kind").GetString()); + + var (unusedExitCode, unusedStdout, unusedStderr) = CaptureConsole(() => + QueryCommandRunner.RunUnused( + ["--db", dbPath, "--json", "--all", "--lang", "csharp", "--limit", "50"], + _jsonOptions)); + using var unusedDocument = ParseJsonOutput(unusedStdout); + var unusedNames = unusedDocument.RootElement + .GetProperty("symbols") + .EnumerateArray() + .Select(symbol => symbol.GetProperty("name").GetString()) + .ToArray(); + + Assert.Equal(CommandExitCodes.Success, unusedExitCode); + Assert.Equal(string.Empty, unusedStderr); + Assert.Contains("MultilineTheory", unusedNames); + Assert.Contains("HelperMethod", unusedNames); + + var (excludedExitCode, excludedStdout, excludedStderr) = CaptureConsole(() => + QueryCommandRunner.RunUnused( + ["--db", dbPath, "--json", "--all", "--lang", "csharp", "--exclude-tests", "--limit", "50"], + _jsonOptions)); + using var excludedDocument = ParseJsonOutput(excludedStdout); + var excludedNames = excludedDocument.RootElement + .GetProperty("symbols") + .EnumerateArray() + .Select(symbol => symbol.GetProperty("name").GetString()) + .ToArray(); + + Assert.Equal(CommandExitCodes.Success, excludedExitCode); + Assert.Equal(string.Empty, excludedStderr); + Assert.DoesNotContain("MultilineTheory", excludedNames); + Assert.Contains("HelperMethod", excludedNames); + Assert.Contains("GenericAttributeHelper", excludedNames); + Assert.Contains("InitializerFollowingHelper", excludedNames); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } +} diff --git a/tests/CodeIndex.Tests/SymbolExtractorCSharpTests.cs b/tests/CodeIndex.Tests/SymbolExtractorCSharpTests.cs index 6cf9ae9d7..a9659dd5a 100644 --- a/tests/CodeIndex.Tests/SymbolExtractorCSharpTests.cs +++ b/tests/CodeIndex.Tests/SymbolExtractorCSharpTests.cs @@ -524,6 +524,227 @@ public void HelperMethod() { } Assert.Contains(symbols, s => s.Kind == "function" && s.Name == "HelperMethod"); } + [Fact] + public void Extract_CSharp_ClassifiesMultilineTestAttributeOwnership_Issue5192_Regression1961() + { + var content = """" + [assembly: Xunit.Theory] + [module: Xunit.Fact] + + namespace Demo.Tests; + + [System.AttributeUsage(System.AttributeTargets.Field)] + public sealed class FieldFactAttribute : System.Attribute { } + + [System.AttributeUsage(System.AttributeTargets.Constructor)] + public sealed class ConstructorFactAttribute : System.Attribute { } + + [System.AttributeUsage(System.AttributeTargets.Property)] + public sealed class IndexerFactAttribute : System.Attribute + { + public IndexerFactAttribute(string marker) { } + } + + [System.AttributeUsage(System.AttributeTargets.Method)] + public sealed class MetadataAttribute : System.Attribute { } + + public sealed class Fact { } + + public class MultilineAttributeTests + { + [Theory] + [InlineData( + 1, + 2, + "closing ] and fake [Fact]", + @"verbatim ] and fake [Theory]", + """raw ] and fake [Test]""", + $"interpolated ] {1 + 2} and fake [TestMethod]", + ']')] + [InlineData( + 3, + 4, + "second multiline data", + @"second verbatim value", + """second raw value""", + $"second interpolated {3 + 4}", + '[')] + public void MultilineInlineData(int left, int right, string normal, string verbatim, string raw, string interpolated, char bracket) { } + + [Xunit.TheoryAttribute, + InlineData(3)] + public void QualifiedTheoryInMultiAttributeSection(int value) { } + + [Fact] + // ownership survives a line comment + [Trait( + "category", + "fast")] + /* ownership also survives a block comment */ + public void CommentsAndMultipleAttributes() { } + + [TestMethod] + public void MsTestMethod() { } + + [DataTestMethod] + public void MsDataTestMethod() { } + + [NUnit.Framework.Test] + public void NUnitTestMethod() { } + + [NUnit.Framework.TestCase( + 4)] + public void NUnitTestCaseMethod(int value) { } + + [NUnit.Framework.TestCaseSource( + nameof(Cases))] + public void NUnitTestCaseSourceMethod(int value) { } + + [Fact] public void SameLineFact() { } + + [method: Fact] + public void ExplicitMethodTargetFact() { } + + [Fact] public void FirstMemberBeforeBoundary() { } + public void FollowingMemberBoundary() { } + + [Obsolete( + "Theory ] [Fact]")] + public void NonTestMultilineAttribute() { } + + [Obsolete("Fact")] + public void LiteralNameDoesNotSpoof() { } + + [return: Xunit.Theory] + public int ReturnTargetDoesNotClassify() => 0; + + [Fact] public int AttributedProperty { get; } + public void MemberAfterAttributedProperty() { } + + [FieldFact] + [System.Obsolete( + "const field marker")] + public const int AttributedConstField = 1; + + [FieldFact] + [System.Obsolete( + "readonly field marker")] + public static readonly int AttributedReadonlyField = 2; + + [ConstructorFact] + [System.Obsolete( + "constructor marker")] + public MultilineAttributeTests() { } + + [ConstructorFact] + [System.Obsolete( + "partial constructor marker")] + partial MultilineAttributeTests(); + + [IndexerFact( + "indexer marker")] + public int this[int index] => index; + + [Metadata< + string, + Fact>] + public void GenericAttributeTypeArgumentDoesNotSpoofTest() { } + + public int[] CollectionExpression => + [ + Fact + ]; public void CollectionExpressionDoesNotSpoofTest() { } + + public int[][] NestedCollectionInitializer = + { + [ + Fact + ] + }; public void NestedCollectionInitializerDoesNotSpoofTest() { } + + public static int Fact => 1; + public static int[] Cases => [4]; + } + """"; + + var symbols = SymbolExtractor.Extract(1, "csharp", content); + var expectedTestMethods = new[] + { + "MultilineInlineData", + "QualifiedTheoryInMultiAttributeSection", + "CommentsAndMultipleAttributes", + "MsTestMethod", + "MsDataTestMethod", + "NUnitTestMethod", + "NUnitTestCaseMethod", + "NUnitTestCaseSourceMethod", + "SameLineFact", + "ExplicitMethodTargetFact", + "FirstMemberBeforeBoundary", + }; + var expectedNonTestMethods = new[] + { + "FollowingMemberBoundary", + "NonTestMultilineAttribute", + "LiteralNameDoesNotSpoof", + "ReturnTargetDoesNotClassify", + "MemberAfterAttributedProperty", + "MultilineAttributeTests", + "CollectionExpressionDoesNotSpoofTest", + "GenericAttributeTypeArgumentDoesNotSpoofTest", + "NestedCollectionInitializerDoesNotSpoofTest", + }; + + Assert.All( + expectedTestMethods, + name => Assert.Contains(symbols, symbol => symbol.Kind == "test.method" && symbol.Name == name)); + Assert.All( + expectedNonTestMethods, + name => Assert.Contains(symbols, symbol => symbol.Kind == "function" && symbol.Name == name)); + Assert.DoesNotContain( + symbols, + symbol => expectedNonTestMethods.Contains(symbol.Name, StringComparer.Ordinal) + && symbol.Kind == "test.method"); + Assert.Contains( + symbols, + symbol => symbol.Kind == "field" && symbol.Name == "AttributedConstField"); + Assert.Contains( + symbols, + symbol => symbol.Kind == "field" && symbol.Name == "AttributedReadonlyField"); + Assert.Contains( + symbols, + symbol => symbol.Kind == "function" + && symbol.Signature?.Contains("this[int index]", StringComparison.Ordinal) == true); + Assert.DoesNotContain( + symbols, + symbol => symbol.Kind == "test.method" + && symbol.Signature?.Contains("this[int index]", StringComparison.Ordinal) == true); + } + + [Fact] + public void Extract_CSharp_TestAttributeBudgetsFailClosed_Issue5192() + { + var tooManyLines = "public class LineBudget\n{\n [Fact]\n" + + string.Concat(Enumerable.Repeat(" // padding\n", 128)) + + " public void BeyondLineBudget() { }\n}"; + var tooManyItems = "public class ItemBudget\n{\n [" + + string.Join(", ", Enumerable.Repeat("Obsolete", 64)) + + ", Fact]\n public void BeyondItemBudget() { }\n}"; + var tooLongName = "public class NameBudget\n{\n [" + + new string('A', 512) + + "Fact]\n public void BeyondNameBudget() { }\n}"; + + Assert.Contains( + SymbolExtractor.Extract(1, "csharp", tooManyLines), + symbol => symbol.Kind == "function" && symbol.Name == "BeyondLineBudget"); + Assert.Contains( + SymbolExtractor.Extract(1, "csharp", tooManyItems), + symbol => symbol.Kind == "function" && symbol.Name == "BeyondItemBudget"); + Assert.Contains( + SymbolExtractor.Extract(1, "csharp", tooLongName), + symbol => symbol.Kind == "function" && symbol.Name == "BeyondNameBudget"); + } + [Fact] public void Extract_CSharp_NormalizesUnicodeEscapedIdentifierNames() {