Skip to content

SONARJAVA-7300 Decouple the Sonar Java frontend from Eclipse JDT core - #6485

Open
aurelien-coet-sonarsource wants to merge 8 commits into
masterfrom
ac/implementation-agnostic-frontend-2
Open

aurelien-coet-sonarsource wants to merge 8 commits into
masterfrom
ac/implementation-agnostic-frontend-2

Conversation

@aurelien-coet-sonarsource

@aurelien-coet-sonarsource aurelien-coet-sonarsource commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary by Gitar

  • Annotation defaults:
    • Annotation element defaults now resolve through Symbol.MethodSymbol.annotationElementDefaultValue() as semantic API types
  • Semantic utilities:
    • Made WRAPPER_TO_PRIMITIVE and PRIMITIVE_TO_WRAPPER public immutable maps in JUtils
  • Test coverage:
    • Removed FrontendApiTest and replaced it with JdtClassesLayoutTest and ClassLayoutTestUtils to check the JDT layout
  • JDT isolation:
    • Moved ECJ-specific parser, symbol, type, and metadata classes into org.sonar.java.jdt behind backend-neutral contracts
  • Parser configuration:
    • Added JParserConfig.Mode dispatch to JDTParserConfig, with batch mode falling back to file-by-file parsing on errors

This will update automatically on new commits.

@hashicorp-vault-sonar-prod hashicorp-vault-sonar-prod Bot changed the title Decouple the Sonar Java frontend from Eclipse JDT core SONARJAVA-7300 Decouple the Sonar Java frontend from Eclipse JDT core Nov 4, 2026
@hashicorp-vault-sonar-prod

hashicorp-vault-sonar-prod Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

SONARJAVA-7300

@aurelien-coet-sonarsource
aurelien-coet-sonarsource force-pushed the ac/implementation-agnostic-frontend-2 branch from d50bf37 to 80b9ad3 Compare November 4, 2026 14:51
@aurelien-coet-sonarsource

Copy link
Copy Markdown
Contributor Author

/pvf

@hashicorp-vault-sonar-prod

Copy link
Copy Markdown
Contributor

📊 Performance Validation complete
🆕 0 new · 🗑️ 0 lost issues
View dashboard

@aurelien-coet-sonarsource
aurelien-coet-sonarsource force-pushed the ac/implementation-agnostic-frontend-2 branch 6 times, most recently from c0370cd to b88c5c0 Compare November 6, 2026 09:51
Base automatically changed from ac/implementation-agnostic-frontend-1 to master November 6, 2026 10:59
@aurelien-coet-sonarsource
aurelien-coet-sonarsource force-pushed the ac/implementation-agnostic-frontend-2 branch from b88c5c0 to c4b1cf5 Compare November 6, 2026 11:00
@aurelien-coet-sonarsource

Copy link
Copy Markdown
Contributor Author

/pvf

@hashicorp-vault-sonar-prod

Copy link
Copy Markdown
Contributor

📊 Performance Validation complete
🆕 0 new · 🗑️ 0 lost issues
View dashboard

@aurelien-coet-sonarsource
aurelien-coet-sonarsource force-pushed the ac/implementation-agnostic-frontend-2 branch 2 times, most recently from 58bc4a9 to d4d7696 Compare November 6, 2026 13:50
@aurelien-coet-sonarsource

Copy link
Copy Markdown
Contributor Author

/pvf

@hashicorp-vault-sonar-prod

Copy link
Copy Markdown
Contributor

📊 Performance Validation complete
🆕 0 new · 🗑️ 0 lost issues
View dashboard

@aurelien-coet-sonarsource
aurelien-coet-sonarsource force-pushed the ac/implementation-agnostic-frontend-2 branch 4 times, most recently from cba5e05 to 7e1aebc Compare November 9, 2026 10:52
@aurelien-coet-sonarsource
aurelien-coet-sonarsource marked this pull request as ready for review November 9, 2026 11:07
@aurelien-coet-sonarsource
aurelien-coet-sonarsource force-pushed the ac/implementation-agnostic-frontend-2 branch 2 times, most recently from c979713 to 5170c8d Compare November 9, 2026 12:56
@aurelien-coet-sonarsource

Copy link
Copy Markdown
Contributor Author

/pvf

@hashicorp-vault-sonar-prod

Copy link
Copy Markdown
Contributor

⚠️ Performance Validation finished without publishing a dashboard. See run logs.

Projects with unexpected PVF results:

  • org.apache.bcel-bcel
  • org.apache.commons-commons-configuration2
  • org.apache.opennlp-opennlp
  • org.codehaus.sonar-plugins.l10n-sonar-l10n-fr-plugin
  • xwiki-commons_master

@hashicorp-vault-sonar-prod

Copy link
Copy Markdown
Contributor

📊 Performance Validation complete
🆕 0 new · 🗑️ 0 lost issues
View dashboard

Projects with unexpected PVF results:

  • org.apache.opennlp-opennlp
  • org.codehaus.sonar-plugins.l10n-sonar-l10n-fr-plugin
  • xwiki-commons_master

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall it looks good to me, there are some minor things that I pointed out.
This PR might be the right place to improve some of the naming and connections that we have in this layer of the frontend, but I don't want to start piling things to do here.

So the following is just FYI, and you can simply tackle the comments on the code

A couple of things that kind of bother me are:

  • we call parser what is actually an AST converter
  • we get an instance of the parser through an object called JParserConfig
  • the JParserConfig has a "parse" method that needs to be implemented
  • the parse method requires a callback parameter called "action", and returns void, instead of returning compilation units 😠
  • JParserConfig is still bound to jdt, as the modes implementations rely on jdt ASTParser

The PR still does a great job at setting a building block for the future, and I understand that you tried to just keep the existing code while abstracting away the jdt parser, I'm good with that, but I think it's worth keeping in mind where we could still improve in making classes more readable and with fewer responsibilities

import java.util.Optional;
import java.util.Set;
import javax.annotation.Nullable;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can revert this changes

Comment on lines +250 to +258
/**
* Returns an annotation element's default value, or null if it has none.
* Values use the same representation as {@link SymbolMetadata.AnnotationValue#value()}:
* constants, type symbols, variable symbols, annotation instances, or arrays of these values.
*/
@Nullable
default Object annotationElementDefaultValue() {
return null;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a small changelog note in the API changes docs

@aurelien-coet-sonarsource aurelien-coet-sonarsource Nov 11, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, I'll add an entry to document this, and take the opportunity to also describe the changes to the API from the two previous PRs related to this one (#6472 and #6479).
On that note, should we bump to a new major version (9), given that we introduce breaking changes to the API ?

Comment on lines +85 to +91
@Override
public JParser astParser() {
return new JDTParser(jdtParser());
}

ASTParser jdtParser() {
ASTParser astParser = ASTParser.newParser(AST.getJLSLatest());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the perfect occasion to also improve on how we name things around here.

I feel like what is currently called JDTParser on our side, is actually an AST converter, more than a parser, as the parsing happens inside org.eclipse.jdt.core.dom.ASTParser that we call in a couple of lines, and then 99.9% of our JDTParser is moving information from an AST to the other.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to ignore this one for now if it's changing too many things

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have opened another PR on top of this one to try and improve this, as well as simplify the parser interface.

}

public abstract void parse(Iterable<? extends InputFile> inputFiles, BooleanSupplier isCanceled,
void parse(Iterable<? extends InputFile> inputFiles, BooleanSupplier isCanceled,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not mandatory, but we could easily take the chance to remove this <? extends InputFile> and just have

Suggested change
void parse(Iterable<? extends InputFile> inputFiles, BooleanSupplier isCanceled,
void parse(Iterable<InputFile> inputFiles, BooleanSupplier isCanceled,

It requires a few changes here and there but it's a fairly small and mechanical change, basically the only reason we have this is so that when processin GeneratedFiles with Jasper, we can at some point do

GeneratedFile generatedFile = generatedJavaFiles.get(smapFile.getGeneratedFile());
    if (generatedFile != null) {

instead of

var file = generatedJavaFiles.get(smapFile.getGeneratedFile());
    if (file instanceof GeneratedFile generatedFile) {
      generatedFile.addSmap(smapFile);
    }

That's it... we can talk about his

Comment on lines +40 to +42
enum Mode {
BATCH,
FILE_BY_FILE;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently there is no contract that enforces a JParserConfig implementation to provide two different implementation based on the selected Mode. This different mode batch/file_by_file is something kind of specific to ECJ at the moment, I think tree-sitter for instance does now have such modes, even if we could manually (and partially) implement them.

That being said, our frontend is somewhat bound to this definition currently, so I think we can keep this separation of modes, but then we should enforce a different implementation for the 2 of them.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a second thought, I don't want to slow down this PR for this reason, for now it's ok to have this implementation.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. As you mention, our current implementation is pretty tightly bound to that model, but I'd also like to try to improve this in #6503.

Use semantic interfaces for shared operations and expose annotation element defaults through MethodSymbol. Extract neutral nullability data while retaining JUtils and JSymbolMetadataNullabilityHelper.
Move JDT parsing and semantic implementations into org.sonar.java.jdt. Use the existing semantic and diagnostic API contracts directly, retain parser and shared utility contracts, and create environment cleanup callbacks in the JDT parser.
@aurelien-coet-sonarsource
aurelien-coet-sonarsource force-pushed the ac/implementation-agnostic-frontend-2 branch from 67336c9 to b308c9c Compare November 13, 2026 13:53
@gitar-bot

gitar-bot Bot commented Nov 13, 2026

Copy link
Copy Markdown
Code Review ✅ Approved 🟡 Medium risk 7 closed / 7 findings

Moving Java parsing and semantic handling behind backend-neutral contracts could introduce AST or symbol-resolution regressions in analysis.

Decouples the Sonar Java frontend from Eclipse JDT core behind backend-neutral contracts, with JDT isolation verified by tests. Resolved issues include the directSuperTypes fix for interfaces and arrays, the unused MapBuilder import, the SE repo test helper compile break, source-path type resolution from #6308, release-API JDK resolution from #6459, and a tautological v.symbol() assertion. No open findings remain.

✅ 7 closed
✅ Bug: directSuperTypes now adds java.lang.Object for interfaces and arrays

📄 java-frontend/src/main/java/org/sonar/java/model/SemanticUtils.java:146-157 📄 java-frontend/src/main/java/org/sonar/java/jdt/JTypeSymbol.java:108-119
The old directSuperTypes read typeBinding.getSuperclass(), which JDT returns as null for interfaces and arrays. The new version calls type.symbol().superClass(), and JTypeSymbol.convertSuperClass() returns java.lang.Object for those cases. So for an interface type, directSuperTypes now includes Object next to its interfaces, which is a behaviour change in a refactor that was meant to keep behaviour the same. There are no production callers yet, so nothing breaks today, but anyone who starts using the public utility gets the different result. To keep the old contract, skip the superclass when type.symbol().isInterface() or when the type is an array.

✅ Quality: Unused MapBuilder import left after switching to Map.of

📄 java-frontend/src/main/java/org/sonar/java/model/SemanticUtils.java:33 📄 java-frontend/src/main/java/org/sonar/java/model/SemanticUtils.java:40-54
This commit swaps MapBuilder.newMap()...build() for Map.of(...) in WRAPPER_TO_PRIMITIVE and PRIMITIVE_TO_WRAPPER. Nothing in SemanticUtils uses MapBuilder anymore, but import org.sonarsource.analyzer.commons.collections.MapBuilder; is still there. That is a new unused-import issue (S1128) in a commit whose goal is to fix the quality gate. Delete the import.

✅ Cross-Repo: SE repo test helper no longer compiles: JParser/JParserConfig API moved

📄 java-frontend/src/main/java/org/sonar/java/jdt/JParserConfig.java:53 📄 java-frontend/src/main/java/org/sonar/java/jdt/JParserConfig.java:70-84 📄 java-frontend/src/main/java/org/sonar/java/jdt/JParser.java:264 📄 java-frontend/src/main/java/org/sonar/java/jdt/JParser.java:272 📄 java-frontend/src/main/java/org/sonar/java/JavaParsers.java
Fix here and in SonarSource/sonar-java-symbolic-execution

Breaks in: SonarSource/sonar-java-symbolic-execution - java-symbolic-execution/java-symbolic-execution-plugin/src/test/java/org/sonar/java/se/utils/JParserTestUtils.java - JParserTestUtils.parse

This breaks SonarSource/sonar-java-symbolic-execution, in java-symbolic-execution/java-symbolic-execution-plugin/src/test/java/org/sonar/java/se/utils/JParserTestUtils.java (JParserTestUtils.parse).

That helper imports org.sonar.java.model.JParser and org.sonar.java.model.JParserConfig. It builds its parser with JParserConfig.Mode.FILE_BY_FILE.create(...).astParser(), passes it to JParser.parse(ASTParser, version, unitName, source), and reads JParserConfig.MAXIMUM_SUPPORTED_JAVA_VERSION. This PR changes all of that:

  • JParser moves to org.sonar.java.jdt as a package-private final class with a package-private parse.
  • JParserConfig moves to org.sonar.java.jdt, loses Mode, and its astParser() becomes package-private.
  • MAXIMUM_SUPPORTED_JAVA_VERSION moves to org.sonar.java.JavaParsers.

Once the SE repo bumps to this frontend, the helper no longer compiles, and neither does any SE test that depends on it. The same pair of APIs came up in earlier reviews (JParser.parse SE dependency, astParser SE dependency). The PR's goal is to decouple the frontend from JDT, so the move is intended and the consumer has to migrate. The replacement already exists: JavaParsers.Mode.FILE_BY_FILE.create(version, classpath).parse(unitName, source).

Here: Record in the release or migration notes that org.sonar.java.model.JParser/JParserConfig were removed, and name JavaParsers.MAXIMUM_SUPPORTED_JAVA_VERSION and JavaParsers.Mode.X.create(...).parse(unitName, source) as the replacements. Coordinate the release with sonar-java-symbolic-execution.

There: In JParserTestUtils.parse, drop the JParserConfig/JParser/ASTParser imports and call the new JavaParsers/JavaParser API.

In SonarSource/sonar-java-symbolic-execution - java-symbolic-execution/java-symbolic-execution-plugin/src/test/java/org/sonar/java/se/utils/JParserTestUtils.java:

import org.sonar.java.JavaParsers;

public static CompilationUnitTree parse(String unitName, String source, List<File> classpath) {
  return JavaParsers.Mode.FILE_BY_FILE
    .create(JavaParsers.MAXIMUM_SUPPORTED_JAVA_VERSION, classpath)
    .parse(unitName, source);
}
✅ Cross-PR: Source-path type resolution from #6308 dropped by the new JDT layout

📄 java-frontend/src/main/java/org/sonar/java/jdt/JDTParserConfig.java:69-70 📄 java-frontend/src/main/java/org/sonar/java/jdt/JDTParserConfig.java:110-112 📄 java-frontend/src/main/java/org/sonar/java/model/JParserConfig.java:44-50 📄 java-frontend/src/main/java/org/sonar/java/model/JParserConfig.java:78-81
Breaks when this PR and #6308 both merge, and the second one keeps this PR's Mode.create/JDTParserConfig.create signatures without threading SourcePath through.

How

  1. In this PR, Mode.create(javaVersion, classpath, ignoreUnnamed) delegates to a 3-argument JDTParserConfig.create, and ParserConfigConstructor still takes 3 arguments (JParserConfig.java:44-50,78-81).
  2. JDTParserConfig.jdtParser() always passes empty source-path and encoding arrays to setEnvironment (JDTParserConfig.java:110-112).
  3. SONARJAVA-7137 Optionally rely on source paths to resolve types during analyses #6308 adds a 4-argument Mode.create(..., sourcePath, ...), which JavaFrontend.scanBatch and JavaAstScanner call. It also adds a sourcePath field to the old abstract class, and its JParserConfigTest overrides public ASTParser astParser().
  4. As a result, the 4-argument callers and the astParser() override no longer compile. If the conflict is resolved by keeping this PR's files, sonar.java.sourcepath resolution is lost without any error.

Fix

  • This PR: If it lands second, add the SourcePath overload to Mode.create, ParserConfigConstructor, JDTParserConfig.create and the Batch/FileByFile constructors, and pass its roots and encodings to setEnvironment in jdtParser().
  • SONARJAVA-7137 Optionally rely on source paths to resolve types during analyses #6308: If it lands second, make the same threading changes in JDTParserConfig, and change the test override to target JDTParserConfig.jdtParser() instead of astParser().

Compared 96acea5 with #6308 at 7068bcd

✅ Cross-PR: Release-API JDK resolution from #6459 lost or uncompilable after refactor

📄 java-frontend/src/main/java/org/sonar/java/jdt/JDTParserConfig.java:57 📄 java-frontend/src/main/java/org/sonar/java/jdt/JDTParserConfig.java:90-104 📄 java-frontend/src/main/java/org/sonar/java/model/JParserConfig.java:29-35
Breaks when this PR and #6459 both merge, and the second one is rebased onto the neutral JParserConfig interface without moving the release-API logic into JDTParserConfig.jdtParser().

How

  1. This PR turns JParserConfig into an interface. Its astParser() now returns JParser, and it no longer has a classpath field (JParserConfig.java:29-35).
  2. The ECJ setup moved to JDTParserConfig.jdtParser(), which still passes the unfiltered classpath to setEnvironment and checks it against the old JRE_JARS set (JDTParserConfig.java:57,110). Nothing in it sets the release option.
  3. SONARJAVA-7279 Resolve the JDK API of the target release through the new jdt-package option #6459 edits the old abstract JParserConfig. It adds OPTION_RELEASE, adds useReleaseApiOfConfiguredJdk(options) operating on classpath, and passes a filtered environment list to setEnvironment. Its tests use astParser() as an ASTParser together with JParser.parse(astParser, ...).
  4. As a result, SONARJAVA-7279 Resolve the JDK API of the target release through the new jdt-package option #6459's code no longer compiles against this layout. If someone resolves the conflict by keeping this PR's version of the file, release-API JDK selection is dropped without any error.

Fix

Compared 96acea5 with #6459 at ac3fd7c

...and 2 more closed from earlier reviews

Verification Rules 1/1 met Knowledge 🔎 2 PRs 4 inspected

Functional validation No results

Rules 1 passed 3 not applicable

✅ Repository license compliance

Knowledge 🔎 2 informed a finding

🔎 JParserConfig.astParser() / JParser.parse(ASTParser, String, String, String): dependency with SonarSource/sonar-java-symbolic-execution informed a finding, resolved -> SE repo test helper no longer compiles: JParser/JParserConfig API moved
🔎 org.sonar.java.model.JParser.parse(ASTParser, String, String, String): dependency with SonarSource/sonar-java-symbolic-execution informed a finding, resolved -> SE repo test helper no longer compiles: JParser/JParserConfig API moved

Broader impact PRs 4 inspected
Cross-PR 4 PRs inspected 0 open impacts

PR 6503: Simplify the parser interface Inspected
PR 6459: SONARJAVA-7279 Resolve the JDK API of the target release through the new jdt-package option Inspected
PR 6497: SONARJAVA-7236 Experiment: supply only the JDK classpath to semsitter Inspected
PR 6501: SONARJAVA-7236 Experiment: let semsitter read sourcepath files that are not analyzed Inspected

Auto-approval Not enabled

Set up

For more verification details, comment gitar display:verbose.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing compact verification results.

Comment with these commands to change the behavior for this request:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@sonarqube-next

Copy link
Copy Markdown
Contributor

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants