Cosmos: Modernize materializer & subdocument projectionbinding by JoasE · Pull Request #38550 · dotnet/efcore

JoasE · 2026-07-02T12:44:02Z

Materialize query results using Utf8JsonReader
Bind projections of embedded types and remove client projection distinct limitations
Remove __jObject backing property
Require projecting out primary key values for owned projections using AsNoTrackingWithIdentityResolution
Throw for any undefined non-value projection (Cosmos: Anonymous and dto projections filter undefined implicitly when projecting a single property #38298 (comment))
Truncate floating point numbers when materializing to fixed point types instead of rounding (Cosmos: Materializer rounds when deserializing floating point numbers as fixed point numbers #38138)

Part of: #34567
Implements: #34067
Fixes: #38138 , #31696

…Improve

…otnet#34567-Materialization

Copilot

Pull request overview

Copilot reviewed 101 out of 105 changed files in this pull request and generated 4 comments.

Files not reviewed (1)

src/EFCore.Cosmos/Properties/CosmosStrings.Designer.cs: Generated file

JoasE · 2026-07-08T10:10:45Z

Tests are failing for linux emulator because the projection order is not preserved there, and the materializer currently depends on this order. This is specifically usefull in NoTrackingWithIdentityResolution cases, where we materialize the id of the root document first before materializing sub documents. Production cosmos does preserve the projection order, but I'm wondering whether this is something we should depend and whether it's a guarantee in cosmos or might be subject to change. Regardless, I've created: Azure/azure-cosmos-db-emulator-docker#335
@AndriySvyryd Thoughts on this?
PS: PR still WIP, needs some cleanup but is close to final form

This appears to have been be planned by the linux emulator team

Copilot

Pull request overview

Copilot reviewed 100 out of 104 changed files in this pull request and generated 4 comments.

Files not reviewed (1)

src/EFCore.Cosmos/Properties/CosmosStrings.Designer.cs: Generated file

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 100 out of 104 changed files in this pull request and generated 2 comments.

Files not reviewed (1)

src/EFCore.Cosmos/Properties/CosmosStrings.Designer.cs: Generated file

Copilot

Pull request overview

Copilot reviewed 100 out of 104 changed files in this pull request and generated 1 comment.

Files not reviewed (1)

src/EFCore.Cosmos/Properties/CosmosStrings.Designer.cs: Generated file

Copilot

Pull request overview

Copilot reviewed 100 out of 104 changed files in this pull request and generated 2 comments.

Files not reviewed (1)

src/EFCore.Cosmos/Properties/CosmosStrings.Designer.cs: Generated file

JoasE · 2026-07-08T13:15:07Z

+                    // @TODO: Ask if there is a better way for this..
+                    // We need to update projection bindings in the subquery shaper to relate to a query expression that can actually provide the correct binding information.
+                    // Then we are able to use the projection binding's query expression directly in ShaperProcessingVisitor, instead of storing the select expression separately there
+                    // This is needed because cosmos can have projections which are subqueries, and those subqueries can have their own projections
+                    // This appears to be needed because ShapedQueryExpression doesn't properly replace projection bindings their query expression when updating the shaper expression.
+                    // But fixing that causes a lot of errors (in other providers?).
+                    new ProjectionBindingQueryProjectionApplyingExpressionVisitor().Visit(subquery.ShaperExpression);


@AndriySvyryd I forgot to point this out
The current ShaperProcessingVisitor for cosmos will directly use the ProjectionBindingExpression.QueryExpression to find the ProjectionExpression for a ProjectionBindingExpression. This is to be able to generate shapers for subqueries in projections. This is different from relational where only the SelectExpression from the final ShapedQueryExpression is used, but relational doesn't have subqueries in projections AFAIK.
What I found odd is that I don't think this code would be needed if ShapedQueryExpression.UpdateQueryExpression actually replaced the ProjectionBindingExpression.QueryExpression of any ProjectionBindingExpressions it found in the ShaperExpression, which it does look like it intends to. However, fixing that did cause a lot of errors in other providers.

Do you find this current fix acceptable?

JoasE added 30 commits April 15, 2026 21:28

ReadShapedCollection

d6d0227

ReadShapedCollection, ignore includes?

184fea2

Fix do not translate non-persisted properties

984208d

Add tests

62730cc

Copy over fix for ToList projections

d33a451

Implement ReadItem and Paging enumarators

f48d075

Allow inheritance hiarchies in JsonEntityMaterializerRewriter? TODO: …

4d937b8

…Improve

Non value projections TODO: Improve

7bc7bd5

Numbers and mathematics

b7d16f8

Fix inheritance

d524374

WIP

4e4fa83

WIP

f91a45c

WIP

654ae55

WIP

c9e0b86

WIP

611d8a6

WIP

0c6a507

WIP

fecd0d0

WIP

fd1adb8

Revert changes

5ab9c5e

WIP: Working non collections

99dc183

Fix nested collection

fe499e6

WIP: CollectionShaper?

1fd10b8

Fix owned mappings

5747ad1

Select method

9c4de60

Select_nested_collection_on_optional_associate

865a911

ShapedQueryCompilingExpressionVisitor fix?

4bc3bcc

Fix tracking variable

04a1dba

Fix ownerMappings

b3960d7

Fix entityEntry

d2adfb0

Merge branch 'feature/dotnet#34067-ProjectSubDocuments' into feature/d…

a0802db

…otnet#34567-Materialization

JoasE mentioned this pull request Jul 8, 2026

Cosmos: Handle late discriminators in materialization #38576

Open

JoasE added 2 commits July 8, 2026 10:39

Created issue for t

8bf6954

Add check for include

d647f71

Copilot AI review requested due to automatic review settings July 8, 2026 09:35

Copilot started reviewing on behalf of JoasE July 8, 2026 09:35 View session

Copilot AI reviewed Jul 8, 2026

View reviewed changes

JoasE added 3 commits July 8, 2026 11:48

Fix numberprojection tests

df1839e

Small fixes

35f57ee

Skip tests on emula

fc185cf

Copilot AI review requested due to automatic review settings July 8, 2026 10:05

Copilot started reviewing on behalf of JoasE July 8, 2026 10:06 View session

Copilot AI reviewed Jul 8, 2026

View reviewed changes

Apply suggestions from code review

eb0c734

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings July 8, 2026 11:15

Copilot started reviewing on behalf of JoasE July 8, 2026 11:15 View session

Copilot AI reviewed Jul 8, 2026

View reviewed changes

Comment thread test/EFCore.Tests/Storage/Json/JsonReaderDataTest.cs

Comment thread ...Core.Cosmos/Query/Internal/CosmosShapedQueryCompilingExpressionVisitor.QueryingEnumerable.cs

JoasE added 3 commits July 8, 2026 13:24

Add test and change comment

0cc5bd7

Fix nullables for projection numbers

622cdc0

Clean

dfaaf34

Copilot AI review requested due to automatic review settings July 8, 2026 11:27

Copilot started reviewing on behalf of JoasE July 8, 2026 11:28 View session

Copilot AI reviewed Jul 8, 2026

View reviewed changes

Comment thread test/EFCore.Cosmos.FunctionalTests/Query/AdHocJsonQueryCosmosTest.cs Outdated

JoasE added 2 commits July 8, 2026 14:00

Clean null discriminator case

783a1cf

Use default for for reading null properties (matches 10.0) and add tests

a086163

Copilot AI review requested due to automatic review settings July 8, 2026 13:01

Copilot started reviewing on behalf of JoasE July 8, 2026 13:01 View session

Copilot AI reviewed Jul 8, 2026

View reviewed changes

Comment thread src/EFCore.Cosmos/Extensions/Internal/CosmosResponseStreamHelper.cs

Comment thread ...smosShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.ClientMethods.cs

JoasE commented Jul 8, 2026

View reviewed changes

JoasE requested a review from AndriySvyryd July 8, 2026 13:25

Uh oh!

Conversation

JoasE commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JoasE commented Jul 8, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

JoasE Jul 8, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

JoasE commented Jul 2, 2026 •

edited

Loading