Repository navigation
fix(base): keep unevaluated keywords beside allOf for sibling refs - #709
Tjmstudios wants to merge 3 commits into
Conversation
The sibling-ref transformer moved every sibling of a $ref into allOf
member 0. unevaluatedProperties and unevaluatedItems only see what
adjacent keywords evaluated, so inside that member they could no longer
see the properties and items the $ref target evaluated. A schema like
{$ref: Base, unevaluatedProperties: false} then rejected every property
Base declares.
Keep both keywords on the wrapper, beside allOf. The other siblings
still go into member 0, and when unevaluated* are the only siblings,
allOf holds just the $ref. The high-level semantic view does the same,
and renderTransformedRefWithSiblings still collapses the wrapper back
to $ref plus siblings, so rendered documents keep their authored shape.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #709 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 315 315
Lines 39758 39833 +75
=========================================
+ Hits 39758 39833 +75
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| for i := 0; i+1 < len(siblingSchemaNode.Content); i += 2 { | ||
| keyNode, valueNode := siblingSchemaNode.Content[i], siblingSchemaNode.Content[i+1] | ||
| if isUnevaluatedKeyword(keyNode.Value) { | ||
| allOfNode.Content = append(allOfNode.Content, keyNode, valueNode) |
There was a problem hiding this comment.
[P2] Preserve the resource scope of the moved keyword. With sibling $id, $defs, and unevaluatedProperties: {$ref: '#/$defs/Extra'}, this moves the reference to the wrapper while $id and $defs stay in allOf[0]. I compiled GetTransformedRefAllOfSchema() with pb33f/jsonschema/v6. The base output compiled. On this head the compiler looked for #/$defs/Extra at the document root and failed. Please preserve the resource scope and add a compile test that keeps the reference intact.
There was a problem hiding this comment.
Fixed. $id, $schema, $vocabulary, $anchor, $dynamicAnchor, $defs and definitions now stay on the wrapper next to unevaluatedProperties/unevaluatedItems, so #/$defs/Extra resolves against the same $id again, and so does a relative $ref like base. The high-level allOf view moves the same keywords. TestSchemaProxy_BesideAllOfKeywordsMatchTransformer fails if the two lists drift apart.
Added the compile test: TestDocument_SiblingRefUnevaluatedKeywordsKeepResourceScope uses your example. It compiles GetTransformedRefAllOfSchema() with pb33f/jsonschema/v6, so if $id or $defs end up back in allOf[0] the compile fails and the test fails. It also compiles the RenderInline() output, and checks both validate like the authored schema: {"a":1} and {"a":1,"x":"s"} pass, {"a":1,"x":2} and {"a":"s"} fail. This adds github.com/pb33f/jsonschema/v6 to go.mod as a test dependency. If you don't want it there, I can drop the compile step and keep the YAML assertions.
I ran the same cases through santhosh-tekuri/jsonschema, kaptinlin/jsonschema, python-jsonschema and openapi-schema-validator as well. The allOf schema, inline render and bundler output now validate the same as the authored schema in all of them.
There was a problem hiding this comment.
One more thing on this: $id, $schema, $vocabulary, $anchor, $dynamicAnchor, $defs and definitions now always move beside allOf when they're siblings of a $ref, not only when unevaluated* is there. They describe the object holding the $ref, so leaving them on the wrapper means refs keep resolving against the same resource either way.
Also pushed a small follow-up for the codecov gap: the empty-schema build in the high-level view can't fail, so I dropped that check, and the boolean branch now has a test for when the outer schema fails to build.
| schema.AllOf[0] = CreateSchemaProxy(&siblingCopy) | ||
| case 1: | ||
| // every authored sibling was an unevaluated* keyword, so allOf only holds the $ref. | ||
| if len(high.NewNodeBuilder(&siblingCopy, nil).Render().Content) > 0 { |
There was a problem hiding this comment.
[P2] Preserve a false boolean schema during inline rendering. I parsed a $ref whose only sibling was unevaluatedProperties: false, set proxy.Schema().BooleanSchema to a pointer to false, and called proxy.MarshalYAMLInline(). The base emitted a false member in allOf. This branch drops that member, so the output can accept an object that the edited schema rejects. Please handle BooleanSchema explicitly before using mapping content to test for an empty sibling.
There was a problem hiding this comment.
Fixed. BooleanSchema is checked before the empty-mapping check now, so with unevaluatedProperties: false as the only sibling and BooleanSchema = false, MarshalYAMLInline() renders allOf: [false, {$ref}] like main did. The "boolean sibling stays in allOf" subtest covers true and false, including the inline render.
With a boolean sibling, $id/$anchor etc. stay on the wrapper since the $ref resolves against them. unevaluated* are dropped, since the boolean replaces the sibling's assertions.
| if isUnevaluatedKeyword(keyNode.Value) { | ||
| allOfNode.Content = append(allOfNode.Content, keyNode, valueNode) | ||
| } else { | ||
| kept = append(kept, keyNode, valueNode) |
There was a problem hiding this comment.
[P3] Allocate kept only when a keyword moves. For a $ref with title, description, and required, I measured 28 allocations per TransformSiblingRef call on this head versus 25 on the base. All three siblings stay in the original node, so the new slice is discarded. On the first keyword that moves, copy the preceding pairs and start collecting the remaining ones.
There was a problem hiding this comment.
Changed. The sibling node is used as the allOf member until a keyword has to move. At that point the earlier pairs get copied and the rest are collected from there. With only title/description/required siblings it's 25 allocs per call, same as main (previous head was 28). TestSiblingRefTransformer_CreateAllOfStructure catches a bad copy of the earlier pairs.
$id, $schema, $vocabulary, $anchor, $dynamicAnchor, $defs and definitions
now stay on the allOf wrapper with unevaluatedProperties and
unevaluatedItems, so a relative $ref still resolves against the authored
$id, $anchor still names the whole object, and $defs pointers still reach
their subschemas. The high-level allOf view moves the same keywords, the
collapse back to the $ref form only applies to OpenAPI 3.1 and later, and
a boolean sibling renders as allOf: [bool, {$ref}] from every entry point.
The member is only copied when a keyword moves.
The document test compiles the allOf schema and the inline render with
pb33f/jsonschema and checks they validate like the authored schema, which
adds github.com/pb33f/jsonschema/v6 to go.mod.
Summary
The sibling-ref transformer moves every sibling of
$refintoallOfmember 0. ForunevaluatedProperties/unevaluatedItemsthat's wrong, since they can't see what the$reftarget evaluated from inside member 0. For$id,$defs,$anchorand the other resource keywords it changes what references resolve against. This keeps both groups on the wrapper, besideallOf.Problem
Closedis transformed to:In 2020-12,
unevaluatedPropertiesonly sees annotations from adjacent keywords. Member 0 has none, so it rejects every property. libopenapi-validator validates againstRenderInline, so it rejects{"a": 1}.unevaluatedItemswith aprefixItemstarget fails the same way. Affects 3.1 and 3.2.Moving only
unevaluated*out isn't enough when the schema has its own$id:If
$idand$defsgo into member 0,#/$defs/Extraon the wrapper andbasein member 1 no longer resolve againsthttps://example.com/schemas/scoped. A$anchorin member 0 names only that member, so a$ref: '#anchor'skips the$reftarget.Changes
createAllOfStructureWithSiblingNode:unevaluatedProperties,unevaluatedItems,$id,$schema,$vocabulary,$anchor,$dynamicAnchor,$defsanddefinitionsstay on the wrapper. Other siblings go in member 0. If no other siblings are left,allOfholds just the$ref. The sibling node is used as member 0 as is unless a keyword moves, so the common case doesn't allocate more thanmain.buildSemanticAllOfSchemaView: same shape in the high-level model, using the same keyword list. A test fails if the two lists drift apart.renderTransformedRefWithSiblings: collapses the new shape back to$ref+ siblings soRenderstill round-trips. Only for 3.1 and later, since 3.0 ignores siblings of$ref.allOf: [<bool>, {$ref}]from every entry point, includingMarshalYAMLInline. Resource keywords stay on the wrapper.unevaluated*are dropped, since the boolean replaces the sibling's assertions.GetTransformedRefAllOfSchemaandBuildTransformedRefSemanticSchemanow return a one-memberallOfwhen the only siblings are keywords that stay on the wrapper. Nothing in the repo indexes member 1. Doc comments updated.Tests
New tests and subtests in
datamodel/low/base/sibling_ref_transformer_test.go,datamodel/high/base/schema_proxy_test.go,bundler/bundler_test.go,document_test.goandrenderer/schema_renderer_test.go.TestDocument_SiblingRefUnevaluatedKeywordsKeepResourceScopecompilesGetTransformedRefAllOfSchema()and theRenderInline()output forScopedwithpb33f/jsonschema/v6and checks they validate like the authored schema. This addsgithub.com/pb33f/jsonschema/v6 v6.0.3to go.mod (andgolang.org/x/textas indirect). It's only used by tests.As a reference, the cases from these tests were run through pb33f/jsonschema v6.0.3, santhosh-tekuri/jsonschema v6.0.3, kaptinlin/jsonschema v0.9.10, python-jsonschema 4.26.0 and openapi-schema-validator 0.9.0. Each checked the authored schema, the
allOfschema, theRenderInlineoutput and the bundler output. On the authored schema all five give the result 2020-12 requires. Whether the libopenapi output gives the same result:main$ref+required+unevaluatedProperties: false{"a": 1}$ref+unevaluatedProperties: false{"a": 1}$ref+unevaluatedItemsoverprefixItems[1]$ref+unevaluatedProperties: false(bundled){"a": 1}$ref: basewith$id, recursive$defsandunevaluatedProperties$anchorbeside$ref, used through$ref: '#anchor'{"t": {"a": "x"}}basedoesn't resolvekaptinlin can't follow a JSON pointer into
definitionsunder 2020-12, so it errors on the authored schema in the last case too.