diff --git a/docs/design/graph-object-binding.md b/docs/design/graph-object-binding.md index eddeed1..72b85ae 100644 --- a/docs/design/graph-object-binding.md +++ b/docs/design/graph-object-binding.md @@ -159,9 +159,18 @@ class Person(LinkedBaseModel): `__set__` is declared under `TYPE_CHECKING` only, so at runtime the descriptor stays **non-data** and the instance-`__dict__` cache from 3.1 is untouched. -`__get_pydantic_core_schema__` builds the schema of the *target*, so the emitted -JSON Schema is byte-identical to the plain annotation - `$ref`, arrays, unions -and forward references included. +`__get_pydantic_core_schema__` builds the schema of the *target*, so what +pydantic sees is identical to the plain annotation - arrays, unions and forward +references included, and both spellings therefore emit the same document. + +What that document says about a link is a separate question. A link serialises +to an IRI, so the published property is `{"type": "string", "x-oold-range": …}`, +or an array of strings for a to-many link - which is what a real OSW schema +carries. The `$ref` / `allOf` form is what `generator.preprocess` builds so that +`datamodel-code-generator` emits `Optional[Bar]` instead of a string field; it +is a code generation shape, and publishing it described a document the library +never writes. A union arm keeps its union: `str | Location | None` genuinely +accepts a literal, a reference or an inline object. #### Optionality is declared, not assumed @@ -325,13 +334,20 @@ read it?" - so two carriers: | declaration | stored as | enforced | on violation | |---|---|---|---| | `Link[T]` - no `None` arm | `_AutoLink.optional = False` | on read | `LinkNotResolved` | -| `OoldField(required=True)` | `_AutoLink.required_iri`, precomputed into `cls.__required_links__`; emitted as `x-oold-required-iri` and into the standard `required` array | in `__init__` | `ValueError: ... is required but not set` | +| `OoldField(required=True)` | `_AutoLink.required_iri`, precomputed into `cls.__required_links__`; reaches the schema as the standard `required` array | in `__init__` | `ValueError: ... is required but not set` | A link is never required at the *pydantic* level, because its value is routed out of the payload before validation - which is why the legacy binding declared every generated link field `Optional[...]` and carried requiredness in the -keyword. A bare `Link[T]` annotation with no default is read as required, which -is what Python means by "no default" everywhere else. +keyword. `x-oold-required-iri` is that keyword, and it stays internal: a schema +states requiredness through `required` alone, and the annotation exists only to +carry the requirement across the point where code generation drops the property +from `required` so the emitted field is `Optional`. + +A bare `Link[T]` annotation with no default is **optional**. "No default means +required" reads well in plain Python, but requiredness propagates into +resolution - the failure mode described next - and links are declared far more +often than they are required, so the terse form is the common case. **Why not put requiredness in the annotation.** `required` -> `Link[T]`, absence -> `Link[T | None]` reads well and was the first proposal. It fails on a diff --git a/docs/how-to/object-graph-mapping.md b/docs/how-to/object-graph-mapping.md index 40ae918..4320bc4 100644 --- a/docs/how-to/object-graph-mapping.md +++ b/docs/how-to/object-graph-mapping.md @@ -228,6 +228,26 @@ alice.knows[0] # a Person, not a str Nothing changes at runtime - same resolution, same JSON Schema. Only what the checker sees changes. +### What a link looks like in the emitted schema + +A link serialises to an IRI, and the schema says so: + +```json +"employer": {"type": "string", "x-oold-range": "https://example.org/Organization"}, +"friends": {"type": "array", "items": {"type": "string"}, + "x-oold-range": "https://example.org/Person"} +``` + +Not a `$ref` to the target. The `$ref` form is what code generation builds so +that the generated field is `Optional[Organization]` rather than a string; +published, it would describe a document this library never writes. A union arm +keeps its union - `str | Location | None` really does accept all three. + +Give every link a `@context` term, and `"@type": "@id"` to make the value a +reference. A strictly array-typed property also needs `"@container": "@set"`, +or a single-element array compacts back to a bare value and no longer matches +the schema. + ### Optionality is declared `Link[T]` reads as `T`, so a chain needs no guard at every hop. `Link[T | None]` @@ -268,21 +288,35 @@ OoldField(required=None, range=None, link=None, **field_kwargs) | argument | effect | |---|---| -| `required` | the link must be supplied at construction; omitting it raises `ValueError`. Emitted as `x-oold-required-iri` **and** into the standard `required` array | +| `required` | the link must be supplied at construction; omitting it raises `ValueError`. Reaches the schema as the standard `required` array, and nothing else | | `range` | target schema IRI, emitted as `x-oold-range`. **Do not pass it**: omitted, it is derived from the annotation, which already names the target | | `link` | marks the field a link where the annotation does not imply it, as in a union arm. Redundant with `Link[T]` / `LinkList[T]` | -| `required_iri` | deprecated spelling of `required`, kept because generated packages pass it. Same emitted keyword | +| `required_iri` | deprecated spelling of `required`, kept because generated packages pass it | | `**field_kwargs` | passed to `pydantic.Field` (`alias`, `description`, `default_factory`, ...). `default=None` is supplied unless you pass a `default_factory` | -A link annotation with **no default at all** means required, as it does anywhere -else in Python: +Requiredness is always explicit. A link annotation with no default is optional, +like one with a bare `OoldField()`: ```python -manager: Link[Organization] # required -manager: Link[Organization] = OoldField() # optional -manager: Link[Organization] = OoldField(required=True) # required, explicit +father: Link["Person"] # optional +father: Link["Person"] = OoldField() # optional +father: Link["Person"] = OoldField(required=True) # required, explicit ``` +"No default means required" reads well in plain Python, but requiredness +propagates into resolution - reading a link constructs the target - so a +required link makes every stored document lacking it unconstructible. A +self-referential link like `father` could then never be satisfied by a real +dataset, and links are declared far more often than they are required. + +!!! note "`x-oold-required-iri` is internal" + A schema states requiredness through `required` and nothing else. + `x-oold-required-iri` is an oold-python annotation on a *field*: it carries + the requirement across the point where code generation has to drop the + property from `required`, so the generated field is `Optional` - a link can + never be required at the pydantic level, because its value is routed out of + the payload before validation. It does not appear in a published schema. + ### Requiredness is a field argument, not the annotation "Must the caller supply it?" and "what do I get when I read it?" are different diff --git a/src/oold/model/_compat.py b/src/oold/model/_compat.py index fb38247..a81557b 100644 --- a/src/oold/model/_compat.py +++ b/src/oold/model/_compat.py @@ -158,6 +158,15 @@ def get_cls_iri(cls) -> Any: ``GenericLinkedBaseModel`` only declares this abstract, so without an implementation it silently returns ``None`` - which would break the downstream callers and the type registry alike. + + This answers **identity**: it merges the ``$id`` with the ``type`` + field's default(s), which are the instances' rdf:type, so the registry + can resolve a document's ``type`` back to a class. It is not a + **location** and must not be used as one - a consumer dereferences a + location, and the two only coincide sometimes. + ``wiki_data.Person`` answers ``["http://www.wikidata.org/entity/Q5", + "Item:Q5"]`` and publishes no schema at either. Emitting + ``x-oold-range`` uses ``$id`` alone; see ``_AutoLink.range_iri``. """ schema = getattr(cls, "model_config", {}).get("json_schema_extra") or {} if callable(schema): diff --git a/src/oold/model/_descriptor.py b/src/oold/model/_descriptor.py index 3b60816..94b5314 100644 --- a/src/oold/model/_descriptor.py +++ b/src/oold/model/_descriptor.py @@ -189,12 +189,20 @@ def _neutralised(info: Any) -> Any: namespace[field_name] = _neutralised(info) continue if field_name not in namespace and _is_link_annotation(annotation): - # `manager: Link[Org]` with nothing assigned. Read as Python reads - # it - no default means required - but a link cannot be required as - # a pydantic field, because its value never reaches validation. - # Left alone, every construction failed with a misleading - # "Field required" about a value that had in fact been supplied. - namespace[field_name] = OoldField(required=True) + # `manager: Link[Org]` with nothing assigned. It still needs a field + # carrying default=None - a link cannot be required at the pydantic + # level, because its value never reaches validation, and left alone + # every construction failed with a misleading "Field required" about + # a value that had in fact been supplied. + # + # It is *not* required in the OO-LD sense: that is what + # OoldField(required=True) says. Requiredness propagates into + # resolution - resolving a link constructs the target - so a + # required link makes every stored document lacking it + # unconstructible, and a self-referential link could never be + # satisfied by a real dataset. Links are declared far more often + # than they are required, so the terse form is the common case. + namespace[field_name] = OoldField() continue # A Field() living in Annotated metadata rather than as the assigned # value was never seen here, so its default survived and was evaluated @@ -221,6 +229,59 @@ def _neutralised(info: Any) -> Any: return defaults +def _permits_a_string(prop: dict) -> bool: + """Whether the property already allows a bare IRI.""" + if prop.get("type") == "string": + return True + for arms in ("anyOf", "oneOf"): + for arm in prop.get(arms) or []: + if isinstance(arm, dict) and arm.get("type") == "string": + return True + return False + + +def _as_reference(prop: dict, many: bool) -> None: + """Rewrite a link property to the shape its instances actually take. + + A link serialises to an IRI - ``{"employer": "ex:acme"}`` - and a published + OO-LD schema says so: ``{"type": "string", "x-oold-range": …}``. The + ``$ref`` / ``allOf`` form is a *code generation* shape, produced by + ``generator.preprocess`` so that datamodel-code-generator emits + ``Optional[Bar]`` rather than a string field. Emitting it describes a + document the library never writes, and the JSON-LD round trip fails on it: + with ``"@type": "@id"`` on the term, an embedded object loses its + properties. + + Union arms are left alone. ``str | Location | None`` genuinely accepts a + literal, a reference or an inline object, and its schema already says so. + """ + if _permits_a_string(prop): + return + # "range" is the legacy spelling of x-oold-range and is still what generated + # packages declare; dropping it here would silently delete the only thing + # marking the property a link in those schemas. + keep = ("title", "description", "default", "range", "format") + kept = {k: v for k, v in prop.items() if k in keep or k.startswith("x-")} + prop.clear() + prop.update(kept) + # OOLD-EXT-6ea3 (SHOULD): an IRI-valued property constrains its lexical form + # with an IRI-family format, and OOLD-EXT-1f92 recommends iri-reference - it + # admits absolute IRIs, compact IRIs and context-relative references alike, + # which is what instances actually carry. It is also the second of the three + # reference signals a frame derivation looks for (OOLD-EXT-68fa, see + # `oold.validation.frame.reference_properties`). A format the declaration + # already states is left alone. + reference: dict[str, Any] = {"type": "string"} + if "format" not in kept: + reference["format"] = "iri-reference" + if many: + prop.pop("format", None) + prop["type"] = "array" + prop["items"] = reference + else: + prop.update(reference) + + def _namespace_annotations(namespace: dict) -> dict: """The annotations of a class body being built, on any Python version. @@ -943,21 +1004,29 @@ def _target_cls(self, owner: Any) -> Any: return target def range_iri(self, owner: Any = None) -> Any: - """The target's schema IRI, for deriving ``x-oold-range``. + """Where the target's schema lives, for deriving ``x-oold-range``. ``Link[T]`` already names the target, so repeating it in ``OoldField(range=...)`` states the same thing twice and lets the two disagree. The schema is derived from the annotation instead. + + A **location**, not an identity. A consumer dereferences this: code + generation fetches the target schema, and a form editor renders the + targets it allows. ``get_cls_iri()`` answers identity - it merges the + ``$id`` with the ``type`` field's default(s), which are the instances' + rdf:type - so deriving the range from it published identities with + nothing to fetch at them, such as a Wikidata class IRI. Where the two + coincide nothing changes; where they differ, only ``$id`` is right. + + A class that does not say where its schema lives contributes no range. + The property is still marked a reference by its ``format`` - the second + signal in ``OOLD-EXT-68fa``. """ target = self._target_cls(owner) - get_iri = getattr(target, "get_cls_iri", None) - if get_iri is None: - return None - try: - return get_iri() or None - except Exception: - # a target that cannot name itself simply contributes no range + extra = getattr(target, "model_config", {}).get("json_schema_extra") if target is not None else None + if callable(extra) or not isinstance(extra, dict): return None + return extra.get("$id") or None def __get__(self, obj: Any, objtype: Any = None) -> Any: if obj is None: @@ -1252,14 +1321,22 @@ def __get_pydantic_json_schema__(cls, core_schema_: Any, handler: Any) -> Any: not resolvable at class-creation time, and a ``Field()`` object shared between models must not be mutated in place. """ - schema = handler(core_schema_) + document = handler(core_schema_) try: - schema = handler.resolve_ref_schema(schema) + schema = handler.resolve_ref_schema(document) except Exception: - return schema + return document + # A self-referential model comes back as {"$defs": …, "$ref": "#/$defs/X"}, + # which leaves the *document* without the $id the OO-LD document tier + # requires - the $id sits on the definition instead. Carry it up, so the + # emitted document identifies itself either way. + if isinstance(document, dict) and isinstance(schema, dict) and document is not schema: + identifier = schema.get("$id") + if identifier and "$id" not in document: + document["$id"] = identifier properties = schema.get("properties") if isinstance(schema, dict) else None if not properties: - return schema + return document link_fields = cls.__link_fields__ aliases = cls.__link_aliases__ required = schema.get("required") @@ -1269,23 +1346,32 @@ def __get_pydantic_json_schema__(cls, core_schema_: Any, handler: Any) -> Any: if descr is None or not isinstance(prop, dict): continue if descr.required_iri: - # A link is never required at the pydantic level - its value is - # routed out of the payload before validation - so pydantic - # leaves it out of `required`. Stating it only in - # x-oold-required-iri would hide the constraint from every - # plain JSON Schema validator. + # A schema states requiredness through the standard `required` + # array and nothing else. A link is never required at the + # pydantic level - its value is routed out of the payload before + # validation - so pydantic leaves it out and it is added here. if required is None: required = schema["required"] = [] if key not in required: required.append(key) - if prop.get("x-oold-range") or prop.get("range"): - continue - iri = descr.range_iri(cls) - if iri: - prop["x-oold-range"] = iri - # the range says "link" on its own; the marker was a stand-in - prop.pop("x-oold-link", None) - return schema + # ... and a required property may not also declare a default: + # nothing satisfies both, and a consumer generating an instance + # from this schema produces one its own schema rejects. + prop.pop("default", None) + # x-oold-required-iri and x-oold-link are oold-python annotations on + # a *field*. The first carries requiredness across the point where + # the property has to leave `required` so the generated field is + # Optional (see generator.preprocess); the second marks a link whose + # target comes from the annotation. Neither is in the OO-LD keyword + # vocabulary, so neither belongs in a published document. + prop.pop("x-oold-required-iri", None) + if not (prop.get("x-oold-range") or prop.get("range")): + iri = descr.range_iri(cls) + if iri: + prop["x-oold-range"] = iri + prop.pop("x-oold-link", None) + _as_reference(prop, descr.many) + return document @classmethod def __pydantic_init_subclass__(cls, **kwargs: Any) -> None: diff --git a/src/oold/static.py b/src/oold/static.py index b313821..6172e43 100644 --- a/src/oold/static.py +++ b/src/oold/static.py @@ -644,15 +644,50 @@ def handle_property(property): return schema -def _export_schema_from_dynamic_model(model_cls: BaseModel | BaseModel_v1) -> dict: +def _own_field_names(model_cls: BaseModel | BaseModel_v1, cutoff_base_cls: tuple = ()) -> set: + """The field names this class introduces, not the ones it inherits. + + ``model_fields`` / ``__fields__`` are **flattened** by pydantic: they hold + every inherited field too, so building a "model itself" copy from them + produced the whole hierarchy again. What a level adds is the difference + against its bases. + + Restating an inherited property is not merely redundant: a restatement can + relax a constraint the ancestor declared, which OOLD-CMP-f3c7 forbids. + """ + + def names(cls) -> set: + fields = getattr(cls, "model_fields", None) + if fields is None: + fields = getattr(cls, "__fields__", None) or {} + return set(fields) + + inherited: set = set() + bases = cutoff_base_cls or getattr(model_cls, "__bases__", ()) + for base in bases: + if base in (BaseModel, BaseModel_v1): + continue + inherited |= names(base) + return names(model_cls) - inherited + + +def _export_schema_from_dynamic_model( + model_cls: BaseModel | BaseModel_v1, + only: set | None = None, +) -> dict: """Export the OO-LD schema of a single pydantic model. Class hierarchy is not considered, only the model itself by generating a model copy without base classes. + + ``only`` restricts the copy to the given field names, which is what makes + the copy a *level* rather than the flattened model. """ if issubclass(model_cls, BaseModel): field_dict = {} for field_name, field in model_cls.model_fields.items(): + if only is not None and field_name not in only: + continue field_dict[field_name] = (field.annotation, field) # create model dynamically @@ -669,6 +704,8 @@ def _export_schema_from_dynamic_model(model_cls: BaseModel | BaseModel_v1) -> di model_cls: BaseModel_v1 = model_cls # type: ignore[assignment] field_dict = {} for field_name, model_field in model_cls.__fields__.items(): + if only is not None and field_name not in only: + continue field_dict[field_name] = (model_field.annotation, model_field.field_info) # create model dynamically @@ -714,7 +751,7 @@ def export_schema( # if partial_mode == PartialSchemaExportMode.BASE_CLASS_CUTOFF: for baseclass in cutoff_base_cls: - if baseclass in [BaseModel, BaseModel_v1]: + if _is_library_base(baseclass): continue schema = _get_schema(baseclass) if schema is not None: @@ -734,18 +771,12 @@ def export_schema( # try the follwing schema extra attributes: $id, iri, class name import_ref = None if issubclass(model_cls, BaseModel): - import_ref = baseclass.model_config.get("json_schema_extra", {}).get("$id", None) - if import_ref is None: - import_ref = baseclass.model_config.get("json_schema_extra", {}).get("iri", None) - if import_ref is None: - import_ref = baseclass.__name__ + extra = baseclass.model_config.get("json_schema_extra") or {} + import_ref = extra.get("$id") or extra.get("iri") or baseclass.__name__ if issubclass(model_cls, BaseModel_v1): - import_ref = baseclass.__config__.schema_extra.get("$id", None) - if import_ref is None: - import_ref = baseclass.__config__.schema_extra.get("iri", None) - if import_ref is None: - import_ref = baseclass.__name__ + extra = getattr(baseclass.__config__, "schema_extra", None) or {} + import_ref = extra.get("$id") or extra.get("iri") or baseclass.__name__ if import_ref is not None: imports.append(import_ref) @@ -787,7 +818,16 @@ def export_schema( elif partial_mode == PartialSchemaExportMode.BASE_CLASS_CUTOFF: # option 2: export the schema of a model up to the specified # base class by cutoff the class hierarchy - model_schema_diff = _export_schema_from_dynamic_model(model_cls) + model_schema_diff = _export_schema_from_dynamic_model( + model_cls, only=_own_field_names(model_cls, cutoff_base_cls) + ) + # ... and compose the level onto its bases. Without this the schema + # hierarchy did not mirror the class hierarchy at all: the `imports` + # below reached @context while nothing reached the schema body, so + # the document claimed an inheritance it never declared - the exact + # inverse of OOLD-CMP-b926 ("every $ref is reflected in @context"). + if imports: + model_schema_diff["allOf"] = [{"$ref": ref} for ref in imports] context = None if "@context" in model_schema_diff: @@ -866,4 +906,96 @@ def export_schema( del result_schema["$ref"] result_schema = _inverse_preprocess(result_schema) + _state_requiredness_in_the_required_array(result_schema) + _prune_unreferenced_defs(result_schema) return result_schema + + +def _is_library_base(cls: Any) -> bool: + """Whether a base contributes no schema of its own. + + ``LinkedBaseModel`` and friends are this library's machinery, not published + schemas, so a subclass must not compose onto them: ``{"$ref": + "LinkedBaseModel"}`` resolves to nothing, and the same string reached + ``@context``. A user's own base without an ``$id`` is a different matter - + it is named by its class, which is the established convention here. + """ + if cls in (BaseModel, BaseModel_v1): + return True + return str(getattr(cls, "__module__", "")).startswith("oold.") + + +def _prune_unreferenced_defs(schema: dict) -> None: + """Drop ``$defs`` entries nothing points at. + + A partial export builds its level from a dynamically created copy of the + model, and pydantic leaves a definition for that copy behind - holding the + *flattened* model, inherited properties and all. Unreferenced, it is not + merely dead weight: it contradicts the level beside it, and every check that + walks a schema tree reports against it. + """ + defs = schema.get("$defs") + if not isinstance(defs, dict): + return + + def referenced(node: Any) -> set: + out: set = set() + if isinstance(node, dict): + for key, value in node.items(): + if key == "$ref" and isinstance(value, str) and value.startswith("#/$defs/"): + out.add(value.split("/")[-1]) + else: + out |= referenced(value) + elif isinstance(node, list): + for item in node: + out |= referenced(item) + return out + + live = referenced({k: v for k, v in schema.items() if k != "$defs"}) + # a kept definition may reference others, so close over the set + while True: + grown = set(live) + for name in live: + if name in defs: + grown |= referenced(defs[name]) + if grown == live: + break + live = grown + + for name in [n for n in defs if n not in live]: + del defs[name] + if not defs: + del schema["$defs"] + + +def _state_requiredness_in_the_required_array(schema: dict) -> None: + """Move link requiredness into ``required``, and take the annotation out. + + A schema states requiredness through the standard ``required`` array and + nothing else. ``x-oold-required-iri`` - spelled ``x_oold_required_iri`` by + pydantic v1, which cannot pass a hyphenated keyword to ``Field()`` - is an + oold-python annotation on a *field*: it carries the requirement across the + point where the property has to leave ``required`` so that the generated + field is ``Optional`` (see ``generator.preprocess``). It is not in the OO-LD + keyword vocabulary and does not belong in a published document. + + The v2 path has already done this in ``__get_pydantic_json_schema__``, so + this is a no-op there; it is what fixes the v1 path, whose schema comes from + pydantic v1 and never passes through that hook. + """ + properties = schema.get("properties") + if not isinstance(properties, dict): + return + required = schema.get("required") + for name, prop in properties.items(): + if not isinstance(prop, dict): + continue + is_required = bool(prop.pop("x-oold-required-iri", False)) | bool(prop.pop("x_oold_required_iri", False)) + if not is_required: + continue + if required is None: + required = schema["required"] = [] + if name not in required: + required.append(name) + # nothing satisfies both a requirement and a default + prop.pop("default", None) diff --git a/tests/test_auto_descriptor_binding.py b/tests/test_auto_descriptor_binding.py index c6b2f78..1e0f01a 100644 --- a/tests/test_auto_descriptor_binding.py +++ b/tests/test_auto_descriptor_binding.py @@ -168,8 +168,9 @@ def test_typed_extras_validate(): def test_extras_reach_the_json_schema(): - prop = Person.model_json_schema()["$defs"]["Person"]["properties"]["knows"] - assert prop["x-oold-range"] == "Person" + schema = Person.model_json_schema() + props = schema.get("properties") or schema["$defs"]["Person"]["properties"] + assert props["knows"]["x-oold-range"] == "Person" @pytest.mark.xfail( diff --git a/tests/test_link_annotation.py b/tests/test_link_annotation.py index eff246b..2031611 100644 --- a/tests/test_link_annotation.py +++ b/tests/test_link_annotation.py @@ -77,7 +77,12 @@ def test_annotation_alone_declares_the_link(): def test_schema_matches_the_plain_spelling(): - """Pydantic is handed the target's schema, so the output is unchanged.""" + """Both spellings declare a link, so both emit the same shape. + + A link serialises to an IRI, and the published schema says so: an array of + strings here, a string for a to-one link. The ``$ref`` form belongs to code + generation, not to the document. + """ def props(model): schema = model.model_json_schema() @@ -85,10 +90,16 @@ def props(model): annotated = props(Person)["knows"] plain = props(Plain)["knows"] - assert annotated["type"] == "array" - assert plain["anyOf"][0]["items"] == {"$ref": "#/$defs/Person"} - # the annotated form carries the None arm it declares - assert {"$ref": "#/$defs/Person"} in annotated["items"]["anyOf"] + reference = {"type": "string", "format": "iri-reference"} + assert annotated["type"] == plain["type"] == "array" + assert annotated["items"] == plain["items"] == reference + # neither model declares a $id, so there is no location to derive a range + # from; the legacy declaration keeps the keyword it states itself + assert "x-oold-range" not in annotated + assert plain["range"] == "Person" + # a to-one link is a bare IRI + assert props(Person)["employer"]["type"] == "string" + assert props(Person)["employer"]["format"] == "iri-reference" def test_construct_by_iri_object_and_json(store): diff --git a/tests/test_notation.py b/tests/test_notation.py index 7784c81..421aa1c 100644 --- a/tests/test_notation.py +++ b/tests/test_notation.py @@ -11,7 +11,7 @@ """ import pytest -from pydantic import Field +from pydantic import ConfigDict, Field from oold.backend.document_store import SimpleDictDocumentStore from oold.backend.interface import SetResolverParam, set_resolver @@ -219,14 +219,34 @@ def test_range_is_derived_from_the_annotation(): """Presence of ``x-oold-range`` is what makes a property a link, so the annotation has to put it there - otherwise the recommended declaration emits a schema that does not round-trip through code generation, and the - only way to get one is to repeat the target in ``OoldField(range=...)``.""" - props = _properties(Person) - assert props["knows"]["x-oold-range"] == Person.get_cls_iri() - assert props["friends"]["x-oold-range"] == Person.get_cls_iri() - assert props["employer"]["x-oold-range"] == Org.get_cls_iri() - assert props["location"]["x-oold-range"] == Location.get_cls_iri() + only way to get one is to repeat the target in ``OoldField(range=...)``. + + It is the target's **location** - its ``$id`` - because a consumer + dereferences it: code generation fetches the schema, a form editor renders + the targets it allows. + """ + + class Located(OoldModel): + model_config = ConfigDict(json_schema_extra={"$id": "https://example.org/NLocated"}) + id: str + type: str | None = "ex:NLocated" + peer: Link["Located"] = OoldField() + + Located.model_rebuild() + props = _properties(Located) + assert props["peer"]["x-oold-range"] == "https://example.org/NLocated" # the marker was a stand-in for the range; it goes once the range is there - assert "x-oold-link" not in props["knows"] + assert "x-oold-link" not in props["peer"] + + +def test_a_class_that_does_not_say_where_its_schema_lives_has_no_range(): + """get_cls_iri() answers identity - it merges the $id with the type field's + defaults, which are the instances' rdf:type. Deriving a range from it + published identities with nothing to fetch at them. Without a $id there is + no location to publish; `format` still marks the property a reference.""" + props = _properties(Person) # declares a type default, no $id + assert "x-oold-range" not in props["knows"] + assert props["knows"]["items"]["format"] == "iri-reference" def test_an_explicit_range_is_not_overwritten(): @@ -258,11 +278,12 @@ class Chain(OoldModel): with pytest.raises(LinkNotResolved): _ = c.father # optional to supply, still mandatory to read - props = _properties(Chain) - assert props["manager"]["x-oold-required-iri"] is True - # a plain JSON Schema validator only sees the standard array + # the schema states requiredness through `required` and nothing else; + # x-oold-required-iri is an oold-python field annotation and stays internal assert "manager" in _required(Chain) assert "father" not in _required(Chain) + assert "x-oold-required-iri" not in _properties(Chain)["manager"] + assert Chain.__link_fields__["manager"].required_iri is True def test_required_iri_is_still_accepted(): @@ -276,14 +297,19 @@ class Old(OoldModel): Old.model_rebuild() with pytest.raises(ValueError, match="manager is required"): Old(id="ex:o") - assert _properties(Old)["manager"]["x-oold-required-iri"] is True + assert "manager" in _required(Old) + assert Old.__link_fields__["manager"].required_iri is True -def test_a_link_annotation_without_a_default_is_required(): - """No default means required, as it does anywhere else in Python. A link is - never required at the pydantic level - its value is routed out before - validation - so this used to fail with a misleading "Field required" about - a value that had in fact been supplied.""" +def test_a_link_annotation_without_a_default_is_optional(): + """Links are declared far more often than they are required, so the terse + form is the common case. + + Requiredness is explicit because it propagates into resolution: resolving a + link constructs the target, so a required link makes every stored document + lacking it unconstructible - and a self-referential link, `father`, could + then never be satisfied by a real dataset. + """ class Bare(OoldModel): id: str @@ -291,6 +317,6 @@ class Bare(OoldModel): manager: Link["Org"] Bare.model_rebuild() + assert Bare(id="ex:b").link_iris("manager") is None # constructs unset assert Bare(id="ex:b", manager="ex:acme").link_iris("manager") == "ex:acme" - with pytest.raises(ValueError, match="manager is required"): - Bare(id="ex:b") + assert Bare.__link_fields__["manager"].required_iri is False diff --git a/tests/test_schema_generation.py b/tests/test_schema_generation.py index 412721c..e2021da 100644 --- a/tests/test_schema_generation.py +++ b/tests/test_schema_generation.py @@ -95,6 +95,10 @@ class MyCustomSchema(MyRootSchema): "MyRootSchema", {"my_property": "https://example.org/my_property"}, ], + # the level composes onto its base, and @context mirrors that in the + # same order - a context entry without a matching $ref claims an + # inheritance the schema never declares (OOLD-CMP-b926, OOLD-CMP-e4a3) + "allOf": [{"$ref": "MyRootSchema"}], "$defs": { "SubObject": { "title": "SubObject", diff --git a/tests/test_validation/test_emitted_schema_conformance.py b/tests/test_validation/test_emitted_schema_conformance.py new file mode 100644 index 0000000..8099b59 --- /dev/null +++ b/tests/test_validation/test_emitted_schema_conformance.py @@ -0,0 +1,150 @@ +"""A schema this library emits must satisfy the spec it implements. + +`oold-python` is the OO-LD reference implementation, and rc.3's own +`docs/migration/from-python.md` points users at `model_json_schema()` as the way +to emit a schema from a model. So what a model emits is not an internal detail: +it is the published artifact, and it has to validate. +""" + +import json + +import pytest +from pydantic import ConfigDict + +from oold.model import LINK_NOTATIONS_ACTIVE, Link, LinkedBaseModel, LinkList, OoldField +from oold.validation import failure_reasons, validate_schema + +pytestmark = pytest.mark.skipif( + not LINK_NOTATIONS_ACTIVE, + reason=( + "OOLD_DESCRIPTOR_BINDING=0 selects the legacy binding, which has no " + "schema-emission hook. It is the deprecated opt-out and is not held to " + "the emission contract; the descriptor binding is what publishes." + ), +) + + +class Org(LinkedBaseModel): + model_config = ConfigDict( + json_schema_extra={ + "@context": { + "schema": "https://schema.org/", + "id": "@id", + "type": "@type", + "name": "schema:name", + }, + "$id": "https://example.org/Organization", + } + ) + id: str + name: str | None = None + + @classmethod + def get_cls_iri(cls): + return "https://example.org/Organization" + + +class Person(LinkedBaseModel): + model_config = ConfigDict( + json_schema_extra={ + "@context": { + "schema": "https://schema.org/", + "id": "@id", + "type": "@type", + "name": "schema:name", + # every link needs a term too, and "@type": "@id" is what makes + # the value a reference: an unmapped property is dropped by + # JSON-LD expansion and the round-trip check catches it + "employer": {"@id": "schema:worksFor", "@type": "@id"}, + "mentor": {"@id": "schema:knows", "@type": "@id"}, + # a strictly array-typed property must declare @container, or a + # single-element array compacts back to a bare value and the + # reconstruction no longer satisfies the schema (OOLD-RT-08f2) + "friends": {"@id": "schema:follows", "@type": "@id", "@container": "@set"}, + }, + "$id": "https://example.org/Person", + } + ) + id: str + name: str | None = None + employer: Link["Org"] = OoldField(required=True) + mentor: Link["Person | None"] = OoldField() + friends: LinkList["Person"] = OoldField() + + @classmethod + def get_cls_iri(cls): + return "https://example.org/Person" + + +Person.model_rebuild() + + +def _emit(tmp_path, schema: dict): + schema.setdefault("$schema", "https://oo-ld.org/latest/meta/oold-meta-schema.json") + path = tmp_path / "Person.json" + path.write_text(json.dumps(schema, indent=2), encoding="utf-8") + return path + + +def test_requiredness_is_stated_only_by_the_required_array(): + """``x-oold-required-iri`` is an oold-python annotation on a *field*, not a + spec keyword. It carries the requirement across the point where the property + has to leave ``required`` so the generated field is Optional (see + ``generator.preprocess``); it must not reach the published document.""" + props = Person.export_schema()["properties"] + assert "employer" in Person.export_schema()["required"] + assert "x-oold-required-iri" not in props["employer"] + assert "x-oold-link" not in props["employer"] + # a required property may not also declare a default - nothing satisfies both + assert "default" not in props["employer"] + # the internal annotation still drives construction + assert Person.__link_fields__["employer"].required_iri is True + + +def test_optional_links_keep_their_derived_range(): + schema = Person.export_schema() + assert "mentor" not in schema.get("required", []) + assert schema["properties"]["mentor"]["x-oold-range"] == "https://example.org/Person" + assert schema["properties"]["friends"]["x-oold-range"] == "https://example.org/Person" + + +def test_export_schema_validates(tmp_path): + report = validate_schema(_emit(tmp_path, Person.export_schema())) + assert report.passed, failure_reasons(report) + + +def test_model_json_schema_validates(tmp_path): + report = validate_schema(_emit(tmp_path, Person.model_json_schema())) + assert report.passed, failure_reasons(report) + + +def test_a_link_declares_an_iri_family_format(): + """OOLD-EXT-6ea3 (SHOULD) wants an IRI-valued property to constrain its + lexical form; OOLD-EXT-1f92 recommends iri-reference, which admits absolute + IRIs, compact IRIs and context-relative references alike. It is also the + second reference signal a frame derivation looks for (OOLD-EXT-68fa).""" + props = Person.export_schema()["properties"] + assert props["employer"]["format"] == "iri-reference" + assert props["friends"]["items"]["format"] == "iri-reference" + assert "format" not in props["friends"] # the array itself is not an IRI + + +def test_a_declared_format_is_not_overwritten(): + """OSW declares `format: autocomplete` on link properties for its UI.""" + from pydantic import Field + + class M(LinkedBaseModel): + model_config = ConfigDict(json_schema_extra={"$id": "https://example.org/M"}) + id: str | None = None + ref: Org | None = Field(None, json_schema_extra={"range": "Org", "format": "autocomplete"}) + + M.model_rebuild() + assert M.export_schema()["properties"]["ref"]["format"] == "autocomplete" + + +def test_the_frame_derivation_still_sees_our_links_as_references(): + """The schema side and the framing side must agree: both say a link is an + IRI. reference_properties is the canonical predicate (OOLD-EXT-68fa).""" + from oold.validation.frame import reference_properties + + assert reference_properties(Person.export_schema()) == ["employer", "mentor", "friends"] diff --git a/tests/test_validation/test_schema_levels.py b/tests/test_validation/test_schema_levels.py new file mode 100644 index 0000000..1f5922f --- /dev/null +++ b/tests/test_validation/test_schema_levels.py @@ -0,0 +1,171 @@ +"""A subclass emits its own schema level, composed onto its bases with allOf. + +The schema hierarchy mirrors the class hierarchy: `Person.json` carries what +`Person` adds and refers to `Entity.json` for the rest. Three rc.3 rules turn +this from a style preference into a requirement - every `$ref` is reflected in +`@context` (OOLD-CMP-b926), multiple `$ref`s list their contexts in allOf order +(OOLD-CMP-e4a3), and composition is narrow-only (OOLD-CMP-f3c7), which a +derived schema restating an inherited property can silently break. +""" + +import json + +import pytest +from pydantic import ConfigDict + +from oold.model import LINK_NOTATIONS_ACTIVE, Link, LinkedBaseModel, OoldField +from oold.static import SchemaExportMode +from oold.validation import failure_reasons, validate_schema + +pytestmark = pytest.mark.skipif( + not LINK_NOTATIONS_ACTIVE, + reason="the legacy binding is the deprecated opt-out and does not publish", +) + +# relative, as the committed fixtures are (tests/data/oold/Contact.schema.json): +# $id, the allOf $ref and the @context entry are the same reference, so a +# schema published beside its base resolves without a network fetch +ENTITY = "Entity.schema.json" +PERSON = "Person.schema.json" + + +class Entity(LinkedBaseModel): + model_config = ConfigDict( + json_schema_extra={ + "@context": { + "schema": "https://schema.org/", + "id": "@id", + "type": "@type", + "name": "schema:name", + }, + "$id": ENTITY, + } + ) + # optional, as the committed Thing.schema.json fixture has it: an instance + # whose only content is its @id compacts back to a bare IRI, so a base whose + # sole required property is the identifier cannot round-trip + id: str | None = None + name: str | None = None + + @classmethod + def get_cls_iri(cls): + return ENTITY + + +class Person(Entity): + model_config = ConfigDict( + json_schema_extra={ + "@context": [ + ENTITY, + {"father": {"@id": "schema:parent", "@type": "@id"}}, + ], + "$id": PERSON, + } + ) + father: Link["Person | None"] = OoldField() + + @classmethod + def get_cls_iri(cls): + return PERSON + + +Person.model_rebuild() + + +def _partial(model_cls) -> dict: + return model_cls.export_schema(mode=SchemaExportMode.PARTIAL) + + +def test_a_subclass_emits_only_its_own_properties(): + """`model_fields` is flattened by pydantic - it holds inherited fields too - + so building the level from it produced a monolithic schema. Restating an + inherited property also risks relaxing it, which OOLD-CMP-f3c7 forbids.""" + schema = _partial(Person) + assert sorted(schema["properties"]) == ["father"] + assert "id" not in schema.get("required", []) + + +def test_the_level_composes_onto_its_base_with_allof(): + schema = _partial(Person) + assert schema["allOf"] == [{"$ref": ENTITY}] + + +def test_the_context_mirrors_allof_in_the_same_order(): + """OOLD-CMP-b926 and OOLD-CMP-e4a3: a schema must be usable as a JSON-LD + context with no further processing, so every $ref is reflected in @context, + and multiple refs appear in allOf order.""" + schema = _partial(Person) + refs = [entry["$ref"] for entry in schema["allOf"]] + context = schema["@context"] + assert isinstance(context, list) + assert [c for c in context if isinstance(c, str)] == refs + + +def test_a_root_class_is_unchanged(): + """Nothing to compose onto, so no allOf and every property is its own.""" + schema = _partial(Entity) + assert "allOf" not in schema + assert sorted(schema["properties"]) == ["id", "name"] + + +def test_full_mode_still_flattens(): + """FULL stays the default and keeps its meaning: one self-contained schema.""" + schema = Entity.export_schema() + assert sorted(schema["properties"]) == ["id", "name"] + assert sorted(Person.export_schema()["properties"]) == ["father", "id", "name"] + + +def test_both_levels_validate(tmp_path): + """Written side by side, so the @context reference resolves locally - + validate_schema reads sibling schemas for exactly this reason.""" + for name, schema in (("Entity.schema", Entity.export_schema()), ("Person.schema", _partial(Person))): + schema.setdefault("$schema", "https://oo-ld.org/latest/meta/oold-meta-schema.json") + (tmp_path / f"{name}.json").write_text(json.dumps(schema, indent=2), encoding="utf-8") + report = validate_schema(tmp_path / "Person.schema.json") + assert report.passed, failure_reasons(report) + + +def test_a_definition_reached_only_through_another_is_kept(): + """Reachability is transitive: `$defs.Outer` is referenced from the body and + itself references `$defs.Inner`, so neither is dead. Pruning only what the + body points at directly would delete Inner and break the schema.""" + from oold.static import _prune_unreferenced_defs + + schema = { + "properties": {"a": {"$ref": "#/$defs/Outer"}}, + "$defs": { + "Outer": {"properties": {"b": {"$ref": "#/$defs/Inner"}}}, + "Inner": {"type": "string"}, + "Orphan": {"type": "string"}, + }, + } + _prune_unreferenced_defs(schema) + assert sorted(schema["$defs"]) == ["Inner", "Outer"] + + +def test_defs_disappears_when_nothing_survives(): + from oold.static import _prune_unreferenced_defs + + schema = {"properties": {}, "$defs": {"Orphan": {"type": "string"}}} + _prune_unreferenced_defs(schema) + assert "$defs" not in schema + + +def test_the_v1_underscore_spelling_is_normalised_too(): + """pydantic v1 cannot pass a hyphenated keyword to Field(), so downstream + spells it with underscores. Both reach `required` and neither survives.""" + from oold.static import _state_requiredness_in_the_required_array + + schema = { + "properties": { + "a": {"type": "string", "x_oold_required_iri": True, "default": None}, + "b": {"type": "string", "x-oold-required-iri": True}, + "c": {"type": "string", "default": None}, + } + } + _state_requiredness_in_the_required_array(schema) + assert sorted(schema["required"]) == ["a", "b"] + assert "default" not in schema["properties"]["a"] + assert not any(k.startswith(("x-oold-required", "x_oold_required")) for k in schema["properties"]["a"]) + # an optional property keeps its default and stays out of `required` + assert schema["properties"]["c"]["default"] is None