fix!: emit schemas that validate against OO-LD v1.0.0-rc.3

docs/design/graph-object-binding.md

@@ -159,9 +159,18 @@ class Person(LinkedBaseModel):
 
 `__set__` is declared under `TYPE_CHECKING` only, so at runtime the descriptor
 stays **non-data** and the instance-`__dict__` cache from 3.1 is untouched.
-`__get_pydantic_core_schema__` builds the schema of the *target*, so the emitted
-JSON Schema is byte-identical to the plain annotation - `$ref`, arrays, unions
-and forward references included.
+`__get_pydantic_core_schema__` builds the schema of the *target*, so what
+pydantic sees is identical to the plain annotation - arrays, unions and forward
+references included, and both spellings therefore emit the same document.
+
+What that document says about a link is a separate question. A link serialises
+to an IRI, so the published property is `{"type": "string", "x-oold-range": …}`,
+or an array of strings for a to-many link - which is what a real OSW schema
+carries. The `$ref` / `allOf` form is what `generator.preprocess` builds so that
+`datamodel-code-generator` emits `Optional[Bar]` instead of a string field; it
+is a code generation shape, and publishing it described a document the library
+never writes. A union arm keeps its union: `str | Location | None` genuinely
+accepts a literal, a reference or an inline object.
 
 #### Optionality is declared, not assumed
 
@@ -325,13 +334,20 @@ read it?" - so two carriers:
 | declaration | stored as | enforced | on violation |
 |---|---|---|---|
 | `Link[T]` - no `None` arm | `_AutoLink.optional = False` | on read | `LinkNotResolved` |
-| `OoldField(required=True)` | `_AutoLink.required_iri`, precomputed into `cls.__required_links__`; emitted as `x-oold-required-iri` and into the standard `required` array | in `__init__` | `ValueError: ... is required but not set` |
+| `OoldField(required=True)` | `_AutoLink.required_iri`, precomputed into `cls.__required_links__`; reaches the schema as the standard `required` array | in `__init__` | `ValueError: ... is required but not set` |
 
 A link is never required at the *pydantic* level, because its value is routed
 out of the payload before validation - which is why the legacy binding declared
 every generated link field `Optional[...]` and carried requiredness in the
-keyword. A bare `Link[T]` annotation with no default is read as required, which
-is what Python means by "no default" everywhere else.
+keyword. `x-oold-required-iri` is that keyword, and it stays internal: a schema
+states requiredness through `required` alone, and the annotation exists only to
+carry the requirement across the point where code generation drops the property
+from `required` so the emitted field is `Optional`.
+
+A bare `Link[T]` annotation with no default is **optional**. "No default means
+required" reads well in plain Python, but requiredness propagates into
+resolution - the failure mode described next - and links are declared far more
+often than they are required, so the terse form is the common case.
 
 **Why not put requiredness in the annotation.** `required` -> `Link[T]`,
 absence -> `Link[T | None]` reads well and was the first proposal. It fails on a

docs/how-to/object-graph-mapping.md

@@ -228,6 +228,26 @@ alice.knows[0]        # a Person, not a str
 Nothing changes at runtime - same resolution, same JSON Schema. Only what the
 checker sees changes.
 
+### What a link looks like in the emitted schema
+
+A link serialises to an IRI, and the schema says so:
+
+```json
+"employer": {"type": "string", "x-oold-range": "https://example.org/Organization"},
+"friends":  {"type": "array", "items": {"type": "string"},
+             "x-oold-range": "https://example.org/Person"}
+```
+
+Not a `$ref` to the target. The `$ref` form is what code generation builds so
+that the generated field is `Optional[Organization]` rather than a string;
+published, it would describe a document this library never writes. A union arm
+keeps its union - `str | Location | None` really does accept all three.
+
+Give every link a `@context` term, and `"@type": "@id"` to make the value a
+reference. A strictly array-typed property also needs `"@container": "@set"`,
+or a single-element array compacts back to a bare value and no longer matches
+the schema.
+
 ### Optionality is declared
 
 `Link[T]` reads as `T`, so a chain needs no guard at every hop. `Link[T | None]`
@@ -268,21 +288,35 @@ OoldField(required=None, range=None, link=None, **field_kwargs)
 
 | argument | effect |
 |---|---|
-| `required` | the link must be supplied at construction; omitting it raises `ValueError`. Emitted as `x-oold-required-iri` **and** into the standard `required` array |
+| `required` | the link must be supplied at construction; omitting it raises `ValueError`. Reaches the schema as the standard `required` array, and nothing else |
 | `range` | target schema IRI, emitted as `x-oold-range`. **Do not pass it**: omitted, it is derived from the annotation, which already names the target |
 | `link` | marks the field a link where the annotation does not imply it, as in a union arm. Redundant with `Link[T]` / `LinkList[T]` |
-| `required_iri` | deprecated spelling of `required`, kept because generated packages pass it. Same emitted keyword |
+| `required_iri` | deprecated spelling of `required`, kept because generated packages pass it |
 | `**field_kwargs` | passed to `pydantic.Field` (`alias`, `description`, `default_factory`, ...). `default=None` is supplied unless you pass a `default_factory` |
 
-A link annotation with **no default at all** means required, as it does anywhere
-else in Python:
+Requiredness is always explicit. A link annotation with no default is optional,
+like one with a bare `OoldField()`:
 
 ```python
-manager: Link[Organization]                          # required
-manager: Link[Organization] = OoldField()            # optional
-manager: Link[Organization] = OoldField(required=True)   # required, explicit
+father: Link["Person"]                            # optional
+father: Link["Person"] = OoldField()              # optional
+father: Link["Person"] = OoldField(required=True) # required, explicit
 ```
 
+"No default means required" reads well in plain Python, but requiredness
+propagates into resolution - reading a link constructs the target - so a
+required link makes every stored document lacking it unconstructible. A
+self-referential link like `father` could then never be satisfied by a real
+dataset, and links are declared far more often than they are required.
+
+!!! note "`x-oold-required-iri` is internal"
+    A schema states requiredness through `required` and nothing else.
+    `x-oold-required-iri` is an oold-python annotation on a *field*: it carries
+    the requirement across the point where code generation has to drop the
+    property from `required`, so the generated field is `Optional` - a link can
+    never be required at the pydantic level, because its value is routed out of
+    the payload before validation. It does not appear in a published schema.
+
 ### Requiredness is a field argument, not the annotation
 
 "Must the caller supply it?" and "what do I get when I read it?" are different

src/oold/model/_compat.py

@@ -158,6 +158,15 @@ def get_cls_iri(cls) -> Any:
         ``GenericLinkedBaseModel`` only declares this abstract, so without an
         implementation it silently returns ``None`` - which would break the
         downstream callers and the type registry alike.
+
+        This answers **identity**: it merges the ``$id`` with the ``type``
+        field's default(s), which are the instances' rdf:type, so the registry
+        can resolve a document's ``type`` back to a class. It is not a
+        **location** and must not be used as one - a consumer dereferences a
+        location, and the two only coincide sometimes.
+        ``wiki_data.Person`` answers ``["http://www.wikidata.org/entity/Q5",
+        "Item:Q5"]`` and publishes no schema at either. Emitting
+        ``x-oold-range`` uses ``$id`` alone; see ``_AutoLink.range_iri``.
         """
         schema = getattr(cls, "model_config", {}).get("json_schema_extra") or {}
         if callable(schema):

src/oold/model/_descriptor.py

@@ -189,12 +189,20 @@ def _neutralised(info: Any) -> Any:
             namespace[field_name] = _neutralised(info)
             continue
         if field_name not in namespace and _is_link_annotation(annotation):
-            # `manager: Link[Org]` with nothing assigned. Read as Python reads
-            # it - no default means required - but a link cannot be required as
-            # a pydantic field, because its value never reaches validation.
-            # Left alone, every construction failed with a misleading
-            # "Field required" about a value that had in fact been supplied.
-            namespace[field_name] = OoldField(required=True)
+            # `manager: Link[Org]` with nothing assigned. It still needs a field
+            # carrying default=None - a link cannot be required at the pydantic
+            # level, because its value never reaches validation, and left alone
+            # every construction failed with a misleading "Field required" about
+            # a value that had in fact been supplied.
+            #
+            # It is *not* required in the OO-LD sense: that is what
+            # OoldField(required=True) says. Requiredness propagates into
+            # resolution - resolving a link constructs the target - so a
+            # required link makes every stored document lacking it
+            # unconstructible, and a self-referential link could never be
+            # satisfied by a real dataset. Links are declared far more often
+            # than they are required, so the terse form is the common case.
+            namespace[field_name] = OoldField()
             continue
         # A Field() living in Annotated metadata rather than as the assigned
         # value was never seen here, so its default survived and was evaluated
@@ -221,6 +229,59 @@ def _neutralised(info: Any) -> Any:
     return defaults
 
 
+def _permits_a_string(prop: dict) -> bool:
+    """Whether the property already allows a bare IRI."""
+    if prop.get("type") == "string":
+        return True
+    for arms in ("anyOf", "oneOf"):
+        for arm in prop.get(arms) or []:
+            if isinstance(arm, dict) and arm.get("type") == "string":
+                return True
+    return False
+
+
+def _as_reference(prop: dict, many: bool) -> None:
+    """Rewrite a link property to the shape its instances actually take.
+
+    A link serialises to an IRI - ``{"employer": "ex:acme"}`` - and a published
+    OO-LD schema says so: ``{"type": "string", "x-oold-range": …}``. The
+    ``$ref`` / ``allOf`` form is a *code generation* shape, produced by
+    ``generator.preprocess`` so that datamodel-code-generator emits
+    ``Optional[Bar]`` rather than a string field. Emitting it describes a
+    document the library never writes, and the JSON-LD round trip fails on it:
+    with ``"@type": "@id"`` on the term, an embedded object loses its
+    properties.
+
+    Union arms are left alone. ``str | Location | None`` genuinely accepts a
+    literal, a reference or an inline object, and its schema already says so.
+    """
+    if _permits_a_string(prop):
+        return
+    # "range" is the legacy spelling of x-oold-range and is still what generated
+    # packages declare; dropping it here would silently delete the only thing
+    # marking the property a link in those schemas.
+    keep = ("title", "description", "default", "range", "format")
+    kept = {k: v for k, v in prop.items() if k in keep or k.startswith("x-")}
+    prop.clear()
+    prop.update(kept)
+    # OOLD-EXT-6ea3 (SHOULD): an IRI-valued property constrains its lexical form
+    # with an IRI-family format, and OOLD-EXT-1f92 recommends iri-reference - it
+    # admits absolute IRIs, compact IRIs and context-relative references alike,
+    # which is what instances actually carry. It is also the second of the three
+    # reference signals a frame derivation looks for (OOLD-EXT-68fa, see
+    # `oold.validation.frame.reference_properties`). A format the declaration
+    # already states is left alone.
+    reference: dict[str, Any] = {"type": "string"}
+    if "format" not in kept:
+        reference["format"] = "iri-reference"
+    if many:
+        prop.pop("format", None)
+        prop["type"] = "array"
+        prop["items"] = reference
+    else:
+        prop.update(reference)
+
+
 def _namespace_annotations(namespace: dict) -> dict:
     """The annotations of a class body being built, on any Python version.
 
@@ -943,21 +1004,29 @@ def _target_cls(self, owner: Any) -> Any:
         return target
 
     def range_iri(self, owner: Any = None) -> Any:
-        """The target's schema IRI, for deriving ``x-oold-range``.
+        """Where the target's schema lives, for deriving ``x-oold-range``.
 
         ``Link[T]`` already names the target, so repeating it in
         ``OoldField(range=...)`` states the same thing twice and lets the two
         disagree. The schema is derived from the annotation instead.
+
+        A **location**, not an identity. A consumer dereferences this: code
+        generation fetches the target schema, and a form editor renders the
+        targets it allows. ``get_cls_iri()`` answers identity - it merges the
+        ``$id`` with the ``type`` field's default(s), which are the instances'
+        rdf:type - so deriving the range from it published identities with
+        nothing to fetch at them, such as a Wikidata class IRI. Where the two
+        coincide nothing changes; where they differ, only ``$id`` is right.
+
+        A class that does not say where its schema lives contributes no range.
+        The property is still marked a reference by its ``format`` - the second
+        signal in ``OOLD-EXT-68fa``.
         """
         target = self._target_cls(owner)
-        get_iri = getattr(target, "get_cls_iri", None)
-        if get_iri is None:
-            return None
-        try:
-            return get_iri() or None
-        except Exception:
-            # a target that cannot name itself simply contributes no range
+        extra = getattr(target, "model_config", {}).get("json_schema_extra") if target is not None else None
+        if callable(extra) or not isinstance(extra, dict):
             return None
+        return extra.get("$id") or None
 
     def __get__(self, obj: Any, objtype: Any = None) -> Any:
         if obj is None:
@@ -1252,14 +1321,22 @@ def __get_pydantic_json_schema__(cls, core_schema_: Any, handler: Any) -> Any:
         not resolvable at class-creation time, and a ``Field()`` object shared
         between models must not be mutated in place.
         """
-        schema = handler(core_schema_)
+        document = handler(core_schema_)
         try:
-            schema = handler.resolve_ref_schema(schema)
+            schema = handler.resolve_ref_schema(document)
         except Exception:
-            return schema
+            return document
+        # A self-referential model comes back as {"$defs": …, "$ref": "#/$defs/X"},
+        # which leaves the *document* without the $id the OO-LD document tier
+        # requires - the $id sits on the definition instead. Carry it up, so the
+        # emitted document identifies itself either way.
+        if isinstance(document, dict) and isinstance(schema, dict) and document is not schema:
+            identifier = schema.get("$id")
+            if identifier and "$id" not in document:
+                document["$id"] = identifier
         properties = schema.get("properties") if isinstance(schema, dict) else None
         if not properties:
-            return schema
+            return document
         link_fields = cls.__link_fields__
         aliases = cls.__link_aliases__
         required = schema.get("required")
@@ -1269,23 +1346,32 @@ def __get_pydantic_json_schema__(cls, core_schema_: Any, handler: Any) -> Any:
             if descr is None or not isinstance(prop, dict):
                 continue
             if descr.required_iri:
-                # A link is never required at the pydantic level - its value is
-                # routed out of the payload before validation - so pydantic
-                # leaves it out of `required`. Stating it only in
-                # x-oold-required-iri would hide the constraint from every
-                # plain JSON Schema validator.
+                # A schema states requiredness through the standard `required`
+                # array and nothing else. A link is never required at the
+                # pydantic level - its value is routed out of the payload before
+                # validation - so pydantic leaves it out and it is added here.
                 if required is None:
                     required = schema["required"] = []
                 if key not in required:
                     required.append(key)
-            if prop.get("x-oold-range") or prop.get("range"):
-                continue
-            iri = descr.range_iri(cls)
-            if iri:
-                prop["x-oold-range"] = iri
-                # the range says "link" on its own; the marker was a stand-in
-                prop.pop("x-oold-link", None)
-        return schema
+                # ... and a required property may not also declare a default:
+                # nothing satisfies both, and a consumer generating an instance
+                # from this schema produces one its own schema rejects.
+                prop.pop("default", None)
+            # x-oold-required-iri and x-oold-link are oold-python annotations on
+            # a *field*. The first carries requiredness across the point where
+            # the property has to leave `required` so the generated field is
+            # Optional (see generator.preprocess); the second marks a link whose
+            # target comes from the annotation. Neither is in the OO-LD keyword
+            # vocabulary, so neither belongs in a published document.
+            prop.pop("x-oold-required-iri", None)
+            if not (prop.get("x-oold-range") or prop.get("range")):
+                iri = descr.range_iri(cls)
+                if iri:
+                    prop["x-oold-range"] = iri
+            prop.pop("x-oold-link", None)
+            _as_reference(prop, descr.many)
+        return document
 
     @classmethod
     def __pydantic_init_subclass__(cls, **kwargs: Any) -> None: