Skip to content

Citation scheme/style models + CSL mapping + flavour parity + JSON Schema + atlas inventory - #76

Open
ronaldtse wants to merge 12 commits into
mainfrom
flavour-parity-and-schema
Open

ronaldtse wants to merge 12 commits into
mainfrom
flavour-parity-and-schema

Conversation

@ronaldtse

@ronaldtse ronaldtse commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Summary

All TODO.cc-citation items completable in this repository are done.

What this PR delivers

Citation scheme/style/index/style as MODELS (TODO 07, flagship)

  • citation/ module: 14 LML models — CitationScheme, CitationStyle, BibliographicIndex, BibliographicStyle + supporting models
  • First style instance: citation/styles/iso-690.yml validated by generated schema

CSL subset mapping + importer (TODO 08)

  • mapping/csl.yaml: 35 type mappings, 57 variable mappings, 8 documented downgrade losses
  • tools/validate_csl.py: gates the mapping against the generated schema
  • tools/csl_to_style.rb: CSL YAML to CitationStyle importer
  • CSL is explicitly a lower-level subset of ISO 690/Relaton

ISO 690 coverage completion (TODO 03)

  • Edition audience, printer + sponsor production roles
  • ValidityType +provenance (chain of custody) and +authenticity (certified/copy/unknown)

Vocabulary parity + schema generation (TODO 02, 10)

  • rake parity gates 19 vocabularies (5 base + 14 flavour DocumentType overlays)
  • rake schema generates relaton/schema/bibitem-2020-12.json (380 definitions)
  • Schema freshness asserted in CI

LML to RNC generation, stage B (TODO 09)

  • tools/generate_rnc.py emits RNC vocab definitions from LML; rake rnc:check validates generated == committed
  • Stage C (full element structure + canonical flip) remains open

Flavour profiles, all 29 (TODO 14)

  • profiles/*.yaml with narrowing-only validation (rake profiles)
  • 7 additional profiles added in final batch (cen, cie, ecma, iana, mpfa, nist, omg)

Parser-based validation (TODO 13)

  • tools/validate_yaml.rb uses Lutaml::Lml::Pipeline.call

Versioning policy (TODO 15)

  • docs/VERSIONING.md: model semver, deprecation cycle, standardization snapshots

ISO 24229 spelling systems (TODO 16)

  • Iso24229Code + LocalizedString.spellingSystem

MODS gap analysis (TODO 11)

  • docs/MODS-GAP-ANALYSIS.md: full mapping table and gap findings

Atlas inventory (TODO 06)

  • inventory.html + inventory.json at the site root

Remaining TODO items (other repos)

  • 04: citation-doc chapter parity (CalConnect/cc-citation-models)
  • 05: element parity gate (CalConnect/cc-citation-models)
  • 07 engine: relaton-render v2 (metanorma/relaton-render)
  • 09 stage C: LML to RNC element-structure generation + canonical flip
  • 12: relaton.org retirement (relaton/relaton.org)

Gate results (all green)

  • 66 PNGs verified
  • lint: 381 types / 32 modules
  • parity: 29 flavour overlays, 19 vocabularies
  • profiles: 29 OK
  • fixtures: XML, YAML, schema, CSL all OK
  • rnc:check: 4 vocabularies OK
  • CI: ubuntu green

Test plan

  • bundle exec rake check fixtures profiles rnc:check — all green
  • CI green on all checks

- Vocabulary parity extended to 14 flavour DocumentType overlays
  (kebab-fold comparison tolerating wire/LML spelling differences incl.
  spaces, underscores and accents). Drift caught and fixed: bsi/iso
  camelCase, ieee missing five wire values, gb out-of-wire values, itu
  missing three (handbook/question/contribution), BibItemType missing
  four (electronic-resource, notated-music, performed-music,
  personal-communication), etsi accent loss in Norme Europeenne.
- TODO 10 core: tools/generate_schema.py emits
  relaton/schema/bibitem-2020-12.json (363 definitions - classes with
  attribute cardinality and LML inheritance, enums, open marker classes
  for polymorphism); tools/validate_schema.py validates the YAML fixture
  against it; rake schema + fixtures:schema wired into both CI
  workflows, CI asserts the committed schema is fresh
- TODO statuses updated (02 done incl. flavours, 09 stage A partially
  realized, 10 core done)
* main:
  Advance basicdoc to the ontology-completion tree
TODO.cc-citation items 06-08:

07 (models + first style):
- citation/ module: 14 LML models - CitationScheme (system +
  disambiguation + name form + localized strings), CitationStyle (scheme +
  templates + per-type template map + sort rules), BibliographicIndex
  (entries + collation + grouping), BibliographicStyle (index + layout);
  CitationSystem enum (ISO 690 Annex A: name-date, numeric, named-tag,
  running-notes, implied); four views rendered; parity + lint green
- citation/styles/iso-690.yml: the ISO 690 name-and-date scheme as DATA,
  validated against the schema generated from the LML (proves the loop:
  model -> schema -> instance -> gate)

08 (mapping + gate):
- mapping/csl.yaml: 35 CSL type -> BibItemType, 57 variable -> attribute
  (with wire forms), 8 documented downgrade losses. CSL is a lower-level
  subset of ISO 690/Relaton - imports downgrade, never extend
- tools/validate_csl.py: every mapping target must exist in the generated
  schema; wired into rake csl + CI (gate: the mapping cannot drift from
  the models)

06 (complete):
- Atlas /inventory.html: every module's types with attributes/values,
  machine-generated from the LML, linked from primary navigation;
  inventory.json also emitted

The citation module is pure addition - the Rakefile, parity, lint, site,
schema, and inventory all discovered it without code changes (OCP proof).
@ronaldtse ronaldtse changed the title Flavour vocabulary parity gate + JSON Schema generation from the LML Citation scheme/style models + CSL mapping + flavour parity + JSON Schema + atlas inventory Aug 22, 2026
first flavour profile

Investigated basicdoc-models (now fully synchronized with the ISO 690
standard document) and adapted the applicable improvements:

- TODO.cc-citation items 13-16 added from the investigation:
  13 parser-based model extraction (lutaml-lml Pipeline.call, not regex)
  14 declarative flavour profiles (narrowing-only, from basicdoc profiles/)
  15 versioning and stability policy (from basicdoc docs/VERSIONING.md)
  16 ISO 24229 spelling systems on localized strings

- 13 done: tools/validate_yaml.rb now uses Lutaml::Lml::Pipeline.call
  for model extraction (classes, enums, attributes with cardinalities)
  instead of regex scanning - robust, type-safe, matches basicdoc's
  validator architecture

- 14 first profile done: profiles/iso.yaml (ISO document-type subset,
  narrowed cardinalities, excluded constructs). Narrowing-only doctrine
  documented; full rollout + parser-based enforcement gate is TODO open.

- 15 done: docs/VERSIONING.md - MAJOR.MINOR.PATCH tied to ISO 690
  milestones; deprecation cycle; standard/ snapshot tags; breaking-change
  audit = rake check; citation style compatibility statement.

- Gate results: lint 365 types / 31 modules, parity 29 overlays,
  fixtures all OK (YAML now parser-based).
* main:
  Adapt basicdoc-models improvements: parser-based validation, versioning, first flavour profile
  Advance basicdoc submodule (contribMetadata restored on blocks)
@ronaldtse
ronaldtse force-pushed the flavour-parity-and-schema branch from c450745 to d7b378b Compare August 26, 2026 07:07
Finish the remaining basicdoc-derived work on top of the citation model
pipeline:

- 14: profiles/*.yaml for 22 flavours, plus tools/validate_profiles.rb and
  rake profiles. Profiles encode narrowing-only flavour constraints;
  profile validation is wired into rake check. Generating the profiles
  exposed missing wire document types in multiple flavour LML enums; those
  enum values and regenerated diagrams are included here.
- 15: docs/VERSIONING.md with model semver, deprecation cycle,
  standardization snapshots, and breaking-change audit policy.
- 16: Iso24229Code + LocalizedString.spellingSystem, wired into the data
  types view and generated schema.
- 11: docs/MODS-GAP-ANALYSIS.md mapping MODS 3.x to Relaton and listing
  actual gaps (shelf locator, record-level metadata, multi-location
  holdings) plus Relaton advantages.
- 13: tools/validate_yaml.rb now extracts classes/enums/attributes with
  lutaml-lml Pipeline.call rather than regex scanning (parser-based where
  the LML parser can parse the file); this mirrors basicdoc-models.

All gates green: render/verify/lint/parity/profiles/fixtures/schema/csl.
RNC generator stage B, MODS gap

All 17 TODO.cc-citation items now done or have their completable portion
done. This commit closes:

- 03: ValidityType +provenance (chain of custody) and +authenticity
  (AuthenticityStatus enum: certified/copy/unknown); RNC updated with
  element definitions; AuthenticityStatus RNC definition added
- 08: tools/csl_to_style.rb — simplified CSL YAML to CitationStyle
  importer (maps types, variables, templates, sort rules using
  mapping/csl.yaml); example input/output validated
- 09: tools/generate_rnc.py — LML-to-RNC vocabulary generator
  (stage B); rake rnc:check validates generated == committed
  with kebab-fold normalization; wired into rake fixtures + CI
- 11: marked done (docs/MODS-GAP-ANALYSIS.md was already written)
- 14: 7 additional profiles (cen, cie, ecma, iana, mpfa, nist, omg)
  for 29 total — all flavours now have declarative profiles
- RNC alignment: spelling-system attribute on LocalizedString,
  edition audience attribute, printer + sponsor role types,
  AuthenticityStatus definition — all now reflected in the grammar

TODO.cc-citation final status:
  01 done | 02 done | 03 done | 04 open (cc repo) | 05 open (cc repo)
  06 done | 07 models+style done | 08 done | 09 stage B done
  10 core done | 11 done | 12 open (relaton.org) | 13 done
  14 done (29 profiles) | 15 done | 16 done

All gates: render/verify/lint/parity/profiles/fixtures/schema/csl/rnc:check
Ontology graph completion, BasicBlock.contribMetadata restoration,
parser-based fixture validation, declarative profiles and the versioning
policy. Gates re-run green: check (65 PNGs, 380 types / 32 modules,
29 overlays / 271 LML files), fixtures, csl, schema (379 definitions,
output unchanged).
…mport

- Atlas /modules/{name}.html: one detail page per module (31 pages, 78
  total). Each page shows every type with its kind, parent, attributes
  (name + type), and enum values — with anchor links from the inventory
  table and the card catalog. The primary navigation gains a Base model
  link.
- basicdoc advanced to 7a53d4e (atlas profiles page, twin parity restored,
  dir/checkbox fixtures)
- RNC alignment: spelling-system, edition audience, printer + sponsor
  roles, AuthenticityStatus all now in the grammar; fixtures regenerate
  green

Also outside this repo:
- relaton.org PR #100: direct import via git submodule + sync script
  (closes #59)
- Issues #67 and #18 closed; #9 and #35 updated with progress
- rake ci: single command for ALL gates (render, verify, lint, parity,
  profiles, fixtures, schema, CSL, RNC check) - what CI runs, what
  developers run
- CLAUDE.md: rewritten for the current architecture (citation module,
  profiles, CSL, RNC generator, construct doctrine, flavour profiles,
  versioning policy, consumers)
- README.adoc: architecture section added; build commands updated with
  rake ci, profiles, csl
- CitationStyle schema: relaton/schema/citation-style-2020-12.json
  emitted alongside the BibliographicItem schema (same $defs, rooted
  at CitationStyle for style instance validation)
- Atlas cross-references: 379 anchor links; attribute types in per-module
  pages now link to type definitions across modules (all_types map)
- Gemfile.lock refreshed after basicdoc advance
- CI schema freshness: directory-level diff covers both schemas
… LML tooling

- citation-style-2020-12.json: 156KB duplicate -> thin $ref wrapper into
  bibitem-2020-12.json#/$defs/CitationStyle
- site: per-module inventory JSON at _site/inventory/{module}.json
- tools/lml_model.rb: shared model discovery + parser-based type index
- Rakefile: validation tools run under bundle exec (moxl version isolation)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant