Blog post: CIEL quality assurance — what's been fixed and what's now routine

[paynejd]

Summary

WHO feedback (received ~2 months ago) cited CIEL's QA history as a reason they have not adopted CIEL and will not recommend it to others — specifically, quality issues identified roughly five years ago that have not been forgotten. Separately, WHO also raised CIEL's governance process, but this ticket is scoped to QA only. Governance is a real, related need but is deliberately a separate future post — do not fold it into this one, even if it would be powerful to eventually pair them.

Goal

Publish a blog post that makes CIEL's quality-assurance work visible and credible to a skeptical technical/institutional reader: what was fixed (one-time remediation) and what QA is now — routine, ideally automated, ongoing process. The details of the specific quality issues WHO flagged five years ago aren't known yet; drafting will need to surface what they were (if discoverable) alongside what's changed since.

Scope

• QA only. No governance/transparency content in this post.
• Cover both: (a) one-time fixes/remediation performed, and (b) what is now a standing, repeatable QA process — with measurable specifics (rule counts, coverage, cadence, etc.) rather than generic assurances.

Known QA infrastructure to draw from (starting points, not exhaustive — confirm current state while drafting)

• CIEL Lab v2 QA rules engine (Drools-based): Document CIEL Lab QA Rules architecture against Drools #2752 (architecture documentation), CIEL Lab: Add QA rule to allow only one active description per locale #2741, QA Rule: FR-07 improvement > Validate not just name_type, but also mapping map types and sources against known vocabularies in QA #2706, Add QA rule for primary or secondary ICD-10 mappings with SAME-AS #2659, Add QA Rules configuration page for Bulk Validation and Concept Editor #2742, Add context filters to QA Rule execution configuration #2743
• OpenMRS validation script + release workflow: Document OpenMRS validation script checks and release workflow integration #2326, QA - Bulk operation: run deterministic OpenMRS validation rules on current CIEL HEAD via OCL API using CIEL Lab #2414
• Historical: [CIEL Source Mgmt QA Workflows] milestone (includeMappings & includeInverseMappings does not work when requesting list of concepts from a repo #53, closed) — early QA workflow work

Assignment

• Draft: Filipe Lopes (@filiperochalopes)
• Edit / review: Andy Kanter (@askanter)

Acceptance criteria

• Post states concretely what QA process exists today (rules, automation, review cadence), with measurable claims — not vague "we take quality seriously" language
• Post distinguishes one-time historical fixes from now-routine/automated process
• Reviewed and approved by Andy Kanter before publish
• Stays in scope: no governance content (flag as input for a follow-up governance post instead)

[askanter]

Since we don't have specific complaints about the QA, it is hard to know or respond to historical concerns. However, I think we should be highlighting (from the content side):

1. Reviewed maps to ICD-10. Codes updated and relationship type being updated.
2. Reviewed maps to SNOMED CT and updating from any retired codes.
3. Standardized casing
4. Added translations for Spanish, French and Russian
5. Ongoing merging of duplicate concepts
6. Filled gaps in RxNORM and WHO ATC mappings

Others, Filipe?

[ocl-automation]

Automated Triage — 2026-09-10

Dimension	Score
Complexity	3
Scope	2
Clarity	3
Dependency	1
Risk	3

Decision: decision/human-review
Type: type/docs
Signals: signal/small-scope

Writing a substantive blog post about QA improvements requires moderate complexity to research and present technical content clearly. Scope is limited to documentation files. Clarity is moderate - the goal and audience are clear but specific content structure and messaging details aren't fully specified. Moderate risk because public-facing content about past quality issues could impact project reputation if handled poorly.

---

Generated by ticket-triage pipeline v3. Model: anthropic/claude-sonnet-4 (openrouter).

[filiperochalopes]

We reinforced name types, map types, and sources accordingly to CIEL value sets/facets for them. Our newest bulk edit through the MCP widget for casing also alerts for potential typos so the curator can fix them (another indirect QA step).

Today we cover more than 30 QA rules, including syntactic and semantic rules.

Reviewed maps to SNOMED CT and updating from any retired codes.

Not just SNOMED I would say, but also ICD-11 update, RxNORM (removed retired mappings)

Refs: https://filiperochalopes.github.io/ciellab-wiki/features/quality-assurance/overview/; https://ciellab.filipelopes.med.br/qa/settings

[filiperochalopes]

QA history and current state

Before: the only QA was a syntactic validator. Substantive data-quality issues surfaced solely through user bug reports or curators running ad hoc SQL against hunches.

Now: a living catalog of 32 rules (FR-01…FR-32) evaluated in two tiers — local (decidable from the payload alone, gates Save in the Concept Editor) and deferred (needs the HEAD/Qdrant/OCL corpus, runs in Bulk Validation). Every rule originated from a real incident: a bug was reported or found, fixed once, and locked in as a permanent regression check — and chasing one issue routinely surfaced siblings, which is how the catalog grew to its current size. This list is reviewed weekly and is the team's starting point for where to look next.

#	Rule	Catches	Severity	Scope
FR-01	Names	Multiple Preferred names, same locale	error	local
FR-02	Names	Duplicate active non-SHORT names, same locale	error	local
FR-03	Names	Multiple SHORT names, same locale	error	local
FR-04	Names	SHORT name marked Preferred	error	local
FR-05	Names	Multiple Fully Specified Names, same locale	error	local
FR-06	Names	Missing an active Fully Specified Name	error	local (bulk only)
FR-07	Names	Name type not one of the 4 canonical OpenMRS types, verbatim	error	local (bulk only)
FR-08	Names/Desc	Locale not supported by the source version	error	local (bulk only)
FR-09	Names	FSN collides with another active concept	error	cross-concept, deferred
FR-10	Names	Preferred synonym duplicated on another active concept	error	cross-concept, deferred
FR-11	Metadata	Missing/invalid 36-char OpenMRS UUID	error	local (bulk only)
FR-12	Mappings	Duplicate mapping (same source+code+map type)	error	local
FR-13	Mappings	CIEL concept missing required SAME-AS to its own code	error	local (bulk only)
FR-15	Names/Desc	HTML-encoded entities present	error	local
FR-18	Names	Bad/inconsistent capitalization (Preferred/FSN)	warning	local
FR-19	Names	Preferred name contains "and/or", "and", "or"	warning	local
FR-21	Metadata	Invalid/non-canonical numeric unit	warning	local
FR-22	Mappings	Mapped to a retired/inactive code in an indexed terminology	warning	external, deferred
FR-23	Mappings	Residual ICD code mapped SAME-AS	warning	local
FR-24	Mappings	Same source+code mapped with different map types	warning	local
FR-25	Mappings	Multiple SAME-AS mappings to one source	warning	local
FR-26	Mappings	Another active concept shares the same SAME-AS target (merge candidate)	warning	cross-concept, deferred
FR-27	Mappings	Primary + secondary ICD-10-WHO mapping, one is SAME-AS	warning	local
FR-28	Descriptions	Multiple active descriptions, same locale	error	local
FR-29	Mappings	Map type not a known canonical OCL map type (verbatim)	error	local
FR-30	Mappings	Mapping target source unknown to CIEL Lab	error	local
FR-31	Metadata	Concept class not a known OCL class	error	local
FR-32	Metadata	Datatype not a known OCL datatype	error	local