Skip to content

ATLAS-5322: Fix glossary bulk import relation parsing and error reporting. - #712

Open
UmeshPatil-1 wants to merge 1 commit into
apache:masterfrom
UmeshPatil-1:ATLAS-5322
Open

ATLAS-5322: Fix glossary bulk import relation parsing and error reporting.#712
UmeshPatil-1 wants to merge 1 commit into
apache:masterfrom
UmeshPatil-1:ATLAS-5322

Conversation

@UmeshPatil-1

@UmeshPatil-1 UmeshPatil-1 commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Problem

  • Relation columns in import CSV support multiple formats (GlossaryName:TermName, TermName@GlossaryName, and same-glossary TermName shorthand), but parsing/lookup was inconsistent.
  • During bulk import, terms created in pass 1 were not always resolvable during pass 2 relation linking because GUID cache was incomplete.
  • A term could be reported as successful in pass 1 and failed in pass 2 without reconciling the final BulkImportResponse.
  • UI showed only the first failure remark instead of a meaningful summary and per-term error lines.

Backend changes (repository)

GlossaryTermUtils.java

  • Added resolveRelatedTermReference() to normalize and validate three relation formats:
    • TermName@GlossaryName (qualified name)
    • GlossaryName:TermName (legacy colon format — backward compatible)
    • TermName (same-glossary shorthand — best-effort, not a hard failure if unresolved)
  • Added lookupGlossaryTermGuid() to resolve related terms from graph or in-import GUID cache.
  • Added cacheImportedTermGuid() so terms created/seen during import are available for relation linking in pass 2.
  • Improved relation error messages for explicit invalid references vs unresolved same-glossary shorthand.
  • Skip blank pipe-separated relation values.
    GlossaryService.java
  • Cache imported term GUIDs during term creation and when term already exists (re-import path).
  • Added reconcileBulkImportResponse() after relation pass:
    • Merge duplicate failure entries per term
    • Remove terms from successImportInfoList if they appear in failedImportInfoList (prevents misleading partial-success reporting)
      GlossaryServiceTest.java
  • Added regression test using glossary_healthcare_filled.csv — expects 20 successes, 0 failures.
  • Updated invalid-relation test expectation: term with failed relation is no longer reported as success after reconciliation.

UI changes

React dashboard (dashboard/)

  • Added glossaryImportUtils.ts with:
    • formatGlossaryImportFailure() → e.g. Patient@Healthcare Glossary: Reference not found
    • buildGlossaryImportFailureSummary() → e.g. Glossary import completed with 1 failure(s) out of 2 term(s). See error details.
  • Updated ImportDialog.tsx and AddUpdateGlossaryForm.tsx to use formatted toast + Error Details list.
  • Added unit tests: glossaryImportUtils.test.ts
    Legacy dashboard (dashboardv2/)
  • Updated ImportLayoutView.js to show summary notification and formatted per-term errors in Error Details modal (always shown when failures exist).

How was this patch tested?

Unit tests

  • GlossaryServiceTest — existing bulk import tests pass with updated invalid-relation expectation
  • GlossaryServiceTest — new test with glossary_healthcare_filled.csv: 20 success, 0 failed
  • glossaryImportUtils.test.ts — failure formatting and summary message

Manual REST tests

Tested using glossary_healthcare_filled.csv (20 data rows + 1 header = 20 terms) and curl commands.

  1. Health check — GET /api/atlas/v2/glossary?limit=1
    Result: HTTP 200

  2. Import healthcare CSV — POST /api/atlas/v2/glossary/import
    Result: Success: 20, Failed: 0

  3. Verify term count — GET /api/atlas/v2/glossary/{guid}
    Result: Term count: 20

  4. Patient term seeAlso relation — GET /api/atlas/v2/glossary/term/{patientTermGuid}
    Result: seeAlso count: 1, linked to Provider

  5. Provider preferredTerms — GET /api/atlas/v2/glossary/term/{providerTermGuid}
    Result: 2 terms — Physician, Nurse

  6. Colon format backward compatibility — import CSV with GlossaryName:TermName references
    Result: Success: 2, Failed: 0

  7. Negative test — invalid explicit @ reference — MissingTerm@BadRelGlossary
    Result: Entry in failedImportInfoList with clear error (not silent drop)

  8. Re-import same CSV (idempotency) — POST /api/atlas/v2/glossary/import
    Result: Success: 20, Failed: 0

  9. Automated verification script — scripts/atlas5322_verify.sh
    Result: ALL CHECKS PASSED

Example import verification:
curl -s -u admin:admin -X POST
-F "file=@glossary_healthcare_filled.csv"
"http://localhost:21000/api/atlas/v2/glossary/import"

Expected: successImportInfoList size = 20, failedImportInfoList empty.

Example relation verification:
GET /api/atlas/v2/glossary/term/{patientTermGuid}

Expected: "seeAlso" contains Provider; Provider has 2 preferredTerms.

UI manual test

  • Glossary import via React UI — on partial failure, toast shows summary count message
  • Error Details modal lists formatted lines: TermName@GlossaryName: remark
  • Legacy dashboard import view shows same improved error summary and details

@UmeshPatil-1 UmeshPatil-1 changed the title ATLAS-5322: Fix glossary bulk import relation parsing and error repor… ATLAS-5322: Fix glossary bulk import relation parsing and error reporting. Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant