Skip to content

feat(dr_egeria_folder): subfolders, exclude and per-file run-as user; manual - #435

Merged
dwolfson merged 2 commits into
odpi:mainfrom
dwolfson:docs/dr-egeria-portal-auto-loading
Oct 8, 2026
Merged

dwolfson merged 2 commits into
odpi:mainfrom
dwolfson:docs/dr-egeria-portal-auto-loading

Conversation

@dwolfson

@dwolfson dwolfson commented Oct 8, 2026

Copy link
Copy Markdown
Member

Why

dr_egeria_folder promises the same _batch.json semantics as the Egeria Portal's folder loader (egeria-workspaces bootstrap_batches.py). The Portal now recurses into subfolders and sets a run-as user per file (odpi/egeria-workspaces#618), so the CLI needs the same rules to keep a folder running the same way from either.

It also fixes a silent regression. Object-form files entries ({"file": …, "userid": …}) never matched the old string comparison, so a folder using them fell back to alphabetical order.

What changes

commands/cat/dr_egeria_folder.py:

  • Subfolders run in place. A files entry can be a file, "sub/x.md" or "sub/". Unlisted files, then unlisted subfolders, follow alphabetically and are reported as ordering warnings; so are stale entries.
  • New exclude field. README.md and output/plumbing folders (dr-egeria-outbox, logs, data, …) are skipped. The outbox folders hold processed copies with live commands.
  • Run-as user. userid is set per folder (inherited by subfolders) or per entry; the nearest declaration wins. Passwords come from EGERIA_BOOTSTRAP_PASSWORD_<USERID>, else --user_pass. The command uses one client per user for the run. --no-manifest-userids runs everything as --userid.
  • A file reachable twice runs once.

docs/dr_egeria_manual.md: new sections on dr_egeria_folder, the _batch.json manifest, and the Portal's automatic loading; it also documents --user_pass.

Testing

  • Same results as the Portal: order and run-as users match the live Portal's /api/bootstrap/batches for every Coco batch. When the CLI runs the Data Hub folder on its own, it runs data-field-naming/ inline at the README's step 6, as intended; in the Portal that subfolder is its own batch.
  • Per-user sign-in works: dr_egeria_folder "coco-workbooks/6. data-privacy" --validate signed in as faithbroker, as its manifest says, and validated 106 commands with 0 failures.
  • Not run: a --process run.

… manual

dr_egeria_folder promises the same _batch.json semantics as the Egeria
Portal's folder loader (egeria-workspaces bootstrap_batches.py), which now
recurses into subfolders and sets a run-as user per file. Bring the CLI
back in line:

- Subfolders run in place: a "files" entry can be a file, "sub/x.md" or
  "sub/"; unlisted files, then unlisted subfolders, follow alphabetically
  and are reported as ordering warnings, as are stale entries.
- New "exclude" field; README.md and output/plumbing folders
  (dr-egeria-outbox, logs, data, ...) are skipped. The outbox folders
  hold processed copies with live commands.
- "userid" per folder (inherited) or per entry as {"file", "userid"};
  nearest declaration wins. Passwords come from
  EGERIA_BOOTSTRAP_PASSWORD_<USERID>, else --user_pass. One client per
  user for the run. --no-manifest-userids runs everything as --userid.
- A file reachable twice runs once.

Fixes a silent regression: object-form "files" entries never matched the
old string comparison, so such a folder fell back to alphabetical order.

Verified: identical order and users to the Portal's discovery for every
Coco batch, and a --validate run of coco-workbooks/6. data-privacy signed
in as faithbroker per its manifest (106 commands, 0 failures).

docs/dr_egeria_manual.md: new sections on dr_egeria_folder, the
_batch.json manifest, and the Portal's automatic loading; documents
--user_pass.

Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
An older folder runner that took its order from a README's "Load order"
code block. It was never registered as a console script in pyproject.toml
and nothing references it; dr_egeria_folder is the supported command and
reads _batch.json instead.

Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
@dwolfson
dwolfson merged commit 35ae6d8 into odpi:main Oct 8, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant