diff --git a/PYEGERIA_ISSUES.md b/PYEGERIA_ISSUES.md index 5cc25828..d7812ba7 100644 --- a/PYEGERIA_ISSUES.md +++ b/PYEGERIA_ISSUES.md @@ -1119,6 +1119,49 @@ on an Egeria Server capability that doesn't exist yet — but the pyegeria/ Dr.Egeria-side work each will need once that capability ships is written into the entry now, so it isn't rediscovered from scratch later. +### ISSUE-126: `ProductManager` and `DigitalBusiness` never set `collection_command_root`, so every inherited `CollectionManager` method that uses it raises `AttributeError` + +**Layer:** pyegeria · **Status:** open, logged only, not fixed (awaiting approval) · **Found:** 2026-10-05, +live cleanup of a throwaway data contract (PR #427's live check). + +**What:** `ProductManager.__init__` and `DigitalBusiness.__init__` call `ServerClient.__init__` directly instead +of `CollectionManager.__init__` (`GlossaryManager` does it correctly). `CollectionManager.__init__` is what sets +`self.collection_command_root`, which 27 of `CollectionManager`'s methods use, so on those two subclasses all of +them fail: + +```python +pm = ProductManager(view_server, platform_url, user, pwd) +pm.delete_collection(guid, cascade=True) +# AttributeError: 'ProductManager' object has no attribute 'collection_command_root' +``` + +**Fix (not made):** call `CollectionManager.__init__` from both constructors, or set the attribute. A test that +constructs each `CollectionManager` subclass and asserts the attribute exists would have caught it. + +--- + +### ISSUE-127: `CollectionManager.delete_collection(cascade=True)` silently never sends `cascadeDelete` -- same shape as ISSUE-62 + +**Layer:** pyegeria · **Status:** open, logged only, not fixed (awaiting approval) · **Found:** 2026-10-05, found +by capturing the request body while diagnosing a failed cleanup. + +**What:** `_async_delete_collection` always pre-fills `body = {"class": "DeleteElementRequestBody"}` when none is +given and hands that dict to `_async_delete_element_request(url, body, cascade)`. +`validate_delete_element_request` applies `cascade_delete` only in its *no body* branch, so with that pre-filled dict +the flag is discarded. Captured: `delete_collection("g", cascade=True)` posts +`{'class': 'DeleteElementRequestBody', 'forLineage': False, 'forDuplicateProcessing': False}` with no +`cascadeDelete`. `MetadataExpert.delete_metadata_element(cascade_delete=True)` does send it. + +**Fix (not made):** do not pre-fill the body (pass `None` so the helper applies the flag), or put `cascadeDelete` in +the pre-filled dict. Worth grepping for other callers that pre-fill a default dict *and* pass a cascade flag. + +**Related, an Egeria quirk found alongside:** even with `cascadeDelete: true`, Egeria refused to delete a +`DataStructure` that still had a member `DataField` (`OMAG-GENERIC-HANDLERS-403-005 ... validateNoMemberDataFields`), +so a data contract imported by `import_data_contract*` has to be removed child-first: DataField, DataStructure, +Agreement. + +--- + ### ISSUE-122: `AssetMaker.get_catalog_targets` / `get_catalog_target` send `metadataElementTypeName="CatalogTarget"` (a relationship type) — server rejects with OMAG-COMMON-400-019, surfaced as SERVER_ERROR_500 **Status: fixed on branch `fix/issue-122-catalog-target-type` (2026-10-04), pending PR/merge** diff --git a/docs/dr_egeria_manual.md b/docs/dr_egeria_manual.md index 4cc6d6e1..caa98016 100644 --- a/docs/dr_egeria_manual.md +++ b/docs/dr_egeria_manual.md @@ -290,6 +290,11 @@ Dr.Egeria organizes its commands into "families," each corresponding to a specif - `Initiate Subscription`: take out a subscription to a digital product by running its subscription type (`ProvisioningActionProcess::::Create Subscription::`) with a `Subscription Requester` and `Destination Data Set`. This provisions the subscription, unlike `Create Digital Subscription`, which only records the element. - `Cancel Subscription`: run the cancel process Egeria attached to the subscription when it was provisioned (found through its resource list). - `Create Element`: the generic create-from-template command, moved here from Asset Maker. +- **Digital Products: subscription types and Bitol documents** (added 2026-10-05): Commands in the Digital Product Manager family that wrap `pyegeria/omvs/product_manager.py` (processors in `md_processing/v2/product_manager.py`). None is a create-or-update of a named element, so there is no `Update` form; each is safe to re-run. + - `Create Subscription Type`: add a one-time, periodic or ongoing-update subscription type to a `Digital Product`. Set `Subscription Kind` to `ONE_TIME`, `PERIODIC` (needs `Subscription Notification Interval`, in **minutes**) or `ONGOING_UPDATE` (also needs `Monitored Resources`, typically the product's asset). Egeria creates the product's notification type and the governance action process that `Initiate Subscription` runs to provision a subscription of this type; re-running reconciles an existing type rather than duplicating it. `Subscription Manager`, `Subscription License Type` and `Subscription Service Level Objective` default to the Baudot subscription manager and to those the product is governed by. + - `Import Data Contract` / `Import Data Product`: catalog an Open Data Contract Standard (ODCS) / Open Data Product Standard (ODPS) document as a `DataSharingAgreement` / a `DigitalProduct`. Re-importing the same version updates it; a new version creates new elements; a document with no `id` is skipped (reported as a failure). + - `Publish Data Contract` / `Publish Data Product`: send such a document to an `Integration Daemon`, which passes it to the connectors that registered a Bitol listener (Advanced). + - The document is read from the file named in `Document File` (YAML or JSON), resolved relative to the markdown file being processed, rather than written inline: YAML routinely contains `---` and `#` lines, which the extractor treats as horizontal rules and headings. - **Schema Maker** (added 2026-09-11): The `pyegeria/omvs/schema_maker.py` surface — `Create`/`Update Schema Type` and `Create`/`Update Schema Attribute` (both upsert-capable, live-verified), `Create Schema Type From Template`/`Create Schema Attribute From Template`, 15 `Link`/`Detach` relationship command pairs (Nested Schema Attribute, Attribute for Schema, Foreign Key, External Schema Type, Map From/To Schema Type, Graph Edge, Query Target, Schema, Relational DB Schema, API Operations/Header/Request/Response, Schema Type Option — `Link Nested Schema Attribute` live-verified), and 3 `Add`/`Remove` classification pairs (Primary Key, Type Embedded Attribute, Calculated Value) on a schema attribute/column. Effectivity dates, Anchor ID, and Anchor Scope IDs are Advanced-level; everything else is Basic. #### `Create Report`'s two Dictionary attributes, worked examples diff --git a/md_processing/data/compact_commands/commands_digital_products_compact.json b/md_processing/data/compact_commands/commands_digital_products_compact.json index fd04677d..6bd3b9ea 100644 --- a/md_processing/data/compact_commands/commands_digital_products_compact.json +++ b/md_processing/data/compact_commands/commands_digital_products_compact.json @@ -2605,6 +2605,181 @@ "max_cardinality": 1, "level": "Advanced", "Journal Entry": "" + }, + "Digital Product": { + "variable_name": "digital_product", + "inUpdate": false, + "attr_labels": "", + "examples": "", + "default_value": "", + "valid_values": [], + "existing_element": "DigitalProduct", + "description": "The digital product the subscription type is added to (name, qualified name or GUID).", + "generated": false, + "style": "Reference Name", + "user_specified": true, + "unique": false, + "input_required": true, + "min_cardinality": 1, + "max_cardinality": 1, + "level": "Basic", + "Journal Entry": "Added 2026-10-05 -- product-manager subscription types and ODCS/ODPS documents." + }, + "Subscription Kind": { + "variable_name": "subscription_kind", + "inUpdate": false, + "attr_labels": "", + "examples": "", + "default_value": "ONE_TIME", + "valid_values": [ + "ONE_TIME", + "PERIODIC", + "ONGOING_UPDATE" + ], + "existing_element": "", + "description": "How often subscribers are notified, and so how often the product's data is delivered: ONE_TIME (a single delivery, typically to evaluate the product), PERIODIC (every Subscription Notification Interval minutes) or ONGOING_UPDATE (when a Monitored Resource changes, but no more often than every Subscription Notification Interval minutes).", + "generated": false, + "style": "Valid Value", + "user_specified": true, + "unique": false, + "input_required": true, + "min_cardinality": 1, + "max_cardinality": 1, + "level": "Basic", + "Journal Entry": "Added 2026-10-05 -- product-manager subscription types and ODCS/ODPS documents." + }, + "Subscription Manager": { + "variable_name": "subscription_manager", + "inUpdate": false, + "attr_labels": "", + "examples": "", + "default_value": "", + "valid_values": [], + "existing_element": "IntegrationConnector", + "description": "The integration connector that notifies subscribers. Defaults to the Baudot Digital Product Subscription Manager from the digital products content pack.", + "generated": false, + "style": "Reference Name", + "user_specified": true, + "unique": false, + "input_required": false, + "min_cardinality": 0, + "max_cardinality": 1, + "level": "Advanced", + "Journal Entry": "Added 2026-10-05 -- product-manager subscription types and ODCS/ODPS documents." + }, + "Subscription License Type": { + "variable_name": "subscription_license_type", + "inUpdate": false, + "attr_labels": "", + "examples": "", + "default_value": "", + "valid_values": [], + "existing_element": "LicenseType", + "description": "The license type granted to a subscriber's asset. Defaults to the first license type the product is governed by.", + "generated": false, + "style": "Reference Name", + "user_specified": true, + "unique": false, + "input_required": false, + "min_cardinality": 0, + "max_cardinality": 1, + "level": "Advanced", + "Journal Entry": "Added 2026-10-05 -- product-manager subscription types and ODCS/ODPS documents." + }, + "Subscription Service Level Objective": { + "variable_name": "subscription_service_level_objective", + "inUpdate": false, + "attr_labels": "", + "examples": "", + "default_value": "", + "valid_values": [], + "existing_element": "ServiceLevelObjective", + "description": "The service level objective the subscription offers. Defaults to the first service level objective the product is governed by.", + "generated": false, + "style": "Reference Name", + "user_specified": true, + "unique": false, + "input_required": false, + "min_cardinality": 0, + "max_cardinality": 1, + "level": "Advanced", + "Journal Entry": "Added 2026-10-05 -- product-manager subscription types and ODCS/ODPS documents." + }, + "Subscription Notification Interval": { + "variable_name": "subscription_notification_interval", + "inUpdate": false, + "attr_labels": "", + "examples": "", + "default_value": "", + "valid_values": [], + "existing_element": "", + "description": "Minutes between notifications -- the delivery interval for a PERIODIC subscription, the minimum interval for an ONGOING_UPDATE one. In MINUTES (unlike the Governance Officer family's 'Notification Interval', which is milliseconds).", + "generated": false, + "style": "Simple Int", + "user_specified": true, + "unique": false, + "input_required": false, + "min_cardinality": 0, + "max_cardinality": 1, + "level": "Basic", + "Journal Entry": "Added 2026-10-05 -- product-manager subscription types and ODCS/ODPS documents." + }, + "Monitored Resources": { + "variable_name": "monitored_resources", + "inUpdate": false, + "attr_labels": "", + "examples": "", + "default_value": "", + "valid_values": [], + "existing_element": "Referenceable", + "description": "For an ONGOING_UPDATE subscription: the elements whose changes trigger a notification -- typically the product's asset.", + "generated": false, + "style": "Reference Name List", + "user_specified": true, + "unique": false, + "input_required": false, + "min_cardinality": 0, + "max_cardinality": 1, + "level": "Basic", + "Journal Entry": "Added 2026-10-05 -- product-manager subscription types and ODCS/ODPS documents." + }, + "Document File": { + "variable_name": "document_file", + "inUpdate": false, + "attr_labels": "", + "examples": "", + "default_value": "", + "valid_values": [], + "existing_element": "", + "description": "Path to a file holding the document (YAML or JSON): an Open Data Contract Standard (ODCS) data contract or an Open Data Product Standard (ODPS) data product. A relative path is resolved against the directory of the markdown file being processed.", + "generated": false, + "style": "Simple", + "user_specified": true, + "unique": false, + "input_required": true, + "min_cardinality": 1, + "max_cardinality": 1, + "level": "Basic", + "Journal Entry": "Added 2026-10-05 -- product-manager subscription types and ODCS/ODPS documents." + }, + "Integration Daemon": { + "variable_name": "integration_daemon", + "inUpdate": false, + "attr_labels": "", + "examples": "", + "default_value": "", + "valid_values": [], + "existing_element": "SoftwareServer", + "description": "The integration daemon (a SoftwareServer) whose Bitol listeners receive the document.", + "generated": false, + "style": "Reference Name", + "user_specified": true, + "unique": false, + "input_required": true, + "min_cardinality": 1, + "max_cardinality": 1, + "level": "Basic", + "Journal Entry": "Added 2026-10-05 -- product-manager subscription types and ODCS/ODPS documents." } }, "bundles": { @@ -2788,6 +2963,34 @@ "ISC Qualified Name", "GUID" ] + }, + "Subscription Type Base": { + "inherits": "Request Base", + "own_attributes": [ + "Digital Product", + "Subscription Kind", + "Display Name", + "Description", + "Identifier", + "Subscription Notification Interval", + "Monitored Resources", + "Subscription Manager", + "Subscription License Type", + "Subscription Service Level Objective" + ] + }, + "Import Data Document Base": { + "inherits": "Request Base", + "own_attributes": [ + "Document File" + ] + }, + "Publish Data Document Base": { + "inherits": "Request Base", + "own_attributes": [ + "Integration Daemon", + "Document File" + ] } }, "commands": { @@ -3037,6 +3240,101 @@ "bundle": "Digital Subscriber", "custom_attributes": [], "Journal Entry": "" + }, + "Create Subscription Type": { + "display_name": "create_subscription_type", + "qn_prefix": "", + "alternate_names": [], + "family": "Digital Product Manager", + "description": "Add a subscription type to a digital product: a one-time, periodic or ongoing-update way to subscribe. Creates the product's notification type and returns the governance action process that Initiate Subscription runs to provision a subscription of this type. Safe to re-run: an existing subscription type is brought into line with the request rather than duplicated.", + "verb": "Create", + "upsert": false, + "attach": false, + "level": "Basic", + "find_method": "", + "find_constraints": "", + "extra_find": "", + "extra_constraints": "", + "OM_TYPE": "SubscribingActionProcess", + "bundle": "Subscription Type Base", + "custom_attributes": [], + "Journal Entry": "Added 2026-10-05 -- product-manager subscription types and ODCS/ODPS documents." + }, + "Import Data Contract": { + "display_name": "import_data_contract", + "qn_prefix": "", + "alternate_names": [], + "family": "Digital Product Manager", + "description": "Catalog an Open Data Contract Standard (ODCS) data contract file directly as an Agreement classified as a DataSharingAgreement. Re-importing the same contract version updates it; a new version creates new elements.", + "verb": "Import", + "upsert": false, + "attach": false, + "level": "Basic", + "find_method": "", + "find_constraints": "", + "extra_find": "", + "extra_constraints": "", + "OM_TYPE": "DataSharingAgreement", + "bundle": "Import Data Document Base", + "custom_attributes": [], + "Journal Entry": "Added 2026-10-05 -- product-manager subscription types and ODCS/ODPS documents." + }, + "Import Data Product": { + "display_name": "import_data_product", + "qn_prefix": "", + "alternate_names": [], + "family": "Digital Product Manager", + "description": "Catalog an Open Data Product Standard (ODPS) data product file directly as a DigitalProduct. Re-importing the same product version updates it; a new version creates new elements.", + "verb": "Import", + "upsert": false, + "attach": false, + "level": "Basic", + "find_method": "", + "find_constraints": "", + "extra_find": "", + "extra_constraints": "", + "OM_TYPE": "DigitalProduct", + "bundle": "Import Data Document Base", + "custom_attributes": [], + "Journal Entry": "Added 2026-10-05 -- product-manager subscription types and ODCS/ODPS documents." + }, + "Publish Data Contract": { + "display_name": "publish_data_contract", + "qn_prefix": "", + "alternate_names": [], + "family": "Digital Product Manager", + "description": "Send an ODCS data contract file to an integration daemon, which passes it on to the integration connectors that registered a Bitol listener.", + "verb": "Publish", + "upsert": false, + "attach": false, + "level": "Advanced", + "find_method": "", + "find_constraints": "", + "extra_find": "", + "extra_constraints": "", + "OM_TYPE": "SoftwareServer", + "bundle": "Publish Data Document Base", + "custom_attributes": [], + "Journal Entry": "Added 2026-10-05 -- product-manager subscription types and ODCS/ODPS documents." + }, + "Publish Data Product": { + "display_name": "publish_data_product", + "qn_prefix": "", + "alternate_names": [], + "family": "Digital Product Manager", + "description": "Send an ODPS data product file to an integration daemon, which passes it on to the integration connectors that registered a Bitol listener.", + "verb": "Publish", + "upsert": false, + "attach": false, + "level": "Advanced", + "find_method": "", + "find_constraints": "", + "extra_find": "", + "extra_constraints": "", + "OM_TYPE": "SoftwareServer", + "bundle": "Publish Data Document Base", + "custom_attributes": [], + "Journal Entry": "Added 2026-10-05 -- product-manager subscription types and ODCS/ODPS documents." } } } diff --git a/md_processing/data/generated_format_sets.json b/md_processing/data/generated_format_sets.json index c979ca79..1da46c29 100644 --- a/md_processing/data/generated_format_sets.json +++ b/md_processing/data/generated_format_sets.json @@ -99901,6 +99901,708 @@ "get_additional_props": null, "question_spec": null }, + "Subscription-Type-DrE-Basic": { + "target_type": "Subscription Type", + "heading": "Subscription-Type-DrE-Basic Attributes", + "description": "Auto-generated format for Subscription Type (Create, Basic).", + "aliases": [], + "annotations": {}, + "family": "Digital Product Manager", + "formats": [ + { + "types": [ + "LIST" + ], + "attributes": [ + { + "name": "Display Name", + "key": "display_name", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Digital Product", + "key": "digital_product", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Kind", + "key": "subscription_kind", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [ + "ONE_TIME", + "PERIODIC", + "ONGOING_UPDATE" + ] + }, + { + "name": "Description", + "key": "description", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Notification Interval", + "key": "subscription_notification_interval", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Monitored Resources", + "key": "monitored_resources", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + } + ], + "columns": [ + { + "name": "Display Name", + "key": "display_name", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Digital Product", + "key": "digital_product", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Kind", + "key": "subscription_kind", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [ + "ONE_TIME", + "PERIODIC", + "ONGOING_UPDATE" + ] + }, + { + "name": "Description", + "key": "description", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Notification Interval", + "key": "subscription_notification_interval", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Monitored Resources", + "key": "monitored_resources", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + } + ] + }, + { + "types": [ + "ALL" + ], + "attributes": [ + { + "name": "Digital Product", + "key": "digital_product", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Kind", + "key": "subscription_kind", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [ + "ONE_TIME", + "PERIODIC", + "ONGOING_UPDATE" + ] + }, + { + "name": "Display Name", + "key": "display_name", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Description", + "key": "description", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Notification Interval", + "key": "subscription_notification_interval", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Monitored Resources", + "key": "monitored_resources", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + } + ], + "columns": [ + { + "name": "Digital Product", + "key": "digital_product", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Kind", + "key": "subscription_kind", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [ + "ONE_TIME", + "PERIODIC", + "ONGOING_UPDATE" + ] + }, + { + "name": "Display Name", + "key": "display_name", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Description", + "key": "description", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Notification Interval", + "key": "subscription_notification_interval", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Monitored Resources", + "key": "monitored_resources", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + } + ] + } + ], + "action": null, + "get_additional_props": null, + "question_spec": null + }, + "Subscription-Type-DrE-Advanced": { + "target_type": "Subscription Type", + "heading": "Subscription-Type-DrE-Advanced Attributes", + "description": "Auto-generated format for Subscription Type (Create, Advanced).", + "aliases": [], + "annotations": {}, + "family": "Digital Product Manager", + "formats": [ + { + "types": [ + "LIST" + ], + "attributes": [ + { + "name": "Display Name", + "key": "display_name", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Effective From", + "key": "effective_from", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Effective Time", + "key": "effective_time", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Effective To", + "key": "effective_to", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Digital Product", + "key": "digital_product", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Kind", + "key": "subscription_kind", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [ + "ONE_TIME", + "PERIODIC", + "ONGOING_UPDATE" + ] + }, + { + "name": "Description", + "key": "description", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Identifier", + "key": "identifier", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Notification Interval", + "key": "subscription_notification_interval", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Monitored Resources", + "key": "monitored_resources", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Manager", + "key": "subscription_manager", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription License Type", + "key": "subscription_license_type", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Service Level Objective", + "key": "subscription_service_level_objective", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + } + ], + "columns": [ + { + "name": "Display Name", + "key": "display_name", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Effective From", + "key": "effective_from", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Effective Time", + "key": "effective_time", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Effective To", + "key": "effective_to", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Digital Product", + "key": "digital_product", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Kind", + "key": "subscription_kind", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [ + "ONE_TIME", + "PERIODIC", + "ONGOING_UPDATE" + ] + }, + { + "name": "Description", + "key": "description", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Identifier", + "key": "identifier", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Notification Interval", + "key": "subscription_notification_interval", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Monitored Resources", + "key": "monitored_resources", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Manager", + "key": "subscription_manager", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription License Type", + "key": "subscription_license_type", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Service Level Objective", + "key": "subscription_service_level_objective", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + } + ] + }, + { + "types": [ + "ALL" + ], + "attributes": [ + { + "name": "Effective From", + "key": "effective_from", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Effective Time", + "key": "effective_time", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Effective To", + "key": "effective_to", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Digital Product", + "key": "digital_product", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Kind", + "key": "subscription_kind", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [ + "ONE_TIME", + "PERIODIC", + "ONGOING_UPDATE" + ] + }, + { + "name": "Display Name", + "key": "display_name", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Description", + "key": "description", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Identifier", + "key": "identifier", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Notification Interval", + "key": "subscription_notification_interval", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Monitored Resources", + "key": "monitored_resources", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Manager", + "key": "subscription_manager", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription License Type", + "key": "subscription_license_type", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Service Level Objective", + "key": "subscription_service_level_objective", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + } + ], + "columns": [ + { + "name": "Effective From", + "key": "effective_from", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Effective Time", + "key": "effective_time", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Effective To", + "key": "effective_to", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Digital Product", + "key": "digital_product", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Kind", + "key": "subscription_kind", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [ + "ONE_TIME", + "PERIODIC", + "ONGOING_UPDATE" + ] + }, + { + "name": "Display Name", + "key": "display_name", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Description", + "key": "description", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Identifier", + "key": "identifier", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Notification Interval", + "key": "subscription_notification_interval", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Monitored Resources", + "key": "monitored_resources", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Manager", + "key": "subscription_manager", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription License Type", + "key": "subscription_license_type", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + }, + { + "name": "Subscription Service Level Objective", + "key": "subscription_service_level_objective", + "value": null, + "format": false, + "detail_spec": null, + "valid_values": [] + } + ] + } + ], + "action": null, + "get_additional_props": null, + "question_spec": null + }, "External-Reference-DrE-Basic": { "target_type": "External Reference", "heading": "External-Reference-DrE-Basic Attributes", diff --git a/md_processing/dr_egeria.py b/md_processing/dr_egeria.py index 3d0f785e..d869ec82 100644 --- a/md_processing/dr_egeria.py +++ b/md_processing/dr_egeria.py @@ -51,6 +51,7 @@ AssetMakerProcessor, InitiateGovernanceActionProcessProcessor, InitiateGovernanceActionTypeProcessor, InitiateSurveyProcessor, InitiateSubscriptionProcessor, CancelSubscriptionProcessor, + SubscriptionTypeProcessor, BitolDocumentProcessor, SchemaElementProcessor, SchemaTemplateProcessor, SchemaLinkProcessor, SchemaClassificationProcessor, ) @@ -414,6 +415,12 @@ def reg(base, cls): reg("Initiate Survey", InitiateSurveyProcessor) reg("Initiate Subscription", InitiateSubscriptionProcessor) reg("Cancel Subscription", CancelSubscriptionProcessor) + + # Digital Products: subscription types and ODCS/ODPS Bitol documents (product_manager.py) + reg("Create Subscription Type", SubscriptionTypeProcessor) + for _bitol_verb in ("Import", "Publish"): + for _bitol_doc in ("Data Contract", "Data Product"): + reg(f"{_bitol_verb} {_bitol_doc}", BitolDocumentProcessor) reg("Create Element", AssetMakerProcessor) # Asset Maker (Automated Curation's create-from-template surface) @@ -679,6 +686,7 @@ async def _debug_make_request(self_inner, request_type, endpoint, context = { "directive": directive, "input_file": input_file, + "input_path": full_file_path, # resolved path, for commands that read files relative to the document "request_id": str(uuid.uuid4()), "debug": debug, } diff --git a/md_processing/v2/__init__.py b/md_processing/v2/__init__.py index 68117b1a..516d97ab 100644 --- a/md_processing/v2/__init__.py +++ b/md_processing/v2/__init__.py @@ -47,4 +47,7 @@ InitiateGovernanceActionProcessProcessor, InitiateGovernanceActionTypeProcessor, InitiateSurveyProcessor, InitiateSubscriptionProcessor, CancelSubscriptionProcessor, ) +from .product_manager import ( + SubscriptionTypeProcessor, BitolDocumentProcessor, +) from .schema_maker import SchemaElementProcessor, SchemaTemplateProcessor, SchemaLinkProcessor, SchemaClassificationProcessor diff --git a/md_processing/v2/extraction.py b/md_processing/v2/extraction.py index 96451ecb..d41cf62d 100644 --- a/md_processing/v2/extraction.py +++ b/md_processing/v2/extraction.py @@ -34,6 +34,11 @@ # row, and every downstream definition/link claiming the never-created # valid value fails with an unrelated-looking message instead. "Setup", "Clear", "Set", + # Digital Products family: "Import Data Contract" / "Import Data Product" (catalog an + # ODCS / ODPS document) and "Publish Data Contract" / "Publish Data Product" (send one to an + # integration daemon). Without these the `## Import ...` / `## Publish ...` headings never match + # cmd_header_rx and silently fall through as prose -- no error, no summary row. + "Import", "Publish", } # Commands that appear as a single word (no separate verb) and map to canonical verb+object diff --git a/md_processing/v2/product_manager.py b/md_processing/v2/product_manager.py new file mode 100644 index 00000000..46a98417 --- /dev/null +++ b/md_processing/v2/product_manager.py @@ -0,0 +1,200 @@ +""" +Product Manager Processors for Dr.Egeria v2. + +Handles the commands of the "Digital Products" family that wrap +`pyegeria/omvs/product_manager.py`'s subscription-type and Bitol-document calls: + +- Create Subscription Type: add a one-time, periodic or ongoing-update subscription + type to a digital product. Egeria creates the product's notification type and the + governance action process that `Initiate Subscription` (Automation family) runs to + provision a subscription of that type. Egeria reconciles an existing subscription + type with the request rather than duplicating it, so the command is safe to re-run. +- Import Data Contract / Import Data Product: catalog an Open Data Contract Standard + (ODCS) / Open Data Product Standard (ODPS) document directly as an Agreement + classified as a DataSharingAgreement / as a DigitalProduct. Re-importing the same + version updates the elements; a new version creates new ones. A document with no + `id` is skipped by Egeria. +- Publish Data Contract / Publish Data Product: send such a document to an integration + daemon, which passes it to the connectors that registered a Bitol listener. + +The documents are read from a file named by the `Document File` attribute (YAML or +JSON), not written inline: YAML routinely contains `---` and `#` lines, which are +markdown horizontal rules and headings to the Dr.Egeria extractor. + +Every command here is an action, not a create-or-update of a named element, so there +is no existing element to look up and no Create<->Update rewrite (same shape as the +Automation family's processors). + +Problems the author can fix in the markdown (a missing file, a PERIODIC type with no +interval) raise ValueError rather than PyegeriaException: a plain-message +PyegeriaException renders with Egeria's "unable to connect to the platform / check the +URL" boilerplate, which is wrong for these. The dispatcher reports either as a failure. +""" +import os +from typing import Any, Dict, List, Optional + +from loguru import logger + +from md_processing.v2.processors import AsyncBaseCommandProcessor + +SUBSCRIPTION_KINDS = ("ONE_TIME", "PERIODIC", "ONGOING_UPDATE") + + +def _v(attributes: Dict[str, Any], name: str, default=None): + value = attributes.get(name, {}).get("value", default) + return default if value in (None, "") else value + + +def _guid(attributes: Dict[str, Any], name: str) -> Optional[str]: + guid = attributes.get(name, {}).get("guid") + return None if not guid or str(guid).startswith("(Planned:") else guid + + +def _guid_list(attributes: Dict[str, Any], name: str) -> List[str]: + guid_list = attributes.get(name, {}).get("guid_list") + if isinstance(guid_list, list): + return [g for g in guid_list if g and not str(g).startswith("(Planned:")] + return [] + + +def normalize_subscription_kind(value: Any) -> str: + """'One-Time' / 'ongoing update' / 'PERIODIC' -> 'ONE_TIME' / 'ONGOING_UPDATE' / 'PERIODIC'.""" + kind = str(value or "ONE_TIME").strip().upper().replace("-", "_").replace(" ", "_") + if kind not in SUBSCRIPTION_KINDS: + raise ValueError( + f"Subscription Kind '{value}' is not one of {', '.join(SUBSCRIPTION_KINDS)}.") + return kind + + +class ProductManagerProcessor(AsyncBaseCommandProcessor): + """Shared behaviour for the Digital Products family's action commands.""" + + def supports_target_element_lookup(self) -> bool: + # An action, not a named element: no as-is lookup, no upsert rewrite. + return False + + async def fetch_as_is(self) -> Optional[Dict[str, Any]]: + return None + + def _result(self, what: str, guid: Optional[str] = None) -> str: + if guid: + self.parsed_output["guid"] = guid + logger.success(f"{self.command.verb} {self.command.object_type}: {what}" + (f" ({guid})" if guid else "")) + return f"\n\n## {self.command.verb} {self.command.object_type}\n\n{what}" + (f": `{guid}`" if guid else "") + "\n" + + +class SubscriptionTypeProcessor(ProductManagerProcessor): + """Create Subscription Type.""" + + def _build(self, attributes: Dict[str, Any]) -> tuple: + product_guid = _guid(attributes, "Digital Product") + if not product_guid: + raise ValueError("'Digital Product' could not be resolved to an existing digital product.") + kind = normalize_subscription_kind(_v(attributes, "Subscription Kind")) + interval = _v(attributes, "Subscription Notification Interval") + monitored = _guid_list(attributes, "Monitored Resources") + + if kind == "PERIODIC" and not interval: + raise ValueError("A PERIODIC subscription type needs a 'Subscription Notification Interval' (minutes).") + if kind == "ONGOING_UPDATE" and not monitored: + raise ValueError( + "An ONGOING_UPDATE subscription type needs at least one 'Monitored Resources' element " + "(typically the product's asset).") + if kind == "ONE_TIME" and (interval or monitored): + self._add_warning("A ONE_TIME subscription type ignores 'Subscription Notification Interval' and " + "'Monitored Resources'; Egeria will not use them.") + if kind == "PERIODIC" and monitored: + self._add_warning("A PERIODIC subscription type ignores 'Monitored Resources'.") + + body: Dict[str, Any] = { + "identifier": _v(attributes, "Identifier"), + "displayName": _v(attributes, "Display Name"), + "description": _v(attributes, "Description"), + "subscriptionManagerGUID": _guid(attributes, "Subscription Manager"), + "licenseTypeGUID": _guid(attributes, "Subscription License Type"), + "serviceLevelObjectiveGUID": _guid(attributes, "Subscription Service Level Objective"), + } + if kind != "ONE_TIME" and interval: + body["notificationInterval"] = int(interval) + if kind == "ONGOING_UPDATE": + body["monitoredResourceGUIDs"] = monitored + return product_guid, kind, {k: v for k, v in body.items() if v is not None} + + async def apply_changes(self) -> str: + attributes = self.parsed_output["attributes"] + product_guid, kind, body = self._build(attributes) + client = self.client.product_manager + create = {"ONE_TIME": client._async_create_one_time_subscription, + "PERIODIC": client._async_create_periodic_subscription, + "ONGOING_UPDATE": client._async_create_ongoing_update_subscription}[kind] + guid = await create(product_guid, body or None) + if not guid: + raise ValueError(f"Egeria did not return a subscription process for the {kind} subscription type.") + label = body.get("identifier") or body.get("displayName") or kind + return self._result(f"Subscription type `{label}` ({kind}) added to product " + f"`{_v(attributes, 'Digital Product', product_guid)}`; " + f"governance action process", guid) + + +class BitolDocumentProcessor(ProductManagerProcessor): + """Import / Publish Data Contract / Data Product -- the documents come from a file.""" + + def _document_path(self, attributes: Dict[str, Any]) -> str: + name = _v(attributes, "Document File") + if not name: + raise ValueError("'Document File' is required.") + path = os.path.expanduser(str(name).strip()) + if not os.path.isabs(path): + source = self.context.get("input_path") or self.context.get("input_file") or "" + path = os.path.join(os.path.dirname(os.path.abspath(os.path.expanduser(source))) if source else os.getcwd(), + path) + return os.path.normpath(path) + + def _read_document(self, attributes: Dict[str, Any]) -> str: + path = self._document_path(attributes) + try: + with open(path, encoding="utf-8") as f: + text = f.read() + except OSError as e: + raise ValueError(f"Cannot read Document File '{path}': {e}") from e + if not text.strip(): + raise ValueError(f"Document File '{path}' is empty.") + return text + + async def validate_only(self) -> str: + result = await super().validate_only() + try: + self._read_document(self.parsed_output.get("attributes", {})) + except ValueError as e: + self._add_warning(str(e)) + return result + + @property + def _is_contract(self) -> bool: + return self.command.object_type == "Data Contract" + + async def apply_changes(self) -> str: + attributes = self.parsed_output["attributes"] + document = self._read_document(attributes) + client = self.client.product_manager + kind = "data contract" if self._is_contract else "data product" + path = self._document_path(attributes) + + if self.command.verb == "Import": + call = client._async_import_data_contract_string if self._is_contract \ + else client._async_import_data_product_string + guid = await call(document) + if not guid: + raise ValueError( + f"Egeria skipped the {kind} in '{path}' -- a document with no 'id' cannot be catalogued.") + return self._result(f"Imported {kind} from `{os.path.basename(path)}` as", guid) + + # Publish + daemon_guid = _guid(attributes, "Integration Daemon") + if not daemon_guid: + raise ValueError("'Integration Daemon' could not be resolved to an existing integration daemon.") + call = client._async_publish_data_contract_string if self._is_contract \ + else client._async_publish_data_product_string + await call(daemon_guid, document) + return self._result(f"Published {kind} `{os.path.basename(path)}` to integration daemon " + f"`{_v(attributes, 'Integration Daemon', daemon_guid)}`") diff --git a/pyegeria/core/_base_server_client.py b/pyegeria/core/_base_server_client.py index 000eb07b..8deea72d 100644 --- a/pyegeria/core/_base_server_client.py +++ b/pyegeria/core/_base_server_client.py @@ -493,9 +493,13 @@ async def _async_make_request( *, timeout: int = None, _retry_on_auth: bool = True, + as_text: bool = False, ) -> Response | str: """Make an asynchronous request to the Egeria API. + `as_text=True` sends a str payload as `text/plain` (using `self.text_headers`) rather than + the default `application/json`, for endpoints that take a raw document (YAML or JSON) string. + Parameters ---------- request_type : str @@ -563,7 +567,7 @@ async def _async_make_request( elif type(payload) is str: response = await self.session.post( endpoint, - headers=self.headers, + headers=self.text_headers if as_text else self.headers, content=payload, timeout=timeout, params=params @@ -617,7 +621,7 @@ async def _async_make_request( if new_token and new_token != "FAILED": return await self._async_make_request( request_type, endpoint, payload, is_json, params, - timeout=timeout, _retry_on_auth=False, + timeout=timeout, _retry_on_auth=False, as_text=as_text, ) additional_info = {"userid": self.user_id} diff --git a/pyegeria/models/__init__.py b/pyegeria/models/__init__.py index 7d4fe548..09a1b77b 100644 --- a/pyegeria/models/__init__.py +++ b/pyegeria/models/__init__.py @@ -17,6 +17,7 @@ DeleteRequestBody, DeleteElementRequestBody, DeleteRelationshipRequestBody, + NewSubscriptionTypeRequestBody, DeleteClassificationRequestBody, InitialClassifications, NewElementRequestBody, diff --git a/pyegeria/models/models.py b/pyegeria/models/models.py index 6a9951d2..ea9622c6 100644 --- a/pyegeria/models/models.py +++ b/pyegeria/models/models.py @@ -924,3 +924,19 @@ class OpenMetadataRootElement(PyegeriaModel): # --- Overall Payload Type --- # The entire JSON is a list of OpenMetadataRootElement OpenMetadataPayload = list[OpenMetadataRootElement] + + +class NewSubscriptionTypeRequestBody(RequestBody): + """Optional body for ProductManager's create_*_subscription methods. Every field is optional: + the server defaults the subscription manager to the Baudot one from the content pack, the + identifier per subscription kind, and the licence type / service level objective to those the + digital product is governed by.""" + class_: Annotated[Literal["NewSubscriptionTypeRequestBody"], Field(alias="class")] + subscription_manager_guid: str | None = None + identifier: str | None = None + display_name: str | None = None + description: str | None = None + license_type_guid: str | None = None + service_level_objective_guid: str | None = None + notification_interval: int | None = None # minutes + monitored_resource_guids: list[str] | None = Field(None, alias="monitoredResourceGUIDs") diff --git a/pyegeria/omvs/product_manager.py b/pyegeria/omvs/product_manager.py index 00df1b23..7f38fefd 100644 --- a/pyegeria/omvs/product_manager.py +++ b/pyegeria/omvs/product_manager.py @@ -20,7 +20,9 @@ DeleteRelationshipRequestBody, UpdateRelationshipRequestBody, DeleteElementRequestBody, + NewSubscriptionTypeRequestBody, ) +from pyegeria.core._globals import NO_ELEMENTS_FOUND from pyegeria.core.utils import dynamic_catch, body_slimmer from loguru import logger @@ -1801,3 +1803,329 @@ def detach_digital_product_dependency_by_id(self, digital_product_dependency_rel """Detach one specific DigitalProductDependency relationship, identified by its own relationship GUID.""" loop = asyncio.get_event_loop() loop.run_until_complete(self._async_detach_digital_product_dependency_by_id(digital_product_dependency_relationship_guid, body)) + + # + # Subscription types, and Open Data Contract / Product Standard (ODCS / ODPS) documents. + # + + async def _async_new_subscription_type(self, digital_product_guid: str, kind: str, + body: Optional[dict | NewSubscriptionTypeRequestBody]) -> Optional[str]: + """POST a (optional) NewSubscriptionTypeRequestBody to .../subscription-types/{kind}; return the GUID of + the governance action process that creates a subscription of this type.""" + url = (f"{self.product_manager_command_root}/digital-products/" + f"{digital_product_guid}/subscription-types/{kind}") + payload = None + if isinstance(body, NewSubscriptionTypeRequestBody): + payload = body.model_dump_json(indent=2, exclude_none=True) + elif isinstance(body, dict): + validated = self._validate_body(NewSubscriptionTypeRequestBody.model_validate, + {"class": "NewSubscriptionTypeRequestBody", **body}) + payload = validated.model_dump_json(indent=2, exclude_none=True) + response = await self._async_make_request("POST", url, payload) + guid = response.json().get("guid") + logger.info(f"Added {kind} subscription type to {digital_product_guid}: {guid}") + return guid + + @dynamic_catch + async def _async_create_one_time_subscription( + self, digital_product_guid: str, body: Optional[dict | NewSubscriptionTypeRequestBody] = None + ) -> Optional[str]: + """Add a one-time subscription type to a digital product: subscribers receive a single notification, and + so a single delivery of the product's data (typically to evaluate it). Async version. + + Parameters + ---------- + digital_product_guid : str + The digital product. + body : dict | NewSubscriptionTypeRequestBody, optional + Every field is optional: subscriptionManagerGUID defaults to the Baudot subscription manager; + identifier to ONE-TIME-SUBSCRIPTION; licenseTypeGUID and serviceLevelObjectiveGUID to those the + product is governed by. + + Returns + ------- + str | None + The GUID of the governance action process that creates a subscription of this type. + + Notes + ----- + Sample JSON body: + ```json + { + "class" : "NewSubscriptionTypeRequestBody", + "identifier" : "EVALUATION-SUBSCRIPTION", + "displayName" : "Evaluation subscription", + "description" : "Delivers the data once to allow an evaluation of the product data." + } + ``` + """ + return await self._async_new_subscription_type(digital_product_guid, "one-time", body) + + @dynamic_catch + def create_one_time_subscription( + self, digital_product_guid: str, body: Optional[dict | NewSubscriptionTypeRequestBody] = None + ) -> Optional[str]: + """Add a one-time subscription type to a digital product. Returns the GUID of the governance action + process that creates a subscription of this type.""" + loop = asyncio.get_event_loop() + return loop.run_until_complete(self._async_create_one_time_subscription(digital_product_guid, body)) + + @dynamic_catch + async def _async_create_periodic_subscription( + self, digital_product_guid: str, body: Optional[dict | NewSubscriptionTypeRequestBody] = None + ) -> Optional[str]: + """Add a periodic subscription type to a digital product: subscribers receive a notification, and so a + delivery of the product's data, every `notificationInterval` minutes. A product offering more than one + periodic subscription type (daily and weekly, say) needs a distinct `identifier` for each. Async version. + + Returns + ------- + str | None + The GUID of the governance action process that creates a subscription of this type. + + Notes + ----- + Sample JSON body: + ```json + { + "class" : "NewSubscriptionTypeRequestBody", + "identifier" : "DAILY-REFRESH-SUBSCRIPTION", + "displayName" : "Daily refresh subscription", + "description" : "Delivers the data once a day.", + "notificationInterval" : 1440 + } + ``` + """ + return await self._async_new_subscription_type(digital_product_guid, "periodic", body) + + @dynamic_catch + def create_periodic_subscription( + self, digital_product_guid: str, body: Optional[dict | NewSubscriptionTypeRequestBody] = None + ) -> Optional[str]: + """Add a periodic subscription type to a digital product. Returns the GUID of the governance action + process that creates a subscription of this type.""" + loop = asyncio.get_event_loop() + return loop.run_until_complete(self._async_create_periodic_subscription(digital_product_guid, body)) + + @dynamic_catch + async def _async_create_ongoing_update_subscription( + self, digital_product_guid: str, body: Optional[dict | NewSubscriptionTypeRequestBody] = None + ) -> Optional[str]: + """Add an ongoing-update subscription type to a digital product: subscribers receive a notification, and + so a delivery of the product's data, whenever one of the `monitoredResourceGUIDs` changes - but no more + often than every `notificationInterval` minutes. The monitored resource is typically the product's + asset. Async version. + + Returns + ------- + str | None + The GUID of the governance action process that creates a subscription of this type. + + Notes + ----- + Sample JSON body: + ```json + { + "class" : "NewSubscriptionTypeRequestBody", + "identifier" : "ONGOING-UPDATE-SUBSCRIPTION", + "displayName" : "Ongoing update subscription", + "description" : "Delivers data updates within an hour of receiving the new data.", + "monitoredResourceGUIDs" : [ "add asset guid here" ], + "notificationInterval" : 10 + } + ``` + """ + return await self._async_new_subscription_type(digital_product_guid, "ongoing-update", body) + + @dynamic_catch + def create_ongoing_update_subscription( + self, digital_product_guid: str, body: Optional[dict | NewSubscriptionTypeRequestBody] = None + ) -> Optional[str]: + """Add an ongoing-update subscription type to a digital product. Returns the GUID of the governance + action process that creates a subscription of this type.""" + loop = asyncio.get_event_loop() + return loop.run_until_complete(self._async_create_ongoing_update_subscription(digital_product_guid, body)) + + async def _async_post_document(self, url: str, document: str | dict) -> Optional[str]: + """POST a data contract / data product document and return the "guid" the server answers with, if any. + A str (YAML or JSON text) is sent as text/plain; a dict (the ODCS / ODPS bean) as JSON.""" + if isinstance(document, str): + response = await self._async_make_request("POST", url, document, as_text=True) + elif isinstance(document, dict): + response = await self._async_make_request("POST", url, document) + else: + raise ValueError("document must be a str (YAML or JSON text) or a dict") + payload = response.json() + return payload.get("guid") if isinstance(payload, dict) else None + + @dynamic_catch + async def _async_publish_data_contract_string(self, integration_daemon_guid: str, document: str) -> None: + """Send an Open Data Contract Standard (ODCS) data contract, as YAML or JSON text, to an integration + daemon, which passes it on to its Bitol listeners. Async version.""" + url = (f"{self.product_manager_command_root}/integration-daemons/" + f"{integration_daemon_guid}/data-contracts/publish-document-string") + await self._async_post_document(url, document) + + @dynamic_catch + def publish_data_contract_string(self, integration_daemon_guid: str, document: str) -> None: + """Send an ODCS data contract (YAML or JSON text) to an integration daemon for its Bitol listeners.""" + loop = asyncio.get_event_loop() + return loop.run_until_complete(self._async_publish_data_contract_string(integration_daemon_guid, document)) + + @dynamic_catch + async def _async_publish_data_product_string(self, integration_daemon_guid: str, document: str) -> None: + """Send an Open Data Product Standard (ODPS) data product, as YAML or JSON text, to an integration + daemon, which passes it on to its Bitol listeners. Async version.""" + url = (f"{self.product_manager_command_root}/integration-daemons/" + f"{integration_daemon_guid}/data-products/publish-document-string") + await self._async_post_document(url, document) + + @dynamic_catch + def publish_data_product_string(self, integration_daemon_guid: str, document: str) -> None: + """Send an ODPS data product (YAML or JSON text) to an integration daemon for its Bitol listeners.""" + loop = asyncio.get_event_loop() + return loop.run_until_complete(self._async_publish_data_product_string(integration_daemon_guid, document)) + + @dynamic_catch + async def _async_import_data_contract_string(self, document: str) -> Optional[str]: + """Catalog an ODCS data contract, supplied as YAML or JSON text, directly in open metadata as an + Agreement classified as a DataSharingAgreement. Returns the new agreement's GUID. Async version.""" + url = f"{self.product_manager_command_root}/data-contracts/import-document-string" + return await self._async_post_document(url, document) + + @dynamic_catch + def import_data_contract_string(self, document: str) -> Optional[str]: + """Catalog an ODCS data contract (YAML or JSON text) as a DataSharingAgreement. Returns its GUID.""" + loop = asyncio.get_event_loop() + return loop.run_until_complete(self._async_import_data_contract_string(document)) + + @dynamic_catch + async def _async_import_data_contract(self, data_contract: dict) -> Optional[str]: + """Catalog an ODCS data contract bean (a dict with apiVersion, kind, id, name, version, status, ...) + directly in open metadata as an Agreement classified as a DataSharingAgreement. Returns the new + agreement's GUID. Async version.""" + url = f"{self.product_manager_command_root}/data-contracts/import-document" + return await self._async_post_document(url, data_contract) + + @dynamic_catch + def import_data_contract(self, data_contract: dict) -> Optional[str]: + """Catalog an ODCS data contract bean as a DataSharingAgreement. Returns its GUID.""" + loop = asyncio.get_event_loop() + return loop.run_until_complete(self._async_import_data_contract(data_contract)) + + @dynamic_catch + async def _async_import_data_product_string(self, document: str) -> Optional[str]: + """Catalog an ODPS data product, supplied as YAML or JSON text, directly in open metadata as a + DigitalProduct. Returns the new product's GUID. Async version.""" + url = f"{self.product_manager_command_root}/data-products/import-document-string" + return await self._async_post_document(url, document) + + @dynamic_catch + def import_data_product_string(self, document: str) -> Optional[str]: + """Catalog an ODPS data product (YAML or JSON text) as a DigitalProduct. Returns its GUID.""" + loop = asyncio.get_event_loop() + return loop.run_until_complete(self._async_import_data_product_string(document)) + + @dynamic_catch + async def _async_import_data_product(self, data_product: dict) -> Optional[str]: + """Catalog an ODPS data product bean (a dict with apiVersion, kind, id, name, version, status, ...) + directly in open metadata as a DigitalProduct. Returns the new product's GUID. Async version.""" + url = f"{self.product_manager_command_root}/data-products/import-document" + return await self._async_post_document(url, data_product) + + @dynamic_catch + def import_data_product(self, data_product: dict) -> Optional[str]: + """Catalog an ODPS data product bean as a DigitalProduct. Returns its GUID.""" + loop = asyncio.get_event_loop() + return loop.run_until_complete(self._async_import_data_product(data_product)) + + @dynamic_catch + async def _async_generate_data_contract(self, agreement_guid: str) -> dict | str: + """Generate the ODCS document that describes an agreement (typically one classified as a + DataSharingAgreement). Returns the document as a dict, or NO_ELEMENTS_FOUND. Async version.""" + url = f"{self.product_manager_command_root}/agreements/{agreement_guid}/data-contract-document" + response = await self._async_make_request("GET", url) + return response.json().get("dataContract") or NO_ELEMENTS_FOUND + + @dynamic_catch + def generate_data_contract(self, agreement_guid: str) -> dict | str: + """Generate the ODCS document that describes an agreement.""" + loop = asyncio.get_event_loop() + return loop.run_until_complete(self._async_generate_data_contract(agreement_guid)) + + @dynamic_catch + async def _async_generate_data_product(self, digital_product_guid: str) -> dict | str: + """Generate the ODPS document that describes a digital product, including the contracts referenced by + its ports. Returns the document as a dict, or NO_ELEMENTS_FOUND. Async version.""" + url = f"{self.product_manager_command_root}/digital-products/{digital_product_guid}/data-product-document" + response = await self._async_make_request("GET", url) + return response.json().get("dataProduct") or NO_ELEMENTS_FOUND + + @dynamic_catch + def generate_data_product(self, digital_product_guid: str) -> dict | str: + """Generate the ODPS document that describes a digital product.""" + loop = asyncio.get_event_loop() + return loop.run_until_complete(self._async_generate_data_product(digital_product_guid)) + + @dynamic_catch + async def _async_get_governance_action_processes_by_name( + self, + name: str, + body: Optional[dict] = None, + start_from: int = 0, + page_size: int = 0, + output_format: str = "JSON", + report_spec: Optional[str | dict] = None, + **kwargs, + ) -> list | str: + """Retrieve the governance action processes with a matching qualified or display name, from the Product + Catalog view service (`/product-catalog/governance-definitions/by-name`). A digital product's + subscription types are such processes -- qualified name + `ProvisioningActionProcess::::Create Subscription::` -- so this finds the process that + Dr.Egeria's `Initiate Subscription` runs. Async version. + + There is no separate Product Catalog client: the view service is the read-only side of the product + manager's, so this lives here, with its own URL. + + Parameters + ---------- + name : str + Qualified or display name to match. + body : dict, optional + A full FilterRequestBody; supersedes `name`. + start_from, page_size : int + Paging. + output_format : str, default="JSON" + One of "JSON", "DICT", "MD", "FORM", "REPORT", or "MERMAID". + report_spec : str | dict, optional + The desired output columns/fields. + + Returns + ------- + list | str + The matching governance action processes, or NO_ELEMENTS_FOUND. + """ + url = (f"{self.platform_url}/servers/{self.view_server}/api/open-metadata/" + f"product-catalog/governance-definitions/by-name") + return await self._async_get_name_request( + url, _type="GovernanceActionProcess", _gen_output=self._generate_referenceable_output, + filter_string=name, start_from=start_from, page_size=page_size, + output_format=output_format, report_spec=report_spec, body=body, **kwargs) + + @dynamic_catch + def get_governance_action_processes_by_name( + self, + name: str, + body: Optional[dict] = None, + start_from: int = 0, + page_size: int = 0, + output_format: str = "JSON", + report_spec: Optional[str | dict] = None, + **kwargs, + ) -> list | str: + """Retrieve the governance action processes with a matching qualified or display name (for example a + digital product's subscription types).""" + loop = asyncio.get_event_loop() + return loop.run_until_complete(self._async_get_governance_action_processes_by_name( + name, body=body, start_from=start_from, page_size=page_size, output_format=output_format, + report_spec=report_spec, **kwargs)) diff --git a/pyegeria/view/base_report_formats.py b/pyegeria/view/base_report_formats.py index 3593f7ec..291f6847 100644 --- a/pyegeria/view/base_report_formats.py +++ b/pyegeria/view/base_report_formats.py @@ -345,6 +345,8 @@ 'Study-Project-DrE-Basic': FormatSet(target_type='Study Project', heading='Study-Project-DrE-Basic Attributes', description='Auto-generated format for Study Project (Create, Basic).', family='Projects', formats=[Format(types=['LIST'], attributes=[Column(name='Display Name', key='display_name'), Column(name='Qualified Name', key='qualified_name'), Column(name='GUID', key='guid'), Column(name='Description', key='description'), Column(name='Legal', key='legal'), Column(name='Authors', key='authors'), Column(name='Actual Completion Date', key='actual_completion_date'), Column(name='Actual Start Date', key='actual_start_date'), Column(name='Mission', key='mission'), Column(name='Planned Completion Date', key='planned_completion_date'), Column(name='Planned Start Date', key='planned_start_date'), Column(name='Priority', key='priority'), Column(name='Project Approach', key='approach'), Column(name='Project Health', key='project_health'), Column(name='Project Identifier', key='project_identifier'), Column(name='Project Management Style', key='management_style'), Column(name='Project Phase', key='project_phase'), Column(name='Project Results Usage', key='results_usage'), Column(name='Project Scope', key='project_scope'), Column(name='Project Type', key='project_type', valid_values=['Project', 'Campaign', 'Task', 'PersonalProject', 'StudyProject', 'Experiment']), Column(name='Purposes', key='purposes'), Column(name='Sub-Projects', key='sub_projects'), Column(name='Success Criteria', key='success_criteria'), Column(name='Version Identifier', key='version_identifier'), Column(name='Content Status', key='content_status', valid_values=['DRAFT', 'PREPARED', 'PROPOSED', 'APPROVED', 'REJECTED', 'ACTIVE', 'DEPRECATED', 'OTHER']), Column(name='Project Status', key='project_status'), Column(name='Category', key='category')]), Format(types=['ALL'], attributes=[Column(name='Category', key='category'), Column(name='Description', key='description'), Column(name='Display Name', key='display_name'), Column(name='GUID', key='guid'), Column(name='Legal', key='legal'), Column(name='Qualified Name', key='qualified_name'), Column(name='URL', key='url'), Column(name='Version Identifier', key='version_identifier'), Column(name='Authors', key='authors'), Column(name='Content Status', key='content_status', valid_values=['DRAFT', 'PREPARED', 'PROPOSED', 'APPROVED', 'REJECTED', 'ACTIVE', 'DEPRECATED', 'OTHER']), Column(name='Actual Completion Date', key='actual_completion_date'), Column(name='Actual Start Date', key='actual_start_date'), Column(name='Mission', key='mission'), Column(name='Planned Completion Date', key='planned_completion_date'), Column(name='Planned Start Date', key='planned_start_date'), Column(name='Priority', key='priority'), Column(name='Project Approach', key='approach'), Column(name='Project Health', key='project_health'), Column(name='Project Identifier', key='project_identifier'), Column(name='Project Management Style', key='management_style'), Column(name='Project Phase', key='project_phase'), Column(name='Project Results Usage', key='results_usage'), Column(name='Project Scope', key='project_scope'), Column(name='Project Status', key='project_status'), Column(name='Project Type', key='project_type', valid_values=['Project', 'Campaign', 'Task', 'PersonalProject', 'StudyProject', 'Experiment']), Column(name='Purposes', key='purposes'), Column(name='Sub-Projects', key='sub_projects'), Column(name='Success Criteria', key='success_criteria')])], action=ActionParameter(function='ProjectManager.find_projects', required_params=['search_string'], optional_params=['sequencing_order', 'sequencing_property', 'page_size', 'start_from', 'starts_with', 'ends_with', 'ignore_case', 'classification_names', 'metadata_element_subtypes', 'metadata_element_type'], spec_params={'include_only_classified_elements': ['StudyProject']})), 'Subject-Area-DrE-Advanced': FormatSet(target_type='Subject Area', heading='Subject-Area-DrE-Advanced Attributes', description='Auto-generated format for Subject Area (Create, Advanced).', family='Collections', formats=[Format(types=['LIST'], attributes=[Column(name='Display Name', key='display_name'), Column(name='Qualified Name', key='qualified_name'), Column(name='GUID', key='guid'), Column(name='Effective From', key='effective_from'), Column(name='Effective Time', key='effective_time'), Column(name='Effective To', key='effective_to'), Column(name='Glossary Term', key='glossary_term'), Column(name='Is Own Anchor', key='is_own_anchor'), Column(name='Zone Membership', key='zone_membership'), Column(name='Additional Properties', key='additional_properties'), Column(name='Confidence Classification', key='confidence_classification', valid_values=['UNCLASSIFIED', 'AD_HOC', 'TRANSACTIONAL', 'AUTHORITATIVE', 'DERIVED', 'OBSOLETE', 'OTHER']), Column(name='Confidentiality Classification', key='confidentiality_classification', valid_values=['UNCLASSIFIED', 'INTERNAL', 'CONFIDENTIAL', 'SENSITIVE', 'RESTRICTED', 'OTHER']), Column(name='Criticality Classification', key='criticality_classification', valid_values=['UNCLASSIFIED', 'MARGINAL', 'IMPORTANT', 'CRITICAL', 'CATASTROPHIC', 'OTHER']), Column(name='Description', key='description'), Column(name='Identifier', key='identifier'), Column(name='Impact Classification', key='impact_classification', valid_values=['UNCLASSIFIED', 'LOW', 'MEDIUM', 'HIGH', 'OTHER']), Column(name='Legal', key='legal'), Column(name='Policy Management Point', key='policy_management_point'), Column(name='Retention Classification', key='retention_classification', valid_values=['UNCLASSIFIED', 'TEMPORARY', 'PROJECT_LIFETIME', 'TEAM_LIFETIME', 'CONTRACT_LIFETIME', 'REGULATED_LIFETIME', 'TIMEBOXED_LIFETIME', 'OTHER']), Column(name='Security Tags', key='security_tags'), Column(name='Authors', key='authors'), Column(name='Purpose', key='purpose'), Column(name='Classifications', key='classifications'), Column(name='Anchor Scope Name', key='anchor_scope_guid'), Column(name='Supplementary Properties', key='supplementary_properties'), Column(name='Version Identifier', key='version_identifier'), Column(name='Status', key='status'), Column(name='Content Status', key='content_status', valid_values=['DRAFT', 'PREPARED', 'PROPOSED', 'APPROVED', 'REJECTED', 'ACTIVE', 'DEPRECATED', 'OTHER']), Column(name='User Defined Content Status', key='user_defined_content_status'), Column(name='User Defined Status', key='user_defined_status'), Column(name='Category', key='category')]), Format(types=['ALL'], attributes=[Column(name='Effective From', key='effective_from'), Column(name='Effective Time', key='effective_time'), Column(name='Effective To', key='effective_to'), Column(name='Glossary Term', key='glossary_term'), Column(name='Is Own Anchor', key='is_own_anchor'), Column(name='Status', key='status'), Column(name='Zone Membership', key='zone_membership'), Column(name='Additional Properties', key='additional_properties'), Column(name='Category', key='category'), Column(name='Confidence Classification', key='confidence_classification', valid_values=['UNCLASSIFIED', 'AD_HOC', 'TRANSACTIONAL', 'AUTHORITATIVE', 'DERIVED', 'OBSOLETE', 'OTHER']), Column(name='Confidentiality Classification', key='confidentiality_classification', valid_values=['UNCLASSIFIED', 'INTERNAL', 'CONFIDENTIAL', 'SENSITIVE', 'RESTRICTED', 'OTHER']), Column(name='Criticality Classification', key='criticality_classification', valid_values=['UNCLASSIFIED', 'MARGINAL', 'IMPORTANT', 'CRITICAL', 'CATASTROPHIC', 'OTHER']), Column(name='Description', key='description'), Column(name='Display Name', key='display_name'), Column(name='GUID', key='guid'), Column(name='Identifier', key='identifier'), Column(name='Impact Classification', key='impact_classification', valid_values=['UNCLASSIFIED', 'LOW', 'MEDIUM', 'HIGH', 'OTHER']), Column(name='Legal', key='legal'), Column(name='Policy Management Point', key='policy_management_point'), Column(name='Qualified Name', key='qualified_name'), Column(name='Retention Classification', key='retention_classification', valid_values=['UNCLASSIFIED', 'TEMPORARY', 'PROJECT_LIFETIME', 'TEAM_LIFETIME', 'CONTRACT_LIFETIME', 'REGULATED_LIFETIME', 'TIMEBOXED_LIFETIME', 'OTHER']), Column(name='Security Tags', key='security_tags'), Column(name='URL', key='url'), Column(name='Version Identifier', key='version_identifier'), Column(name='Authors', key='authors'), Column(name='Content Status', key='content_status', valid_values=['DRAFT', 'PREPARED', 'PROPOSED', 'APPROVED', 'REJECTED', 'ACTIVE', 'DEPRECATED', 'OTHER']), Column(name='User Defined Content Status', key='user_defined_content_status'), Column(name='Purpose', key='purpose'), Column(name='User Defined Status', key='user_defined_status'), Column(name='Classifications', key='classifications'), Column(name='Anchor Scope Name', key='anchor_scope_guid'), Column(name='Supplementary Properties', key='supplementary_properties')])], action=ActionParameter(function='CollectionManager.find_collections', required_params=['search_string'], optional_params=['sequencing_order', 'sequencing_property', 'page_size', 'start_from', 'starts_with', 'ends_with', 'ignore_case', 'classification_names', 'metadata_element_subtypes', 'metadata_element_type'], spec_params={'metadata_element_type': 'SubjectArea'})), 'Subject-Area-DrE-Basic': FormatSet(target_type='Subject Area', heading='Subject-Area-DrE-Basic Attributes', description='Auto-generated format for Subject Area (Create, Basic).', family='Collections', formats=[Format(types=['LIST'], attributes=[Column(name='Display Name', key='display_name'), Column(name='Qualified Name', key='qualified_name'), Column(name='GUID', key='guid'), Column(name='Description', key='description'), Column(name='Legal', key='legal'), Column(name='Authors', key='authors'), Column(name='Purpose', key='purpose'), Column(name='Version Identifier', key='version_identifier'), Column(name='Content Status', key='content_status', valid_values=['DRAFT', 'PREPARED', 'PROPOSED', 'APPROVED', 'REJECTED', 'ACTIVE', 'DEPRECATED', 'OTHER']), Column(name='Category', key='category')]), Format(types=['ALL'], attributes=[Column(name='Category', key='category'), Column(name='Description', key='description'), Column(name='Display Name', key='display_name'), Column(name='GUID', key='guid'), Column(name='Legal', key='legal'), Column(name='Qualified Name', key='qualified_name'), Column(name='URL', key='url'), Column(name='Version Identifier', key='version_identifier'), Column(name='Authors', key='authors'), Column(name='Content Status', key='content_status', valid_values=['DRAFT', 'PREPARED', 'PROPOSED', 'APPROVED', 'REJECTED', 'ACTIVE', 'DEPRECATED', 'OTHER']), Column(name='Purpose', key='purpose')])], action=ActionParameter(function='CollectionManager.find_collections', required_params=['search_string'], optional_params=['sequencing_order', 'sequencing_property', 'page_size', 'start_from', 'starts_with', 'ends_with', 'ignore_case', 'classification_names', 'metadata_element_subtypes', 'metadata_element_type'], spec_params={'metadata_element_type': 'SubjectArea'})), + 'Subscription-Type-DrE-Advanced': FormatSet(target_type='Subscription Type', heading='Subscription-Type-DrE-Advanced Attributes', description='Auto-generated format for Subscription Type (Create, Advanced).', family='Digital Product Manager', formats=[Format(types=['LIST'], attributes=[Column(name='Display Name', key='display_name'), Column(name='Effective From', key='effective_from'), Column(name='Effective Time', key='effective_time'), Column(name='Effective To', key='effective_to'), Column(name='Digital Product', key='digital_product'), Column(name='Subscription Kind', key='subscription_kind', valid_values=['ONE_TIME', 'PERIODIC', 'ONGOING_UPDATE']), Column(name='Description', key='description'), Column(name='Identifier', key='identifier'), Column(name='Subscription Notification Interval', key='subscription_notification_interval'), Column(name='Monitored Resources', key='monitored_resources'), Column(name='Subscription Manager', key='subscription_manager'), Column(name='Subscription License Type', key='subscription_license_type'), Column(name='Subscription Service Level Objective', key='subscription_service_level_objective')]), Format(types=['ALL'], attributes=[Column(name='Effective From', key='effective_from'), Column(name='Effective Time', key='effective_time'), Column(name='Effective To', key='effective_to'), Column(name='Digital Product', key='digital_product'), Column(name='Subscription Kind', key='subscription_kind', valid_values=['ONE_TIME', 'PERIODIC', 'ONGOING_UPDATE']), Column(name='Display Name', key='display_name'), Column(name='Description', key='description'), Column(name='Identifier', key='identifier'), Column(name='Subscription Notification Interval', key='subscription_notification_interval'), Column(name='Monitored Resources', key='monitored_resources'), Column(name='Subscription Manager', key='subscription_manager'), Column(name='Subscription License Type', key='subscription_license_type'), Column(name='Subscription Service Level Objective', key='subscription_service_level_objective')])]), + 'Subscription-Type-DrE-Basic': FormatSet(target_type='Subscription Type', heading='Subscription-Type-DrE-Basic Attributes', description='Auto-generated format for Subscription Type (Create, Basic).', family='Digital Product Manager', formats=[Format(types=['LIST'], attributes=[Column(name='Display Name', key='display_name'), Column(name='Digital Product', key='digital_product'), Column(name='Subscription Kind', key='subscription_kind', valid_values=['ONE_TIME', 'PERIODIC', 'ONGOING_UPDATE']), Column(name='Description', key='description'), Column(name='Subscription Notification Interval', key='subscription_notification_interval'), Column(name='Monitored Resources', key='monitored_resources')]), Format(types=['ALL'], attributes=[Column(name='Digital Product', key='digital_product'), Column(name='Subscription Kind', key='subscription_kind', valid_values=['ONE_TIME', 'PERIODIC', 'ONGOING_UPDATE']), Column(name='Display Name', key='display_name'), Column(name='Description', key='description'), Column(name='Subscription Notification Interval', key='subscription_notification_interval'), Column(name='Monitored Resources', key='monitored_resources')])]), 'Task-DrE-Advanced': FormatSet(target_type='Task', heading='Task-DrE-Advanced Attributes', description='Auto-generated format for Task (Create, Advanced).', family='Projects', formats=[Format(types=['LIST'], attributes=[Column(name='Display Name', key='display_name'), Column(name='Qualified Name', key='qualified_name'), Column(name='GUID', key='guid'), Column(name='Effective From', key='effective_from'), Column(name='Effective Time', key='effective_time'), Column(name='Effective To', key='effective_to'), Column(name='Glossary Term', key='glossary_term'), Column(name='Is Own Anchor', key='is_own_anchor'), Column(name='Zone Membership', key='zone_membership'), Column(name='Additional Properties', key='additional_properties'), Column(name='Confidence Classification', key='confidence_classification', valid_values=['UNCLASSIFIED', 'AD_HOC', 'TRANSACTIONAL', 'AUTHORITATIVE', 'DERIVED', 'OBSOLETE', 'OTHER']), Column(name='Confidentiality Classification', key='confidentiality_classification', valid_values=['UNCLASSIFIED', 'INTERNAL', 'CONFIDENTIAL', 'SENSITIVE', 'RESTRICTED', 'OTHER']), Column(name='Criticality Classification', key='criticality_classification', valid_values=['UNCLASSIFIED', 'MARGINAL', 'IMPORTANT', 'CRITICAL', 'CATASTROPHIC', 'OTHER']), Column(name='Description', key='description'), Column(name='Identifier', key='identifier'), Column(name='Impact Classification', key='impact_classification', valid_values=['UNCLASSIFIED', 'LOW', 'MEDIUM', 'HIGH', 'OTHER']), Column(name='Legal', key='legal'), Column(name='Policy Management Point', key='policy_management_point'), Column(name='Retention Classification', key='retention_classification', valid_values=['UNCLASSIFIED', 'TEMPORARY', 'PROJECT_LIFETIME', 'TEAM_LIFETIME', 'CONTRACT_LIFETIME', 'REGULATED_LIFETIME', 'TIMEBOXED_LIFETIME', 'OTHER']), Column(name='Security Tags', key='security_tags'), Column(name='Authors', key='authors'), Column(name='Actual Completion Date', key='actual_completion_date'), Column(name='Actual Start Date', key='actual_start_date'), Column(name='Mission', key='mission'), Column(name='Planned Completion Date', key='planned_completion_date'), Column(name='Planned Start Date', key='planned_start_date'), Column(name='Priority', key='priority'), Column(name='Project Approach', key='approach'), Column(name='Project Health', key='project_health'), Column(name='Project Identifier', key='project_identifier'), Column(name='Project Management Style', key='management_style'), Column(name='Project Phase', key='project_phase'), Column(name='Project Results Usage', key='results_usage'), Column(name='Project Scope', key='project_scope'), Column(name='Project Type', key='project_type', valid_values=['Project', 'Campaign', 'Task', 'PersonalProject', 'StudyProject', 'Experiment']), Column(name='Purposes', key='purposes'), Column(name='Sub-Projects', key='sub_projects'), Column(name='Success Criteria', key='success_criteria'), Column(name='Classifications', key='classifications'), Column(name='Anchor Scope Name', key='anchor_scope_guid'), Column(name='Supplementary Properties', key='supplementary_properties'), Column(name='Version Identifier', key='version_identifier'), Column(name='Status', key='status'), Column(name='Content Status', key='content_status', valid_values=['DRAFT', 'PREPARED', 'PROPOSED', 'APPROVED', 'REJECTED', 'ACTIVE', 'DEPRECATED', 'OTHER']), Column(name='User Defined Content Status', key='user_defined_content_status'), Column(name='Project Status', key='project_status'), Column(name='User Defined Status', key='user_defined_status'), Column(name='Category', key='category')]), Format(types=['ALL'], attributes=[Column(name='Effective From', key='effective_from'), Column(name='Effective Time', key='effective_time'), Column(name='Effective To', key='effective_to'), Column(name='Glossary Term', key='glossary_term'), Column(name='Is Own Anchor', key='is_own_anchor'), Column(name='Status', key='status'), Column(name='Zone Membership', key='zone_membership'), Column(name='Additional Properties', key='additional_properties'), Column(name='Category', key='category'), Column(name='Confidence Classification', key='confidence_classification', valid_values=['UNCLASSIFIED', 'AD_HOC', 'TRANSACTIONAL', 'AUTHORITATIVE', 'DERIVED', 'OBSOLETE', 'OTHER']), Column(name='Confidentiality Classification', key='confidentiality_classification', valid_values=['UNCLASSIFIED', 'INTERNAL', 'CONFIDENTIAL', 'SENSITIVE', 'RESTRICTED', 'OTHER']), Column(name='Criticality Classification', key='criticality_classification', valid_values=['UNCLASSIFIED', 'MARGINAL', 'IMPORTANT', 'CRITICAL', 'CATASTROPHIC', 'OTHER']), Column(name='Description', key='description'), Column(name='Display Name', key='display_name'), Column(name='GUID', key='guid'), Column(name='Identifier', key='identifier'), Column(name='Impact Classification', key='impact_classification', valid_values=['UNCLASSIFIED', 'LOW', 'MEDIUM', 'HIGH', 'OTHER']), Column(name='Legal', key='legal'), Column(name='Policy Management Point', key='policy_management_point'), Column(name='Qualified Name', key='qualified_name'), Column(name='Retention Classification', key='retention_classification', valid_values=['UNCLASSIFIED', 'TEMPORARY', 'PROJECT_LIFETIME', 'TEAM_LIFETIME', 'CONTRACT_LIFETIME', 'REGULATED_LIFETIME', 'TIMEBOXED_LIFETIME', 'OTHER']), Column(name='Security Tags', key='security_tags'), Column(name='URL', key='url'), Column(name='Version Identifier', key='version_identifier'), Column(name='Authors', key='authors'), Column(name='Content Status', key='content_status', valid_values=['DRAFT', 'PREPARED', 'PROPOSED', 'APPROVED', 'REJECTED', 'ACTIVE', 'DEPRECATED', 'OTHER']), Column(name='User Defined Content Status', key='user_defined_content_status'), Column(name='Actual Completion Date', key='actual_completion_date'), Column(name='Actual Start Date', key='actual_start_date'), Column(name='Mission', key='mission'), Column(name='Planned Completion Date', key='planned_completion_date'), Column(name='Planned Start Date', key='planned_start_date'), Column(name='Priority', key='priority'), Column(name='Project Approach', key='approach'), Column(name='Project Health', key='project_health'), Column(name='Project Identifier', key='project_identifier'), Column(name='Project Management Style', key='management_style'), Column(name='Project Phase', key='project_phase'), Column(name='Project Results Usage', key='results_usage'), Column(name='Project Scope', key='project_scope'), Column(name='Project Status', key='project_status'), Column(name='Project Type', key='project_type', valid_values=['Project', 'Campaign', 'Task', 'PersonalProject', 'StudyProject', 'Experiment']), Column(name='Purposes', key='purposes'), Column(name='Sub-Projects', key='sub_projects'), Column(name='Success Criteria', key='success_criteria'), Column(name='User Defined Status', key='user_defined_status'), Column(name='Classifications', key='classifications'), Column(name='Anchor Scope Name', key='anchor_scope_guid'), Column(name='Supplementary Properties', key='supplementary_properties')])], action=ActionParameter(function='ProjectManager.find_projects', required_params=['search_string'], optional_params=['sequencing_order', 'sequencing_property', 'page_size', 'start_from', 'starts_with', 'ends_with', 'ignore_case', 'classification_names', 'metadata_element_subtypes', 'metadata_element_type'], spec_params={'include_only_classified_elements': ['Task']})), 'Task-DrE-Basic': FormatSet(target_type='Task', heading='Task-DrE-Basic Attributes', description='Auto-generated format for Task (Create, Basic).', family='Projects', formats=[Format(types=['LIST'], attributes=[Column(name='Display Name', key='display_name'), Column(name='Qualified Name', key='qualified_name'), Column(name='GUID', key='guid'), Column(name='Description', key='description'), Column(name='Legal', key='legal'), Column(name='Authors', key='authors'), Column(name='Actual Completion Date', key='actual_completion_date'), Column(name='Actual Start Date', key='actual_start_date'), Column(name='Mission', key='mission'), Column(name='Planned Completion Date', key='planned_completion_date'), Column(name='Planned Start Date', key='planned_start_date'), Column(name='Priority', key='priority'), Column(name='Project Approach', key='approach'), Column(name='Project Health', key='project_health'), Column(name='Project Identifier', key='project_identifier'), Column(name='Project Management Style', key='management_style'), Column(name='Project Phase', key='project_phase'), Column(name='Project Results Usage', key='results_usage'), Column(name='Project Scope', key='project_scope'), Column(name='Project Type', key='project_type', valid_values=['Project', 'Campaign', 'Task', 'PersonalProject', 'StudyProject', 'Experiment']), Column(name='Purposes', key='purposes'), Column(name='Sub-Projects', key='sub_projects'), Column(name='Success Criteria', key='success_criteria'), Column(name='Version Identifier', key='version_identifier'), Column(name='Content Status', key='content_status', valid_values=['DRAFT', 'PREPARED', 'PROPOSED', 'APPROVED', 'REJECTED', 'ACTIVE', 'DEPRECATED', 'OTHER']), Column(name='Project Status', key='project_status'), Column(name='Category', key='category')]), Format(types=['ALL'], attributes=[Column(name='Category', key='category'), Column(name='Description', key='description'), Column(name='Display Name', key='display_name'), Column(name='GUID', key='guid'), Column(name='Legal', key='legal'), Column(name='Qualified Name', key='qualified_name'), Column(name='URL', key='url'), Column(name='Version Identifier', key='version_identifier'), Column(name='Authors', key='authors'), Column(name='Content Status', key='content_status', valid_values=['DRAFT', 'PREPARED', 'PROPOSED', 'APPROVED', 'REJECTED', 'ACTIVE', 'DEPRECATED', 'OTHER']), Column(name='Actual Completion Date', key='actual_completion_date'), Column(name='Actual Start Date', key='actual_start_date'), Column(name='Mission', key='mission'), Column(name='Planned Completion Date', key='planned_completion_date'), Column(name='Planned Start Date', key='planned_start_date'), Column(name='Priority', key='priority'), Column(name='Project Approach', key='approach'), Column(name='Project Health', key='project_health'), Column(name='Project Identifier', key='project_identifier'), Column(name='Project Management Style', key='management_style'), Column(name='Project Phase', key='project_phase'), Column(name='Project Results Usage', key='results_usage'), Column(name='Project Scope', key='project_scope'), Column(name='Project Status', key='project_status'), Column(name='Project Type', key='project_type', valid_values=['Project', 'Campaign', 'Task', 'PersonalProject', 'StudyProject', 'Experiment']), Column(name='Purposes', key='purposes'), Column(name='Sub-Projects', key='sub_projects'), Column(name='Success Criteria', key='success_criteria')])], action=ActionParameter(function='ProjectManager.find_projects', required_params=['search_string'], optional_params=['sequencing_order', 'sequencing_property', 'page_size', 'start_from', 'starts_with', 'ends_with', 'ignore_case', 'classification_names', 'metadata_element_subtypes', 'metadata_element_type'], spec_params={'include_only_classified_elements': ['Task']})), 'Team-DrE-Advanced': FormatSet(target_type='Team', heading='Team-DrE-Advanced Attributes', description='Auto-generated format for Team (Create, Advanced).', family='Actor Manager', formats=[Format(types=['LIST'], attributes=[Column(name='Display Name', key='display_name'), Column(name='Qualified Name', key='qualified_name'), Column(name='GUID', key='guid'), Column(name='Effective From', key='effective_from'), Column(name='Effective Time', key='effective_time'), Column(name='Effective To', key='effective_to'), Column(name='Glossary Term', key='glossary_term'), Column(name='Is Own Anchor', key='is_own_anchor'), Column(name='Zone Membership', key='zone_membership'), Column(name='Additional Properties', key='additional_properties'), Column(name='Confidence Classification', key='confidence_classification', valid_values=['UNCLASSIFIED', 'AD_HOC', 'TRANSACTIONAL', 'AUTHORITATIVE', 'DERIVED', 'OBSOLETE', 'OTHER']), Column(name='Confidentiality Classification', key='confidentiality_classification', valid_values=['UNCLASSIFIED', 'INTERNAL', 'CONFIDENTIAL', 'SENSITIVE', 'RESTRICTED', 'OTHER']), Column(name='Criticality Classification', key='criticality_classification', valid_values=['UNCLASSIFIED', 'MARGINAL', 'IMPORTANT', 'CRITICAL', 'CATASTROPHIC', 'OTHER']), Column(name='Description', key='description'), Column(name='Identifier', key='identifier'), Column(name='Impact Classification', key='impact_classification', valid_values=['UNCLASSIFIED', 'LOW', 'MEDIUM', 'HIGH', 'OTHER']), Column(name='Legal', key='legal'), Column(name='Policy Management Point', key='policy_management_point'), Column(name='Retention Classification', key='retention_classification', valid_values=['UNCLASSIFIED', 'TEMPORARY', 'PROJECT_LIFETIME', 'TEAM_LIFETIME', 'CONTRACT_LIFETIME', 'REGULATED_LIFETIME', 'TIMEBOXED_LIFETIME', 'OTHER']), Column(name='Security Tags', key='security_tags'), Column(name='Team Type', key='team_type'), Column(name='Classifications', key='classifications'), Column(name='Anchor Scope Name', key='anchor_scope_guid'), Column(name='Supplementary Properties', key='supplementary_properties'), Column(name='Version Identifier', key='version_identifier'), Column(name='Status', key='status'), Column(name='User Defined Status', key='user_defined_status'), Column(name='Category', key='category')]), Format(types=['ALL'], attributes=[Column(name='Effective From', key='effective_from'), Column(name='Effective Time', key='effective_time'), Column(name='Effective To', key='effective_to'), Column(name='Glossary Term', key='glossary_term'), Column(name='Is Own Anchor', key='is_own_anchor'), Column(name='Status', key='status'), Column(name='Zone Membership', key='zone_membership'), Column(name='Additional Properties', key='additional_properties'), Column(name='Category', key='category'), Column(name='Confidence Classification', key='confidence_classification', valid_values=['UNCLASSIFIED', 'AD_HOC', 'TRANSACTIONAL', 'AUTHORITATIVE', 'DERIVED', 'OBSOLETE', 'OTHER']), Column(name='Confidentiality Classification', key='confidentiality_classification', valid_values=['UNCLASSIFIED', 'INTERNAL', 'CONFIDENTIAL', 'SENSITIVE', 'RESTRICTED', 'OTHER']), Column(name='Criticality Classification', key='criticality_classification', valid_values=['UNCLASSIFIED', 'MARGINAL', 'IMPORTANT', 'CRITICAL', 'CATASTROPHIC', 'OTHER']), Column(name='Description', key='description'), Column(name='Display Name', key='display_name'), Column(name='GUID', key='guid'), Column(name='Identifier', key='identifier'), Column(name='Impact Classification', key='impact_classification', valid_values=['UNCLASSIFIED', 'LOW', 'MEDIUM', 'HIGH', 'OTHER']), Column(name='Legal', key='legal'), Column(name='Policy Management Point', key='policy_management_point'), Column(name='Qualified Name', key='qualified_name'), Column(name='Retention Classification', key='retention_classification', valid_values=['UNCLASSIFIED', 'TEMPORARY', 'PROJECT_LIFETIME', 'TEAM_LIFETIME', 'CONTRACT_LIFETIME', 'REGULATED_LIFETIME', 'TIMEBOXED_LIFETIME', 'OTHER']), Column(name='Security Tags', key='security_tags'), Column(name='URL', key='url'), Column(name='Version Identifier', key='version_identifier'), Column(name='Team Type', key='team_type'), Column(name='User Defined Status', key='user_defined_status'), Column(name='Classifications', key='classifications'), Column(name='Anchor Scope Name', key='anchor_scope_guid'), Column(name='Supplementary Properties', key='supplementary_properties')])], action=ActionParameter(function='ActorManager.find_actor_profiles', required_params=['search_string'], optional_params=['sequencing_order', 'sequencing_property', 'page_size', 'start_from', 'starts_with', 'ends_with', 'ignore_case', 'classification_names', 'metadata_element_subtypes', 'metadata_element_type'], spec_params={'metadata_element_type': 'Team'})), diff --git a/sample-data/templates/advanced/Digital Product Manager/Create_Subscription_Type.md b/sample-data/templates/advanced/Digital Product Manager/Create_Subscription_Type.md new file mode 100644 index 00000000..63440dc2 --- /dev/null +++ b/sample-data/templates/advanced/Digital Product Manager/Create_Subscription_Type.md @@ -0,0 +1,346 @@ +___ + +## Create Subscription Type +> Add a subscription type to a digital product: a one-time, periodic or ongoing-update way to subscribe. Creates the product's notification type and returns the governance action process that Initiate Subscription runs to provision a subscription of this type. Safe to re-run: an existing subscription type is brought into line with the request rather than duplicated. + +### Digital Product +> **Input Required**: True + +> **Attribute Type**: Reference Name + +> **Description**: The digital product the subscription type is added to (name, qualified name or GUID). + + +### Subscription Kind +> **Input Required**: True + +> **Attribute Type**: Valid Value + +> **Description**: How often subscribers are notified, and so how often the product's data is delivered: ONE_TIME (a single delivery, typically to evaluate the product), PERIODIC (every Subscription Notification Interval minutes) or ONGOING_UPDATE (when a Monitored Resource changes, but no more often than every Subscription Notification Interval minutes). + +> **Valid Values**: ONE_TIME,PERIODIC,ONGOING_UPDATE + +> **Default Value**: ONE_TIME + + +### Display Name +> **Input Required**: True + +> **Attribute Type**: Simple + +> **Description**: The common name of an element. + +> **Alternative Labels**: "Term Name" + + +### Journal Entry +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: A text entry into a journal. + + +### Description +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: A description. + + +### Category +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: A user specified category name that can be used for example, to define product types or agreement types. + +> **Alternative Labels**: Category Name + + +### Qualified Name +> **Input Required**: False + +> **Attribute Type**: QN + +> **Description**: A unique qualified name for the element. Generated using the qualified name pattern if not user specified. + + +### Subscription Notification Interval +> **Input Required**: False + +> **Attribute Type**: Simple Int + +> **Description**: Minutes between notifications -- the delivery interval for a PERIODIC subscription, the minimum interval for an ONGOING_UPDATE one. In MINUTES (unlike the Governance Officer family's 'Notification Interval', which is milliseconds). + + +### Monitored Resources +> **Input Required**: False + +> **Attribute Type**: Reference Name List + +> **Description**: For an ONGOING_UPDATE subscription: the elements whose changes trigger a notification -- typically the product's asset. + + +### Version Identifier +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: Published product version identifier. + +> **Alternative Labels**: Version + +> **Default Value**: 1.0 + + +### GUID +> **Input Required**: False + +> **Attribute Type**: GUID + +> **Description**: A system generated unique identifier. + +> **Alternative Labels**: Guid; guid + + +### URL +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: Link to supporting information + + +### Search Keywords +> **Input Required**: False + +> **Attribute Type**: Simple List + +> **Description**: Keywords to facilitate finding the element + + +### Effective From +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The beginning of when an element is viewable. + + +### Effective Time +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The time at which an element must be effective in order to be returned by the request. + + +### Effective To +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The ending time at which an element is visible. + + +### External Source GUID +> **Input Required**: False + +> **Attribute Type**: GUID + +> **Description**: The unique identifier of an external source. + + +### External Source Name +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The name of an external source + + +### For Duplicate Processing +> **Input Required**: False + +> **Attribute Type**: Bool + +> **Description**: Flag indicating if the request is to support duplicate processing. + + +### For Lineage +> **Input Required**: False + +> **Attribute Type**: Bool + +> **Description**: Flag indicating if the request is to support lineage. + + +### Request ID +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: A user provided or system generated request id for a conversation. + + +### Identifier +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: An identier + +> **Alternative Labels**: ID + + +### Subscription Manager +> **Input Required**: False + +> **Attribute Type**: Reference Name + +> **Description**: The integration connector that notifies subscribers. Defaults to the Baudot Digital Product Subscription Manager from the digital products content pack. + + +### Subscription License Type +> **Input Required**: False + +> **Attribute Type**: Reference Name + +> **Description**: The license type granted to a subscriber's asset. Defaults to the first license type the product is governed by. + + +### Subscription Service Level Objective +> **Input Required**: False + +> **Attribute Type**: Reference Name + +> **Description**: The service level objective the subscription offers. Defaults to the first service level objective the product is governed by. + + +### Status +> **Input Required**: False + +> **Attribute Type**: Valid Value + +> **Description**: The status of the digital product. There is a list of valid values that this conforms to. + +> **Default Value**: ACTIVE + + +### User Defined Status +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: Only valid if Status is set to OTHER. User defined & managed status values. + + +### Classifications +> **Input Required**: false + +> **Attribute Type**: Named DICT + +> **Description**: Optionally specify the initial classifications for a collection. Multiple classifications can be specified. + +> **Alternative Labels**: classification + +> | Parameter Name | Parameter Value | +> |---|---| +> | example_key | example_value | + + +### Is Own Anchor +> **Input Required**: False + +> **Attribute Type**: Bool + +> **Description**: Generally True. + +> **Alternative Labels**: Own Anchor + +> **Default Value**: True + + +### Anchor ID +> **Input Required**: False + +> **Attribute Type**: Reference Name + +> **Description**: Anchor identity for the collection. Typically a qualified name but if display name is unique then it could be used (not recommended) + + +### Parent ID +> **Input Required**: False + +> **Attribute Type**: Reference Name + +> **Description**: Unique name of the parent element. + +> **Alternative Labels**: Parent; + + +### Parent Relationship Type Name +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The kind of the relationship to the parent element. + + +### Anchor Scope Name +> **Input Required**: False + +> **Attribute Type**: Reference Name + +> **Description**: Optional qualified name of an anchor scope. + + +### Parent at End1 +> **Input Required**: False + +> **Attribute Type**: Bool + +> **Description**: Is the parent at end1 of the relationship? + +> **Default Value**: True + + +### Merge Update +> **Input Required**: False + +> **Attribute Type**: Bool + +> **Description**: If True, only those attributes specified in the update will be updated; If False, any attributes not provided during the update will be set to None. + +> **Alternative Labels**: Merge + +> **Default Value**: True + + +### Additional Properties +> **Input Required**: False + +> **Attribute Type**: Dictionary + +> **Description**: Additional user defined values organized as name value pairs in a dictionary. + +> | Parameter Name | Parameter Value | +> |---|---| +> | example_key | example_value | + + +### Supplementary Properties +> **Input Required**: False + +> **Attribute Type**: Named DICT + +> **Description**: Provide supplementary information to the element using the structure of a glossary term + +> | Parameter Name | Parameter Value | +> |---|---| +> | example_key | example_value | + + +___ diff --git a/sample-data/templates/advanced/Digital Product Manager/Import_Data_Contract.md b/sample-data/templates/advanced/Digital Product Manager/Import_Data_Contract.md new file mode 100644 index 00000000..05a8e2b9 --- /dev/null +++ b/sample-data/templates/advanced/Digital Product Manager/Import_Data_Contract.md @@ -0,0 +1,86 @@ +___ + +## Import Data Contract +> Catalog an Open Data Contract Standard (ODCS) data contract file directly as an Agreement classified as a DataSharingAgreement. Re-importing the same contract version updates it; a new version creates new elements. + +### Document File +> **Input Required**: True + +> **Attribute Type**: Simple + +> **Description**: Path to a file holding the document (YAML or JSON): an Open Data Contract Standard (ODCS) data contract or an Open Data Product Standard (ODPS) data product. A relative path is resolved against the directory of the markdown file being processed. + + +### Journal Entry +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: A text entry into a journal. + + +### Effective From +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The beginning of when an element is viewable. + + +### Effective Time +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The time at which an element must be effective in order to be returned by the request. + + +### Effective To +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The ending time at which an element is visible. + + +### External Source GUID +> **Input Required**: False + +> **Attribute Type**: GUID + +> **Description**: The unique identifier of an external source. + + +### External Source Name +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The name of an external source + + +### For Duplicate Processing +> **Input Required**: False + +> **Attribute Type**: Bool + +> **Description**: Flag indicating if the request is to support duplicate processing. + + +### For Lineage +> **Input Required**: False + +> **Attribute Type**: Bool + +> **Description**: Flag indicating if the request is to support lineage. + + +### Request ID +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: A user provided or system generated request id for a conversation. + + +___ diff --git a/sample-data/templates/advanced/Digital Product Manager/Import_Data_Product.md b/sample-data/templates/advanced/Digital Product Manager/Import_Data_Product.md new file mode 100644 index 00000000..daac2740 --- /dev/null +++ b/sample-data/templates/advanced/Digital Product Manager/Import_Data_Product.md @@ -0,0 +1,86 @@ +___ + +## Import Data Product +> Catalog an Open Data Product Standard (ODPS) data product file directly as a DigitalProduct. Re-importing the same product version updates it; a new version creates new elements. + +### Document File +> **Input Required**: True + +> **Attribute Type**: Simple + +> **Description**: Path to a file holding the document (YAML or JSON): an Open Data Contract Standard (ODCS) data contract or an Open Data Product Standard (ODPS) data product. A relative path is resolved against the directory of the markdown file being processed. + + +### Journal Entry +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: A text entry into a journal. + + +### Effective From +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The beginning of when an element is viewable. + + +### Effective Time +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The time at which an element must be effective in order to be returned by the request. + + +### Effective To +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The ending time at which an element is visible. + + +### External Source GUID +> **Input Required**: False + +> **Attribute Type**: GUID + +> **Description**: The unique identifier of an external source. + + +### External Source Name +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The name of an external source + + +### For Duplicate Processing +> **Input Required**: False + +> **Attribute Type**: Bool + +> **Description**: Flag indicating if the request is to support duplicate processing. + + +### For Lineage +> **Input Required**: False + +> **Attribute Type**: Bool + +> **Description**: Flag indicating if the request is to support lineage. + + +### Request ID +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: A user provided or system generated request id for a conversation. + + +___ diff --git a/sample-data/templates/advanced/Digital Product Manager/Publish_Data_Contract.md b/sample-data/templates/advanced/Digital Product Manager/Publish_Data_Contract.md new file mode 100644 index 00000000..2ecadd66 --- /dev/null +++ b/sample-data/templates/advanced/Digital Product Manager/Publish_Data_Contract.md @@ -0,0 +1,94 @@ +___ + +## Publish Data Contract +> Send an ODCS data contract file to an integration daemon, which passes it on to the integration connectors that registered a Bitol listener. + +### Integration Daemon +> **Input Required**: True + +> **Attribute Type**: Reference Name + +> **Description**: The integration daemon (a SoftwareServer) whose Bitol listeners receive the document. + + +### Document File +> **Input Required**: True + +> **Attribute Type**: Simple + +> **Description**: Path to a file holding the document (YAML or JSON): an Open Data Contract Standard (ODCS) data contract or an Open Data Product Standard (ODPS) data product. A relative path is resolved against the directory of the markdown file being processed. + + +### Journal Entry +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: A text entry into a journal. + + +### Effective From +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The beginning of when an element is viewable. + + +### Effective Time +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The time at which an element must be effective in order to be returned by the request. + + +### Effective To +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The ending time at which an element is visible. + + +### External Source GUID +> **Input Required**: False + +> **Attribute Type**: GUID + +> **Description**: The unique identifier of an external source. + + +### External Source Name +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The name of an external source + + +### For Duplicate Processing +> **Input Required**: False + +> **Attribute Type**: Bool + +> **Description**: Flag indicating if the request is to support duplicate processing. + + +### For Lineage +> **Input Required**: False + +> **Attribute Type**: Bool + +> **Description**: Flag indicating if the request is to support lineage. + + +### Request ID +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: A user provided or system generated request id for a conversation. + + +___ diff --git a/sample-data/templates/advanced/Digital Product Manager/Publish_Data_Product.md b/sample-data/templates/advanced/Digital Product Manager/Publish_Data_Product.md new file mode 100644 index 00000000..af21d342 --- /dev/null +++ b/sample-data/templates/advanced/Digital Product Manager/Publish_Data_Product.md @@ -0,0 +1,94 @@ +___ + +## Publish Data Product +> Send an ODPS data product file to an integration daemon, which passes it on to the integration connectors that registered a Bitol listener. + +### Integration Daemon +> **Input Required**: True + +> **Attribute Type**: Reference Name + +> **Description**: The integration daemon (a SoftwareServer) whose Bitol listeners receive the document. + + +### Document File +> **Input Required**: True + +> **Attribute Type**: Simple + +> **Description**: Path to a file holding the document (YAML or JSON): an Open Data Contract Standard (ODCS) data contract or an Open Data Product Standard (ODPS) data product. A relative path is resolved against the directory of the markdown file being processed. + + +### Journal Entry +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: A text entry into a journal. + + +### Effective From +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The beginning of when an element is viewable. + + +### Effective Time +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The time at which an element must be effective in order to be returned by the request. + + +### Effective To +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The ending time at which an element is visible. + + +### External Source GUID +> **Input Required**: False + +> **Attribute Type**: GUID + +> **Description**: The unique identifier of an external source. + + +### External Source Name +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: The name of an external source + + +### For Duplicate Processing +> **Input Required**: False + +> **Attribute Type**: Bool + +> **Description**: Flag indicating if the request is to support duplicate processing. + + +### For Lineage +> **Input Required**: False + +> **Attribute Type**: Bool + +> **Description**: Flag indicating if the request is to support lineage. + + +### Request ID +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: A user provided or system generated request id for a conversation. + + +___ diff --git a/sample-data/templates/basic/Digital Product Manager/Create_Subscription_Type.md b/sample-data/templates/basic/Digital Product Manager/Create_Subscription_Type.md new file mode 100644 index 00000000..83b4f08c --- /dev/null +++ b/sample-data/templates/basic/Digital Product Manager/Create_Subscription_Type.md @@ -0,0 +1,124 @@ +___ + +## Create Subscription Type +> Add a subscription type to a digital product: a one-time, periodic or ongoing-update way to subscribe. Creates the product's notification type and returns the governance action process that Initiate Subscription runs to provision a subscription of this type. Safe to re-run: an existing subscription type is brought into line with the request rather than duplicated. + +### Digital Product +> **Input Required**: True + +> **Attribute Type**: Reference Name + +> **Description**: The digital product the subscription type is added to (name, qualified name or GUID). + + +### Subscription Kind +> **Input Required**: True + +> **Attribute Type**: Valid Value + +> **Description**: How often subscribers are notified, and so how often the product's data is delivered: ONE_TIME (a single delivery, typically to evaluate the product), PERIODIC (every Subscription Notification Interval minutes) or ONGOING_UPDATE (when a Monitored Resource changes, but no more often than every Subscription Notification Interval minutes). + +> **Valid Values**: ONE_TIME,PERIODIC,ONGOING_UPDATE + +> **Default Value**: ONE_TIME + + +### Display Name +> **Input Required**: True + +> **Attribute Type**: Simple + +> **Description**: The common name of an element. + +> **Alternative Labels**: "Term Name" + + +### Journal Entry +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: A text entry into a journal. + + +### Description +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: A description. + + +### Category +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: A user specified category name that can be used for example, to define product types or agreement types. + +> **Alternative Labels**: Category Name + + +### Qualified Name +> **Input Required**: False + +> **Attribute Type**: QN + +> **Description**: A unique qualified name for the element. Generated using the qualified name pattern if not user specified. + + +### Subscription Notification Interval +> **Input Required**: False + +> **Attribute Type**: Simple Int + +> **Description**: Minutes between notifications -- the delivery interval for a PERIODIC subscription, the minimum interval for an ONGOING_UPDATE one. In MINUTES (unlike the Governance Officer family's 'Notification Interval', which is milliseconds). + + +### Monitored Resources +> **Input Required**: False + +> **Attribute Type**: Reference Name List + +> **Description**: For an ONGOING_UPDATE subscription: the elements whose changes trigger a notification -- typically the product's asset. + + +### Version Identifier +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: Published product version identifier. + +> **Alternative Labels**: Version + +> **Default Value**: 1.0 + + +### GUID +> **Input Required**: False + +> **Attribute Type**: GUID + +> **Description**: A system generated unique identifier. + +> **Alternative Labels**: Guid; guid + + +### URL +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: Link to supporting information + + +### Search Keywords +> **Input Required**: False + +> **Attribute Type**: Simple List + +> **Description**: Keywords to facilitate finding the element + + +___ diff --git a/sample-data/templates/basic/Digital Product Manager/Import_Data_Contract.md b/sample-data/templates/basic/Digital Product Manager/Import_Data_Contract.md new file mode 100644 index 00000000..516d5a5e --- /dev/null +++ b/sample-data/templates/basic/Digital Product Manager/Import_Data_Contract.md @@ -0,0 +1,22 @@ +___ + +## Import Data Contract +> Catalog an Open Data Contract Standard (ODCS) data contract file directly as an Agreement classified as a DataSharingAgreement. Re-importing the same contract version updates it; a new version creates new elements. + +### Document File +> **Input Required**: True + +> **Attribute Type**: Simple + +> **Description**: Path to a file holding the document (YAML or JSON): an Open Data Contract Standard (ODCS) data contract or an Open Data Product Standard (ODPS) data product. A relative path is resolved against the directory of the markdown file being processed. + + +### Journal Entry +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: A text entry into a journal. + + +___ diff --git a/sample-data/templates/basic/Digital Product Manager/Import_Data_Product.md b/sample-data/templates/basic/Digital Product Manager/Import_Data_Product.md new file mode 100644 index 00000000..1adf3269 --- /dev/null +++ b/sample-data/templates/basic/Digital Product Manager/Import_Data_Product.md @@ -0,0 +1,22 @@ +___ + +## Import Data Product +> Catalog an Open Data Product Standard (ODPS) data product file directly as a DigitalProduct. Re-importing the same product version updates it; a new version creates new elements. + +### Document File +> **Input Required**: True + +> **Attribute Type**: Simple + +> **Description**: Path to a file holding the document (YAML or JSON): an Open Data Contract Standard (ODCS) data contract or an Open Data Product Standard (ODPS) data product. A relative path is resolved against the directory of the markdown file being processed. + + +### Journal Entry +> **Input Required**: False + +> **Attribute Type**: Simple + +> **Description**: A text entry into a journal. + + +___ diff --git a/tests/micro-tests/test_product_manager_commands.py b/tests/micro-tests/test_product_manager_commands.py new file mode 100644 index 00000000..ca0f9f29 --- /dev/null +++ b/tests/micro-tests/test_product_manager_commands.py @@ -0,0 +1,214 @@ +""" +Digital Products family commands (md_processing/v2/product_manager.py) through the real +dispatcher with a fake client: Create Subscription Type, Import / Publish Data Contract and +Data Product -- plus the extractor recognising the new Import / Publish verbs. +""" +from typing import Any, Dict, List + +import pytest + +from pyegeria import NO_ELEMENTS_FOUND + +from md_processing.dr_egeria import setup_dispatcher +from md_processing.v2.extraction import DrECommand, UniversalExtractor + +ELEMENTS = { + "Hospital Data": "product-1", + "Hospital Asset": "asset-1", + "Hospital Feed": "asset-2", + "Bitol Daemon": "daemon-1", + "Baudot Manager": "manager-1", +} +CONTRACT = "apiVersion: v3.1.0\nkind: DataContract\nid: 6d0f\nname: Weekly measurements\nversion: 1.0.0\n" + + +class _FakeProductManager: + def __init__(self, import_guid="agreement-1"): + self.calls: List[tuple] = [] + self.import_guid = import_guid + + async def _async_create_one_time_subscription(self, guid, body=None): + self.calls.append(("one-time", guid, body)); return "gap-one-time" + + async def _async_create_periodic_subscription(self, guid, body=None): + self.calls.append(("periodic", guid, body)); return "gap-periodic" + + async def _async_create_ongoing_update_subscription(self, guid, body=None): + self.calls.append(("ongoing-update", guid, body)); return "gap-ongoing" + + async def _async_import_data_contract_string(self, document): + self.calls.append(("import-contract", document)); return self.import_guid + + async def _async_import_data_product_string(self, document): + self.calls.append(("import-product", document)); return self.import_guid + + async def _async_publish_data_contract_string(self, daemon_guid, document): + self.calls.append(("publish-contract", daemon_guid, document)) + + async def _async_publish_data_product_string(self, daemon_guid, document): + self.calls.append(("publish-product", daemon_guid, document)) + + +class _FakeClient: + def __init__(self, **kwargs): + self.product_manager = _FakeProductManager(**kwargs) + + async def __async_get_guid__(self, qualified_name=None, display_name=None, property_name="qualifiedName", + tech_type=None, **kwargs): + guid = ELEMENTS.get(qualified_name or display_name) + return guid or NO_ELEMENTS_FOUND + + +def _cmd(verb: str, object_type: str, attributes: Dict[str, Any]) -> DrECommand: + raw = f"## {verb} {object_type}\n" + "".join(f"### {k}\n{v}\n" for k, v in attributes.items()) + return DrECommand(verb=verb, object_type=object_type, attributes=attributes, raw_block=raw) + + +async def _run(command: DrECommand, context=None, **client_kwargs): + client = _FakeClient(**client_kwargs) + ctx = {"directive": "process", **(context or {})} + results = await setup_dispatcher(client).dispatch_batch([command], ctx) + return results[0], client.product_manager.calls + + +# --- Create Subscription Type ---------------------------------------------------------------- + +@pytest.mark.asyncio +async def test_one_time_subscription_type_with_defaults_left_to_egeria(): + result, calls = await _run(_cmd("Create", "Subscription Type", { + "Digital Product": "Hospital Data", "Subscription Kind": "ONE_TIME", + "Display Name": "Evaluation subscription"})) + + assert result["status"] == "success", result + [(kind, guid, body)] = calls + assert (kind, guid) == ("one-time", "product-1") + assert body == {"displayName": "Evaluation subscription"} # nothing unset is sent + assert "gap-one-time" in result["output"] + + +@pytest.mark.asyncio +async def test_periodic_subscription_type_sends_interval_in_minutes_as_int(): + result, calls = await _run(_cmd("Create", "Subscription Type", { + "Digital Product": "Hospital Data", "Subscription Kind": "periodic", # case-insensitive + "Display Name": "Daily refresh", "Identifier": "DAILY-REFRESH-SUBSCRIPTION", + "Subscription Notification Interval": "1440", "Subscription Manager": "Baudot Manager"})) + + assert result["status"] == "success", result + [(kind, guid, body)] = calls + assert kind == "periodic" + assert body["notificationInterval"] == 1440 and isinstance(body["notificationInterval"], int) + assert body["identifier"] == "DAILY-REFRESH-SUBSCRIPTION" + assert body["subscriptionManagerGUID"] == "manager-1" + + +@pytest.mark.asyncio +async def test_ongoing_update_subscription_type_sends_monitored_resource_guids(): + result, calls = await _run(_cmd("Create", "Subscription Type", { + "Digital Product": "Hospital Data", "Subscription Kind": "ONGOING_UPDATE", + "Display Name": "Ongoing", "Subscription Notification Interval": "10", + "Monitored Resources": "Hospital Asset, Hospital Feed"})) + + assert result["status"] == "success", result + [(kind, guid, body)] = calls + assert kind == "ongoing-update" + assert body["monitoredResourceGUIDs"] == ["asset-1", "asset-2"] + assert body["notificationInterval"] == 10 + + +@pytest.mark.asyncio +@pytest.mark.parametrize("attributes, message", [ + ({"Subscription Kind": "PERIODIC"}, "Subscription Notification Interval"), + ({"Subscription Kind": "ONGOING_UPDATE", "Subscription Notification Interval": "5"}, "Monitored Resources"), + ({"Subscription Kind": "HOURLY"}, "Subscription Kind"), +]) +async def test_subscription_type_rejects_incomplete_requests_before_calling_egeria(attributes, message): + result, calls = await _run(_cmd("Create", "Subscription Type", { + "Digital Product": "Hospital Data", "Display Name": "x", **attributes})) + + assert result["status"] != "success" + assert message in (result.get("message", "") + result.get("analysis", "")) + assert calls == [] + + +@pytest.mark.asyncio +async def test_subscription_type_needs_an_existing_product(): + result, calls = await _run(_cmd("Create", "Subscription Type", { + "Digital Product": "No Such Product", "Subscription Kind": "ONE_TIME", "Display Name": "x"})) + + assert result["status"] != "success" + assert calls == [] + + +# --- Import / Publish documents -------------------------------------------------------------- + +def _doc(tmp_path, name="contract.yaml", text=CONTRACT): + (tmp_path / name).write_text(text) + return {"input_path": str(tmp_path / "doc.md")} + + +@pytest.mark.asyncio +@pytest.mark.parametrize("object_type, call", [("Data Contract", "import-contract"), ("Data Product", "import-product")]) +async def test_import_reads_the_file_relative_to_the_markdown_document(tmp_path, object_type, call): + context = _doc(tmp_path) + result, calls = await _run(_cmd("Import", object_type, {"Document File": "contract.yaml"}), context) + + assert result["status"] == "success", result + assert calls == [(call, CONTRACT)] + assert "agreement-1" in result["output"] + + +@pytest.mark.asyncio +async def test_import_that_egeria_skips_is_reported_not_swallowed(tmp_path): + context = _doc(tmp_path) + result, calls = await _run(_cmd("Import", "Data Contract", {"Document File": "contract.yaml"}), context, + import_guid=None) + + assert result["status"] != "success" + assert "skipped" in result["message"] + + +@pytest.mark.asyncio +@pytest.mark.parametrize("object_type, call", [("Data Contract", "publish-contract"), ("Data Product", "publish-product")]) +async def test_publish_sends_the_document_to_the_resolved_daemon(tmp_path, object_type, call): + context = _doc(tmp_path) + result, calls = await _run(_cmd("Publish", object_type, { + "Integration Daemon": "Bitol Daemon", "Document File": "contract.yaml"}), context) + + assert result["status"] == "success", result + assert calls == [(call, "daemon-1", CONTRACT)] + + +@pytest.mark.asyncio +@pytest.mark.parametrize("verb, extra", [("Import", {}), ("Publish", {"Integration Daemon": "Bitol Daemon"})]) +async def test_missing_or_empty_document_file_fails_before_calling_egeria(tmp_path, verb, extra): + context = _doc(tmp_path, "empty.yaml", " \n") + for name in ("does-not-exist.yaml", "empty.yaml"): + result, calls = await _run(_cmd(verb, "Data Contract", {"Document File": name, **extra}), context) + assert result["status"] != "success" + assert calls == [] + + +@pytest.mark.asyncio +async def test_absolute_document_path_is_used_as_given(tmp_path): + (tmp_path / "abs.yaml").write_text(CONTRACT) + result, calls = await _run(_cmd("Import", "Data Contract", {"Document File": str(tmp_path / "abs.yaml")}), + {"input_path": "/somewhere/else/doc.md"}) + + assert result["status"] == "success", result + assert calls == [("import-contract", CONTRACT)] + + +# --- the new verbs must be recognised as commands, not prose -------------------------------- + +@pytest.mark.parametrize("heading, verb, object_type", [ + ("## Import Data Contract", "Import", "Data Contract"), + ("## Import Data Product", "Import", "Data Product"), + ("## Publish Data Contract", "Publish", "Data Contract"), + ("## Publish Data Product", "Publish", "Data Product"), + ("## Create Subscription Type", "Create", "Subscription Type"), +]) +def test_new_headings_are_extracted_as_commands(heading, verb, object_type): + text = f"\n___\n\n{heading}\n\n### Document File\nc.yaml\n\n___\n" + commands = [c for c in UniversalExtractor(text).extract_commands() if c.is_command] + + assert [(c.verb, c.object_type) for c in commands] == [(verb, object_type)] diff --git a/tests/micro-tests/test_product_manager_subscriptions_and_documents.py b/tests/micro-tests/test_product_manager_subscriptions_and_documents.py new file mode 100644 index 00000000..c202daee --- /dev/null +++ b/tests/micro-tests/test_product_manager_subscriptions_and_documents.py @@ -0,0 +1,188 @@ +""" +ProductManager: subscription types (one-time / periodic / ongoing-update) and the Open Data +Contract / Product Standard (ODCS / ODPS) import, publish and generate calls +(Egeria-api-product-manager.http). No live server: the HTTP call is captured. +""" +import json + +import httpx +import pytest + +from pyegeria.core._globals import NO_ELEMENTS_FOUND +from pyegeria.omvs.product_manager import ProductManager + +ROOT = "/api/open-metadata/product-manager" +YAML_DOC = "apiVersion: v3.1.0\nkind: DataContract\nid: 6d0f\nname: Weekly measurements\n" + + +class _Response: + def __init__(self, payload): + self._payload = payload + + def json(self): + return self._payload + + +def _client(calls: list, payload: dict, monkeypatch) -> ProductManager: + monkeypatch.setattr(ProductManager, "check_connection", lambda self: "test") + client = ProductManager("view-server", "https://localhost:1", "user", "pwd") + + async def fake_make_request(method, url, payload_=None, is_json=True, params=None, *, + timeout=None, as_text=False, **kwargs): + calls.append({"method": method, "url": url, "as_text": as_text, + "body": payload_ if isinstance(payload_, (dict, type(None))) else + (payload_ if as_text else json.loads(payload_))}) + return _Response(payload) + + client._async_make_request = fake_make_request + return client + + +@pytest.mark.asyncio +@pytest.mark.parametrize("method, kind", [ + ("_async_create_one_time_subscription", "one-time"), + ("_async_create_periodic_subscription", "periodic"), + ("_async_create_ongoing_update_subscription", "ongoing-update"), +]) +async def test_subscription_types_post_to_the_kind_and_return_the_process_guid(method, kind, monkeypatch): + calls: list = [] + client = _client(calls, {"class": "GUIDResponse", "guid": "gap-guid"}, monkeypatch) + body = {"identifier": "ID", "displayName": "Name", "notificationInterval": 10, + "monitoredResourceGUIDs": ["asset-1"]} + + guid = await getattr(client, method)("product-1", body) + + assert guid == "gap-guid" + assert calls[0]["method"] == "POST" + assert calls[0]["url"].endswith(f"{ROOT}/digital-products/product-1/subscription-types/{kind}") + assert calls[0]["body"]["class"] == "NewSubscriptionTypeRequestBody" + assert calls[0]["body"]["monitoredResourceGUIDs"] == ["asset-1"] # explicit alias, not "...Guids" + assert calls[0]["body"]["notificationInterval"] == 10 + + +@pytest.mark.asyncio +async def test_subscription_body_is_optional(monkeypatch): + calls: list = [] + client = _client(calls, {"guid": "g"}, monkeypatch) + + await client._async_create_one_time_subscription("product-1") + + assert calls[0]["body"] is None + + +@pytest.mark.asyncio +@pytest.mark.parametrize("method, tail", [ + ("_async_publish_data_contract_string", "data-contracts/publish-document-string"), + ("_async_publish_data_product_string", "data-products/publish-document-string"), +]) +async def test_publish_sends_the_document_as_text_to_the_daemon(method, tail, monkeypatch): + calls: list = [] + client = _client(calls, {"class": "VoidResponse"}, monkeypatch) + + result = await getattr(client, method)("daemon-1", YAML_DOC) + + assert result is None + assert calls[0]["url"].endswith(f"{ROOT}/integration-daemons/daemon-1/{tail}") + assert calls[0]["as_text"] is True and calls[0]["body"] == YAML_DOC + + +@pytest.mark.asyncio +@pytest.mark.parametrize("method, tail", [ + ("_async_import_data_contract_string", "data-contracts/import-document-string"), + ("_async_import_data_product_string", "data-products/import-document-string"), +]) +async def test_import_string_sends_text_and_returns_guid(method, tail, monkeypatch): + calls: list = [] + client = _client(calls, {"guid": "new-guid"}, monkeypatch) + + guid = await getattr(client, method)(YAML_DOC) + + assert guid == "new-guid" + assert calls[0]["url"].endswith(f"{ROOT}/{tail}") + assert calls[0]["as_text"] is True and calls[0]["body"] == YAML_DOC + + +@pytest.mark.asyncio +@pytest.mark.parametrize("method, tail", [ + ("_async_import_data_contract", "data-contracts/import-document"), + ("_async_import_data_product", "data-products/import-document"), +]) +async def test_import_bean_sends_json_unchanged_and_returns_guid(method, tail, monkeypatch): + calls: list = [] + client = _client(calls, {"guid": "new-guid"}, monkeypatch) + bean = {"apiVersion": "v3.1.0", "kind": "DataContract", "id": "6d0f", "name": "n", "status": "active"} + + guid = await getattr(client, method)(bean) + + assert guid == "new-guid" + assert calls[0]["url"].endswith(f"{ROOT}/{tail}") + assert calls[0]["as_text"] is False and calls[0]["body"] == bean # no class / forLineage injected + + +@pytest.mark.asyncio +async def test_import_rejects_a_wrong_document_type_before_any_request(monkeypatch): + calls: list = [] + client = _client(calls, {}, monkeypatch) + + with pytest.raises(Exception): + await client._async_import_data_contract(12345) + + assert calls == [] + + +@pytest.mark.asyncio +@pytest.mark.parametrize("method, arg, tail, key", [ + ("_async_generate_data_contract", "agr-1", "agreements/agr-1/data-contract-document", "dataContract"), + ("_async_generate_data_product", "prod-1", "digital-products/prod-1/data-product-document", "dataProduct"), +]) +async def test_generate_returns_the_document(method, arg, tail, key, monkeypatch): + calls: list = [] + client = _client(calls, {key: {"kind": "x", "id": "1"}}, monkeypatch) + + doc = await getattr(client, method)(arg) + + assert doc == {"kind": "x", "id": "1"} + assert calls[0]["method"] == "GET" and calls[0]["url"].endswith(f"{ROOT}/{tail}") + + +@pytest.mark.asyncio +async def test_generate_with_no_document_says_so_rather_than_returning_empty(monkeypatch): + client = _client([], {"class": "DataContractResponse"}, monkeypatch) + + assert await client._async_generate_data_contract("agr-1") == NO_ELEMENTS_FOUND + + +@pytest.mark.asyncio +async def test_as_text_reaches_the_http_layer_as_text_plain(monkeypatch): + """The real _async_make_request, with only httpx stubbed: a str payload sent as_text goes out as + text/plain; the default stays application/json.""" + monkeypatch.setattr(ProductManager, "check_connection", lambda self: "test") + client = ProductManager("view-server", "https://localhost:1", "user", "pwd") + seen: list = [] + + async def fake_post(endpoint, **kwargs): + seen.append(kwargs["headers"]["Content-Type"]) + return httpx.Response(200, json={"relatedHTTPCode": 200, "guid": "g"}, request=httpx.Request("POST", endpoint)) + + monkeypatch.setattr(client.session, "post", fake_post) + + await client._async_make_request("POST", "https://localhost:1/x", YAML_DOC, as_text=True) + await client._async_make_request("POST", "https://localhost:1/x", YAML_DOC) + + assert seen == ["text/plain", "application/json"] + + +@pytest.mark.asyncio +async def test_governance_action_processes_by_name_uses_the_product_catalog_url(monkeypatch): + calls: list = [] + client = _client(calls, {"elements": [{"elementHeader": {"guid": "gap-1"}}]}, monkeypatch) + + result = await client._async_get_governance_action_processes_by_name( + "ProvisioningActionProcess::Hospital Data::Create Subscription::EVAL") + + assert result == [{"elementHeader": {"guid": "gap-1"}}] + assert calls[0]["method"] == "POST" + assert calls[0]["url"].endswith("/api/open-metadata/product-catalog/governance-definitions/by-name") + assert calls[0]["body"]["filter"] == "ProvisioningActionProcess::Hospital Data::Create Subscription::EVAL" + # a filter by name, not by an element type (a wrong type name is rejected by the server) + assert "metadataElementTypeName" not in calls[0]["body"]