Allow Template upgrade properties to be set and remove properties that no longer exist.

[James Chapman (JC-wk)]

Resolves #4732 #4730

What is being addressed

Currently if you add a new property to a template there is no way to specify this property before an upgrade is ran.
The upgrade may fail due to the missing property (although the template version is still incremented)
The user then has to click update and supply the property.
Similarly when removing a property from a template and running an upgrade, the property still exists on the resource.

Todo

• Enhancement: Current template should only be retrieved once.
• To check Pipeline Template JSON Schema versions #4795 and fix issue Error sending create / update request Unevaluated properties are not allowed ('your_property_name' was unexpected) occurs when updatable is false #4728
• To allow us to merge in the Unit Test Initial UI tests #4794 first so the UI tests can be added
• Possibly handle cases such as where an enum value is removed (not the field entirely) and that value is currently chosen

How is this addressed

• Adds a form to allow the user to specify new properties prior to the upgrade
• Removes any template properties that no longer exist in the new template
• Added tests
• Updated CHANGELOG.md
• Increment API version

[github-actions]

Unit Test Results

1 055 tests   1 055 ✅  33s ⏱️
   29 suites      0 💤
    2 files        0 ❌

Results for commit 56e4d80.

♻️ This comment has been updated with latest results.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated no new comments.

Files not reviewed (1)

• ui/app/package-lock.json: Generated file

Suppressed comments (1)

api_app/db/repositories/resources.py:462

• Array-of-object paths cannot be authorized correctly here. _get_leaf_properties emits paths such as redirect_uris.name, but this walker only descends through properties (the child schema is under items.properties), and get_nested_val likewise cannot cross the persisted list. Consequently, an upgrade that adds redirect_uris.value and sends each full item—as the new UI does—rejects the unchanged existing name as non-updateable. Make the schema/current-value traversal array-aware and cover the end-to-end array upgrade case in the repository tests.

                    current = current[part].get("properties", {})

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 1 comment.

Files not reviewed (1)

• ui/app/package-lock.json: Generated file

Suppressed comments (1)

ui/app/src/utils/schemaUpgradeUtils.ts:267

• When an array item gains one field, this copies the entire array schema because arrays expose item properties under items, not direct properties. The upgrade form consequently renders existing item fields as editable, and setNestedValue later sends the full edited items. Editing an existing non-updateable sibling then makes the upgrade fail (or changes unrelated updateable data). Prune items.properties to the newly added item fields while retaining existing sibling values only in the PATCH payload.

        prunedProperties[propName] = { ...(propSchema as any) };

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated no new comments.

Files not reviewed (1)

• ui/app/package-lock.json: Generated file

Suppressed comments (4)

api_app/db/repositories/resources.py:638

• For arrays of objects, get_nested_val returns all existing leaf values as a list, so this membership check treats each submitted value as unchanged without preserving item identity or cardinality. A PATCH to a non-updateable array can therefore delete, duplicate, reorder, or recombine items as long as each leaf value appeared somewhere in the old array; _deep_dict_update then replaces the entire array. Compare the complete array value for non-updateable arrays, or retain item indices while validating leaves.

            if current_properties is not None and is_upgrade:
                has_existing, existing_val = get_nested_val(current_properties, prop_path)
                if has_existing and (existing_val == prop_val or (isinstance(existing_val, list) and prop_val in existing_val)):
                    return True

api_app/db/repositories/resources.py:424

• This collects property names from every upgrade pipeline step, including steps that patch other resources. Only the main step supplies properties for the resource being upgraded; for example, the Health Services pipeline's other step supplies rule_collections to the firewall template. If the upgraded template has a new property with the same name, this code incorrectly exempts it from required validation. Restrict this set to stepId == "main".

This issue also appears on line 635 of the same file.

            for step in pipeline[action]:
                if "properties" in step and step["properties"]:
                    for prop in step["properties"]:
                        if isinstance(prop, dict) and prop.get("name"):
                            properties.add(prop["name"])

ui/app/src/components/shared/ConfirmUpgradeResource.tsx:335

• This treats properties sent to downstream resources by any upgrade pipeline step as if the backend will populate them on the resource currently being upgraded. Only properties on the main step belong to this resource; a same-named new property is otherwise removed from the form and omitted from the PATCH, defeating the new-property upgrade flow. Filter to step.stepId === "main" before collecting names.

        if (newTemplate?.pipeline?.upgrade) {
          newTemplate.pipeline.upgrade.forEach((step: any) => {
            if (step.properties) {
              step.properties.forEach((prop: any) => {
                pipelineProps.add(prop.name);
              });
            }
          });

ui/app/src/utils/schemaUpgradeUtils.ts:229

• Nested traversal only consults currentSchema.properties. If the parent object is defined in an active allOf.then/else branch (as data_source_config is in the OHDSI schema), nextSchema becomes undefined, so a newly added required child is reported as optional. Because formHasErrors starts false and is only updated by onChange, the Upgrade button can initially remain enabled and submit a payload that the backend rejects. Resolve the next schema from the active conditional branch when it is absent from top-level properties.

    const nextSchema = currentSchema.properties ? currentSchema.properties[part] : undefined;
    currentSchema = nextSchema;
    currState = currState ? currState[part] : undefined;

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated no new comments.

Files not reviewed (1)

• ui/app/package-lock.json: Generated file

Suppressed comments (2)

api_app/db/repositories/resources.py:566

• When part is an array index, this switches to the item schema but then processes the index as though it were an item property (and always selects item 0). Thus redirect_uris.0.value checks whether "0" is required instead of "value". An item field that existed as optional in the old template and becomes required in the target is consequently rejected as non-updateable even when the upgrade supplies it.

                if isinstance(curr_schema.get("items"), dict):
                    curr_schema = curr_schema["items"]
                    curr_state = curr_state[0] if isinstance(curr_state, list) and curr_state else {}

api_app/db/repositories/resources.py:293

• Array-item schema paths omit indexes (for example, redirect_uris.value), but _get_leaf_properties emits instance paths such as redirect_uris.0.value. This direct comparison never matches when only an item field was removed, so that deleted field remains on every persisted array item and can also make validation against the target item schema fail. Normalize array instance paths to schema paths and remove the deleted field across all items.

            for path in existing_paths:
                if any(path == tp or path.startswith(tp + ".") for tp in removed_template_paths):

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 1 comment.

Files not reviewed (1)

• ui/app/package-lock.json: Generated file

Suppressed comments (1)

ui/app/src/utils/schemaUpgradeUtils.ts:167

• matchesIfCondition only handles const and enum; every other JSON Schema constraint is reduced to a presence check. For example, { properties: { count: { minimum: 1 } } } matches count: 0 here even though AJV selects the other branch. The reduced form can therefore show/require the wrong fields and send values from the wrong branch. Use AJV to evaluate the complete if schema so this logic agrees with the form validator and API.

// Utility to check if a simple JSON Schema condition matches the current state
export const matchesIfCondition = (ifSchema: any, state: any): boolean => {
  if (!ifSchema || !ifSchema.properties) return false;
  for (const [key, cond] of Object.entries(ifSchema.properties)) {
    const val = getNestedValue(state, key);
    if (cond && typeof cond === "object") {

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 2 comments.

Files not reviewed (1)

• ui/app/package-lock.json: Generated file

Suppressed comments (6)

ui/app/src/components/shared/ConfirmUpgradeResource.tsx:663

• The disabled prop runs a fairly heavy inline computation on every render (deep merge + per-key schema lookups + nested value reads), which can be noticeable for templates with many properties. Consider computing a memoized isUpgradeDisabled (e.g., via useMemo) and reusing a cached combinedState so renders remain cheap and logic is easier to debug.

              <PrimaryButton
                disabled={
                  !selectedVersion ||
                  loadingSchema ||
                  formHasErrors ||
                  (newPropertiesToFill.length > 0 &&
                    (() => {
                      const combinedState = mergePropertyValues(props.resource.properties, newPropertyValues);
                      return newPropertiesToFill.some((key) => {
                        if (!isKeyActiveInTemplate(newTemplateSchema, key, combinedState)) {
                          return false;
                        }
                        const valInState = getNestedValue(combinedState, key);
                        const valInNew = getNestedValue(newPropertyValues, key);
                        const propSchema = getSchemaProperty(newTemplateSchema, key);

                        // Check if an enum value that IS present is invalid.
                        // Do NOT block when the value is absent — the required-field check below handles that.
                        if (
                          propSchema &&
                          propSchema.enum &&
                          valInState !== undefined &&
                          valInState !== null &&
                          valInState !== "" &&
                          !propSchema.enum.includes(valInState)
                        ) {
                          return true;
                        }

                        // Check if required field is empty
                        if (isPropertyRequiredInState(newTemplateSchema, key, combinedState)) {
                          return valInNew === "" || valInNew === undefined || valInNew === null;
                        }
                        return false;
                      });
                    })())
                }

ui/app/src/utils/schemaUpgradeUtils.ts:29

• The comment claims arrays-of-objects are treated as atomic leaves because array-index traversal isn’t supported, but this file does generate indexed paths (e.g., redirect_uris.0.value) and both getNestedValue/setNestedValue can traverse numeric indices. Updating this comment would prevent future readers from making incorrect assumptions when evolving the upgrade logic.

      // Include the object container itself so required-object detection works, then recurse into children.
      // Arrays-of-objects are treated as atomic leaves (getNestedValue/setNestedValue don't support array-index traversal).
      keys.push(prefix + key);

ui/app/tsconfig.json:17

• ignoreDeprecations suppresses deprecation reporting globally, which can hide future-breaking changes in dependencies/TS libs. If this is to unblock a specific deprecation, consider scoping the fix (upgrading the dependency or addressing the usage) or documenting the specific deprecations being suppressed and a plan to remove this flag.

    "ignoreDeprecations": "5.0",

ui/app/src/utils/schemaUpgradeUtils.ts:185

• Several new core utilities (conditional required evaluation, schema pruning, conditional block extraction) are central to upgrade correctness but don’t appear to have targeted unit tests in this PR. Adding focused tests for isPropertyRequiredInState (allOf then/else required), pruneSchemaNode (required pruning and nested object behavior), and extractConditionalBlocks (including keys referenced only via required) would reduce regressions as templates become more complex.

export const isPropertyRequiredInState = (templateSchema: any, path: string, state: any): boolean => {

ui/app/src/utils/schemaUpgradeUtils.ts:247

• Several new core utilities (conditional required evaluation, schema pruning, conditional block extraction) are central to upgrade correctness but don’t appear to have targeted unit tests in this PR. Adding focused tests for isPropertyRequiredInState (allOf then/else required), pruneSchemaNode (required pruning and nested object behavior), and extractConditionalBlocks (including keys referenced only via required) would reduce regressions as templates become more complex.

export const pruneSchemaNode = (schemaNode: any, activeKeys: string[]): any => {

ui/app/src/utils/schemaUpgradeUtils.ts:340

• Several new core utilities (conditional required evaluation, schema pruning, conditional block extraction) are central to upgrade correctness but don’t appear to have targeted unit tests in this PR. Adding focused tests for isPropertyRequiredInState (allOf then/else required), pruneSchemaNode (required pruning and nested object behavior), and extractConditionalBlocks (including keys referenced only via required) would reduce regressions as templates become more complex.

export const extractConditionalBlocks = (schema: any, newKeys: string[]) => {

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 1 comment.

Files not reviewed (1)

• ui/app/package-lock.json: Generated file

Suppressed comments (3)

ui/app/src/components/shared/ConfirmUpgradeResource.tsx:138

• For required properties that are missing in formData, this sets an empty string as a placeholder. That is incorrect for non-string schemas (e.g., required boolean/number/object/array without a default) and can lead to sending type-invalid PATCH payloads that the backend will reject. Instead, avoid synthesizing a value of the wrong type: either (a) omit the key entirely and rely on form validation + disabling the Upgrade button, or (b) derive an appropriate placeholder from the property schema type (or default) before setting.

  const extractNewPropertyValues = (formData: any, templateSchema: any, keys: string[]) => {
    const updatedNewVals: Record<string, any> = {};
    keys.forEach((key) => {
      if (isKeyActiveInTemplate(templateSchema, key, formData)) {
        const val = getNestedValue(formData, key);
        if (val !== undefined) {
          setNestedValue(updatedNewVals, key, val, formData);
        } else if (isPropertyRequiredInState(templateSchema, key, formData)) {
          setNestedValue(updatedNewVals, key, "", formData);
        }
      }
    });
    return updatedNewVals;
  };

ui/app/src/components/shared/ConfirmUpgradeResource.tsx:666

• The disabled prop runs a fairly heavy computation inline on every render (merges state, walks conditionals, repeatedly resolves schema properties). This can become noticeable for larger schemas/forms. Move this logic into a useMemo (e.g., isUpgradeDisabled) keyed on the relevant dependencies (selectedVersion, loadingSchema, formHasErrors, newPropertiesToFill, newTemplateSchema, props.resource.properties, newPropertyValues).

              <PrimaryButton
                disabled={
                  !selectedVersion ||
                  loadingSchema ||
                  formHasErrors ||
                  (newPropertiesToFill.length > 0 &&
                    (() => {
                      const combinedState = mergePropertyValues(props.resource.properties, newPropertyValues);
                      return newPropertiesToFill.some((key) => {
                        if (!isKeyActiveInTemplate(newTemplateSchema, key, combinedState)) {
                          return false;
                        }
                        const valInState = getNestedValue(combinedState, key);
                        const valInNew = getNestedValue(newPropertyValues, key);
                        const propSchema = getSchemaProperty(newTemplateSchema, key);

                        // Check if an enum value that IS present is invalid.
                        // Do NOT block when the value is absent — the required-field check below handles that.
                        if (
                          propSchema &&
                          propSchema.enum &&
                          valInState !== undefined &&
                          valInState !== null &&
                          valInState !== "" &&
                          !propSchema.enum.includes(valInState)
                        ) {
                          return true;
                        }

                        // Check if required field is empty
                        if (isPropertyRequiredInState(newTemplateSchema, key, combinedState)) {
                          return valInNew === "" || valInNew === undefined || valInNew === null;
                        }
                        return false;
                      });
                    })())
                }
                text="Upgrade"
                onClick={() => upgradeCall()}
              />

api_app/db/repositories/resources.py:350

• This cleanup only removes top-level properties exclusive to an inactive allOf branch. If a branch introduces nested fields under an object that exists in both branches (or is top-level), stale nested keys can remain and still violate unevaluatedProperties: false or nested additionalProperties constraints. Consider collecting dotted paths for branch-defined nested properties (similar to _get_all_property_keys_from_template) and removing them via _remove_property_by_path, rather than only popping the first segment.

            for condition in enriched_target_template.get("allOf", []):
                if not isinstance(condition, dict) or "if" not in condition:
                    continue
                matches_if = self._matches_if_condition(condition["if"], post_patch_props)
                active_branch = condition.get("then", {}) if matches_if else condition.get("else", {})
                inactive_branch = condition.get("else", {}) if matches_if else condition.get("then", {})

                inactive_props = set((inactive_branch or {}).get("properties", {}).keys())
                active_props = set((active_branch or {}).get("properties", {}).keys())
                top_level_props = set(enriched_target_template.get("properties", {}).keys())

                # Only remove props that are *exclusive* to the inactive branch
                exclusive_inactive = inactive_props - active_props - top_level_props
                for prop_key in exclusive_inactive:
                    resource.properties.pop(prop_key, None)

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 1 comment.

Files not reviewed (1)

• ui/app/package-lock.json: Generated file

Suppressed comments (3)

api_app/db/repositories/resources.py:639

• The docstring states pipeline properties are allowed (item 3), but is_leaf_allowed does not check pipeline_properties, and the surrounding validation intentionally rejects user PATCHes for non-updateable pipeline properties (as covered by new tests). Please update the docstring to match the actual rules (either remove/clarify item 3, or explicitly describe that pipeline props are only allowed to be retained/validated but not user-modified).

        def is_leaf_allowed(prop_path: str, prop_val: Any) -> bool:
            """
            Determines whether a patched leaf property path is permitted.
            Allowed if:
            1. Explicitly marked updateable: true in the template schema on the property itself
               OR on any of its ancestor objects (top-level or via allOf clauses).
            2. Introduced as a new property path during a template upgrade.
            3. Referenced as a pipeline property in the template's install/upgrade pipeline.
            4. Retains its existing value from the resource during an upgrade (data preservation of untouched fields).
            5. Absent from the persisted resource and required by the active target schema during an upgrade.
            """

ui/app/src/utils/schemaUpgradeUtils.ts:265

• pruneSchemaNode currently does repeated linear scans over activeKeys (includes, then filter) for every property, which can become expensive on larger schemas/forms. Consider converting activeKeys to a Set for exact matches and/or pre-grouping keys by top-level prefix once (e.g. a map from propName -> subkeys) so the pruning work is closer to O(n) rather than O(n*m).

  for (const [propName, propSchema] of Object.entries(schemaNode.properties)) {
    if (partGuard(propName)) continue;

    const exactMatch = activeKeys.includes(propName);
    const matchingSubKeys = activeKeys
      .filter((k) => k.startsWith(propName + "."))
      .map((k) => k.slice(propName.length + 1));

ui/app/src/components/shared/ConfirmUpgradeResource.tsx:445

• props.resource.resourcePath appears in the dependency array but isn't referenced inside this effect. Keeping unused dependencies can cause unnecessary schema refetches if resourcePath changes (or if the resource object is recreated with a different resourcePath). Remove it from the dependency list or use it in the effect if it’s intended to influence the fetch.

  }, [
    selectedVersion,
    props.resource.id,
    props.resource.resourceType,
    props.resource.templateName,
    props.resource.templateVersion,
    props.resource.resourcePath,
    props.parentWorkspaceService?.id,
    props.parentWorkspaceService?.templateName,
    workspaceCtx.workspace?.id,
    workspaceCtx.workspaceApplicationIdURI,
    apiCall,
  ]);

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 2 comments.

Files not reviewed (1)

• ui/app/package-lock.json: Generated file

Suppressed comments (4)

api_app/db/repositories/resources.py:308

• This removal detection does multiple any(...startswith...) scans inside loops over existing_paths, removed_template_paths, and target_properties, which can become quadratic as template/property sizes grow. Consider precomputing prefix lookups (e.g., build a prefix trie or maintain a set of all prefixes present in target_properties) so membership/prefix checks are O(1) per path.

            removed_template_paths = {path for path in current_template_properties if path not in target_properties}

            # Remove at the highest path that is completely absent from the target template,
            # for properties that were present in the current template but absent from target template.
            existing_paths = [path for path, _ in self._get_leaf_properties(resource.properties)]
            removed_top_paths: set[str] = set()
            for path in existing_paths:
                schema_path = ".".join(part for part in path.split(".") if not part.isdigit())
                if any(schema_path == tp or schema_path.startswith(tp + ".") for tp in removed_template_paths):
                    # Find the shortest prefix of this path that is fully absent from the target
                    parts = schema_path.split(".")
                    remove_at = schema_path
                    for i in range(1, len(parts)):
                        prefix = ".".join(parts[:i])
                        # If this prefix itself is absent from target_properties and no sub-key
                        # of it exists in target_properties, remove at this level
                        if prefix not in target_properties and not any(
                            tp == prefix or tp.startswith(prefix + ".") for tp in target_properties
                        ):
                            remove_at = prefix
                            break
                    removed_top_paths.add(remove_at)

api_app/db/repositories/resources.py:467

• validate_patch has grown to include many nested helper functions and multiple responsibilities (upgrade detection, schema resolution, leaf-permission evaluation, required adjustment, and final JSONSchema validation). This makes the logic harder to audit and safely extend. Consider extracting the nested helpers into dedicated private methods (or a small helper class) and leaving validate_patch as orchestration; this also makes the rule set easier to unit test in isolation.

    async def validate_patch(self, resource_patch: ResourcePatch, resource_template_repo: ResourceTemplateRepository, resource_template: ResourceTemplate, resource_action: str, current_properties: Optional[dict] = None, target_template: Optional[ResourceTemplate] = None):
        # get the enriched (combined) template for the old/current template
        enriched_template = resource_template_repo.enrich_template(resource_template, is_update=True)

        # get the old template properties (including allOf and system_properties) for comparison during upgrades
        old_template_properties = self._get_all_property_keys_from_template(enriched_template)

        # get the schema for the target version if upgrade is happening
        if resource_patch.templateVersion is not None:
            # fetch the template for the target version if not already provided
            if not target_template:
                parent_service_name = None
                if resource_template.resourceType == ResourceType.UserResource:
                    parent_service_name = getattr(resource_template, "parentWorkspaceService", None)

                target_template = await resource_template_repo.get_template_by_name_and_version(
                    resource_template.name,
                    resource_patch.templateVersion,
                    resource_template.resourceType,
                    parent_service_name=parent_service_name
                )
            enriched_template = resource_template_repo.enrich_template(target_template, is_update=True)

        # Helper to get property schema definition from properties or allOf using dotted path
        def get_prop_schema(schema_dict: dict, path: str) -> Optional[dict]:
            """
            Resolves a property definition dict from top-level properties or allOf conditional clauses
            using a dotted property path (e.g. 'parent_object.child_prop').
            """

ui/app/src/utils/schemaUpgradeUtils.ts:12

• JSON round-tripping will drop undefined object fields, coerce undefined array entries to null, and will throw on circular references; it can also distort non-JSON types (e.g., Date). If these helpers may touch non-trivial resource properties, consider using structuredClone (where available) or a safer deep-clone utility with clearer type/behavior guarantees.

export const clonePropertyValues = <T>(value: T): T => JSON.parse(JSON.stringify(value));

ui/app/src/components/shared/ConfirmUpgradeResource.tsx:117

• templateUsesWsAuth is a constant that is always false, which makes it easy to miss that template GET auth is intentionally fixed. Consider removing the flag and directly passing undefined (or extracting a clearly named helper like getTemplateAuth() that documents why it never uses workspace auth).

  // Template GET endpoints (templateGetPath) always use TRE API authentication,
  // even for UserResource templates, because they use paths like:
  // /workspace-service-templates/{name} (not /workspaces/{id}/...)
  const templateUsesWsAuth = false;

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 1 comment.

Files not reviewed (1)

• ui/app/package-lock.json: Generated file

Suppressed comments (1)

ui/app/src/utils/schemaUpgradeUtils.ts:70

• Cloning the whole source array also copies item fields that the target template removed. If an upgrade both removes items[].old_field and adds items[].new_field, extracting the new field sends old_field back in the PATCH; target validation may reject it, or an updateable parent array may accept and reintroduce it after backend pruning. Preserve only source item fields that still exist in the target schema.

    const sourceValue = currentSource && typeof currentSource === "object" ? currentSource[part] : undefined;
    if (!(part in current) || typeof current[part] !== "object" || current[part] === null) {
      current[part] = Array.isArray(sourceValue) ? clonePropertyValues(sourceValue) : {};

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 1 comment.

Files not reviewed (1)

• ui/app/package-lock.json: Generated file

Suppressed comments (3)

api_app/db/repositories/resources.py:497

• This deep merge now runs for every PATCH, not only template upgrades. For an existing updateable array of objects, submitting a same-length replacement no longer replaces each item: any omitted fields are silently retained from the old item. That changes the established update behavior and makes it impossible to remove an optional item field without also changing the array length. Keep the index-wise preservation for upgrade patches only, and retain the previous top-level replacement semantics for ordinary updates.

            if resource_patch.properties is not None and len(resource_patch.properties) > 0:
                self._deep_dict_update(resource.properties, resource_patch.properties)

ui/app/src/utils/schemaUpgradeUtils.ts:36

• The new property discovery recurses through nested properties and array items, but it never visits an allOf located inside one of those nested schema nodes. Consequently, a property newly introduced by parent.allOf[].then.properties is not detected or rendered, so a required value cannot be supplied before upgrade. Traverse nested conditional branches while preserving the dotted parent prefix.

    } else if (value && typeof value === "object" && "properties" in value) {
      // Include the object container itself so required-object detection works, then recurse into children.
      // Array item properties are traversed with indexed paths such as "items.0.value".
      keys.push(prefix + key);
      keys = keys.concat(getAllPropertyKeys((value as any)["properties"], prefix + key + ".", currentData));

CHANGELOG.md:49

• The COMPONENTS table is release-generated and must not be edited in feature pull requests. This newly added snapshot is also inconsistent with this PR's ui version bump (0.8.31 here versus 0.9.0 in package.json). Remove the added COMPONENTS block and let the release process generate it.

COMPONENTS:

| name | version |
| ----- | ----- |
| devops | 0.6.4 |

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated no new comments.

Files not reviewed (1)

• ui/app/package-lock.json: Generated file

Suppressed comments (2)

api_app/db/repositories/resources.py:705

• The is_new_on_upgrade fast path returns before the array-length check below. For a non-updateable array whose item schema gains a new optional field, a caller can therefore submit a longer/shorter array containing only that new field; every leaf is considered new, _deep_merge_dicts replaces the whole array, and existing protected items can be lost. Apply the cardinality check before accepting new item fields (unless the array itself is explicitly updateable).

                                    break
                curr_schema = next_schema
                if isinstance(curr_state, dict):
                    curr_state = curr_state.get(part)
                elif isinstance(curr_state, list) and part.isdigit() and int(part) < len(curr_state):
                    curr_state = curr_state[int(part)]
                else:

api_app/db/repositories/resources.py:613

• Validating the full merged state breaks ordinary updates that switch an allOf selector. For example, the base workspace schema defines client_id in the Manual branch and create_aad_groups in the Automatic branch (templates/workspaces/base/template_schema.json:263-326). Switching auth_type to Manual leaves the persisted create_aad_groups in valid_current_properties, so unevaluatedProperties: false rejects the patch before it can be saved. Inactive-branch properties need to be pruned for non-version patches as well (or excluded from the validation state).

                target_template = await resource_template_repo.get_template_by_name_and_version(
                    resource_template.name,
                    resource_patch.templateVersion,
                    resource_template.resourceType,
                    parent_service_name=parent_service_name
                )
            enriched_template = resource_template_repo.enrich_template(target_template, is_update=True)

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated no new comments.

Files not reviewed (1)

• ui/app/package-lock.json: Generated file

Suppressed comments (5)

api_app/db/repositories/resources.py:731

• is_new_on_upgrade returns before the array-length guard below. A PATCH containing only a newly introduced array-item field can therefore submit fewer or more items for a non-updateable array; _deep_dict_update then replaces the whole list when lengths differ, deleting or replacing existing items. Enforce unchanged cardinality for non-updateable arrays before granting the new-leaf allowance.

            is_new_on_upgrade = (
                is_upgrade
                and schema_path in target_template_properties
                and schema_path not in old_template_properties
            )

ui/app/src/utils/schemaUpgradeUtils.ts:170

• This lookup only searches direct properties and root-level allOf. The recursive key collector already emits paths from nested conditionals (for example parent.conditional), but this resolver returns null for those paths; consequently defaults are not applied, required input does not block Upgrade, and the property is omitted from the PATCH. Resolve conditional branches at every schema level and reuse that resolver in active-key checks.

  let prop = getSchemaPropertyFromProperties(template.properties, path);
  if (prop) return prop;

  if (template.allOf) {

ui/app/src/utils/schemaUpgradeUtils.ts:422

• collectConditionalKeys includes fields from then and else, so adding a field inside a branch falsely treats that field as a selector change and pulls every sibling branch field into the upgrade form and PATCH. This can expose existing non-updateable fields that the backend will reject if edited. Trigger dependencies only when a supplied key is referenced by the condition's if schema.

    const conditionalKeys = collectConditionalKeys(condition);
    if (!conditionalKeys.some((key) => triggerTopKeys.has(key.split(".")[0]))) return;

ui/app/src/utils/schemaUpgradeUtils.ts:82

• Array items are filtered only at their first level. If an allowed item property is itself an object, cloning it wholesale retains nested fields removed from the target schema; the upgrade PATCH then reintroduces those fields after backend pruning and fails as an unexpected-property validation error. Recursively clone object/array values according to each target subschema.

    const filteredItem: Record<string, any> = {};
    for (const key of Object.keys(schema.items.properties)) {
      if (partGuard(key) || item[key] === undefined) continue;
      filteredItem[key] = clonePropertyValues(item[key]);
    }

api_app/db/repositories/resources.py:513

• The pruned post_patch_props is discarded, while resource.properties is pruned using the pre-patch selector state. If an upgrade changes a selector from a matching to a non-matching value (for example, enable_backup: true to false), properties from the old then branch remain persisted; with unevaluatedProperties: false, they can also make target-schema validation fail. Use the post-patch branch state for validation and final persistence while retaining the original values separately for updateability checks.

            self._remove_inactive_branch_properties(enriched_target_template, post_patch_props)
            self._remove_inactive_branch_properties(enriched_target_template, resource.properties)