[Feat] Support Data Cloud DataSpace-scoped Metadata Types

[aparnagoy]

feat: support Data Cloud (D360) dataspace-scoped metadata types

Summary

Adds first-class SDR support for Data Cloud dataspace-scoped metadata types.
These types do not fit any existing adapter/transformer because their on-disk
representation is unlike every other metadata type:

• Each component is a pair of generic .json files (there is no -meta.xml):
  • ‹name›.json — the payload (the component definition).
  • ‹name›.meta.json — a sidecar carrying relationship metadata
    (componentType, componentName, dataspaceName, retrieveWith, dependsOn).
• Both files are nested under a wrapper directory
  d360/dataspace/‹dataspace›/‹typeDir›/ that must be preserved verbatim.
• The component fullName is dataspace-scoped: ‹dataspace›.‹name›
  (e.g. ‹ComponentType›:‹dataspace›.‹name›), where ‹dataspace› is the path
  segment immediately above the type directory.

The registry, resolver, converter, and deploy-closure logic are taught to handle
that shape end-to-end, in both retrieve (metadata → source) and deploy
(source → metadata) directions.

The runtime pieces are type-agnostic: they derive everything from the path
shape and the sidecar convention, so onboarding an additional dataspace-scoped
type is a registry change (a type entry plus a directory mapping), never a
code change. This keeps the design open for extension and closed for
modification — new types plug in without touching the adapter, transformer, or
closure logic.

Design

Three new pieces of behavior, wired through the existing strategy indirection so
no core class is modified. All three share the single adapter/transformer
strategy id d360:

Layer	Component	Responsibility
registry	metadataRegistry.json	Register each type with the shared d360 adapter + transformer strategies; index the json suffix so bare .json files are recognized as metadata; strict directory names disambiguate one type's ‹typeDir› from another's.
resolve	D360SourceAdapter	Build the ‹dataspace›.‹name› fullName from the path; treat the .meta.json sidecar as part of its sibling payload component (never a component of its own).
convert	D360MetadataTransformer	Preserve the full d360/… path in both directions and co-write the sidecar alongside the payload.
collections	expandD360ComponentSet	Build the minimal deploy closure by walking each component's sidecar retrieveWith set.

Why the json suffix index matters

Resolution gates every file through isMetadata(), which calls
registry.getTypeBySuffix(extName(path)). A suffix that is not present in the
global suffixes index is silently skipped, so bare .json files were never
considered metadata. Indexing json is what lets these components resolve at
all; resolveType then disambiguates the concrete type per-directory via the
strict folder name (each type's registered directoryName), so a single shared
json suffix mapping serves every dataspace-scoped type.

Sidecar handling and the name-collision guard

baseName() splits on the first ., so ‹name›.json and ‹name›.meta.json
both reduce to ‹name›. D360SourceAdapter.populate() therefore must
exclude *.meta.json; otherwise the sidecar would resolve to the same
dataspace-scoped name as the payload and collide. The sidecar is not lost — the
transformer re-emits it verbatim next to the payload on write.

retrieveWith vs dependsOn

The sidecar declares two distinct relationship lists:

• retrieveWith — the components that travel together with this one as a
  single deployable unit. This is what the deploy closure walks.
• dependsOn — informational metadata about what a component references
  (the components it points at). It is not used to build the closure.

End-to-end flow (across repos)

The stack is always user → CLI → SDR → Core (server). The CLI is the command
entry point, SDR is the library that resolves/converts/packages the components
and computes the deploy closure, and Core owns the on-disk/wire contract (it
emits the two-file layout and the server-populated retrieveWith on retrieve,
and validates/applies the unit on deploy). SDR carries no per-type logic — the
d360 strategy is selected purely from the registry.

Layer	Repo	Responsibility
CLI (plugin)	plugin-deploy-retrieve	Builds the component set; on deploy, expands the retrieveWith closure before handing off to SDR.
SDR (library)	source-deploy-retrieve (this PR)	Registry + D360SourceAdapter + D360MetadataTransformer that resolve/convert the two-file layout both directions, and expandD360ComponentSet which computes the deploy closure.
Core (server)	Data Cloud platform	On retrieve emits the d360/… payload + .meta.json sidecar and populates each component's retrieveWith. On deploy validates and applies the unit.

What crosses the SDR boundary

Call	Input into SDR	What SDR does	Returns	To whom
retrieve	requested ComponentSet + org connection	issues the Metadata API retrieve, extracts the returned zip, resolves it (D360SourceAdapter), converts metadata → source (D360MetadataTransformer.toSourceFormat), co-writes the .meta.json sidecar	RetrieveResult + FileResponse[]	CLI (writes files, prints)
deploy closure	full project ComponentSet + requested ComponentSet (+ registry)	indexes d360 members by sidecar componentName, walks retrieveWith transitively	expanded ComponentSet (requested + closure)	CLI (feeds it into the deploy)
deploy	expanded ComponentSet + org connection	resolves, converts source → metadata (D360MetadataTransformer.toMetadataFormat), packages the zip, submits the Metadata API deploy, polls	DeployResult	CLI → user

sf project retrieve start — metadata → source

sequenceDiagram
    actor User
    participant CLI as CLI (plugin-deploy-retrieve)
    participant SDR as SDR (source-deploy-retrieve)
    participant Core as Core (server)
    User->>CLI: sf project retrieve start --metadata ‹Type›:‹dataspace›.‹name›
    CLI->>SDR: ComponentSetBuilder.build(requested) → componentSet.retrieve(org)
    SDR->>Core: Metadata API retrieve request
    Core-->>SDR: unpackaged.zip (payload + .meta.json sidecar,<br/>server-populated retrieveWith)
    SDR->>SDR: resolve (D360SourceAdapter) + convert to source (D360MetadataTransformer.toSourceFormat)
    SDR-->>CLI: RetrieveResult + FileResponse[]
    CLI-->>User: writes force-app/main/default/d360/dataspace/…<br/>(payload + sidecar), prints result

Loading

Call stack inside SDR (retrieve):

componentSet.retrieve(...)
└─ MetadataApiRetrieve.start() → poll checkStatus() → post()
   ├─ extract unpackaged.zip into a ZipTreeContainer
   ├─ MetadataResolver.getComponentsFromPath (over the zip tree)
   │  ├─ RegistryAccess.getTypeBySuffix('json')          # suffix now indexed
   │  ├─ RegistryAccess.resolveTypeFromStrictFolder      # per-type directoryName
   │  └─ SourceAdapterFactory.getAdapter('d360') → D360SourceAdapter
   │     └─ populate(): skip *.meta.json, name = ‹dataspace›.‹name›
   └─ MetadataConverter.convert(components, 'source', …)
      └─ ComponentConverter → MetadataTransformerFactory.getTransformer → D360MetadataTransformer
         └─ toSourceFormat → getWriteInfos: payload + .meta.json sidecar
            └─ getD360Destination: join(main/default, trimUntil(path, 'd360'))
# returns → RetrieveResult { fileProperties, success, … } + FileResponse[]

sf project deploy start — source → metadata

sequenceDiagram
    actor User
    participant CLI as CLI (plugin-deploy-retrieve)
    participant SDR as SDR (source-deploy-retrieve)
    participant Core as Core (server)
    User->>CLI: sf project deploy start --source-dir …
    CLI->>SDR: ComponentSetBuilder.build(requested)
    SDR-->>CLI: requested ComponentSet
    CLI->>CLI: hasD360Components(requested)?
    CLI->>SDR: ComponentSetBuilder.build(whole project) → full
    CLI->>SDR: expandD360ComponentSet(full, requested, registry)
    SDR-->>CLI: requested + transitive retrieveWith closure
    CLI->>SDR: componentSet.deploy(org)  (MetadataApiDeploy)
    SDR->>Core: Metadata API deploy (payload + sidecar under d360/…)
    Core-->>SDR: DeployResult
    SDR-->>CLI: DeployResult
    CLI-->>User: prints result

Loading

Call stack inside SDR (deploy):

# 1) closure (CLI calls the exported SDR function)
expandD360ComponentSet(full, requested, registry)
├─ index d360 members by sidecar componentName
├─ seed worklist with requested members
└─ while worklist: add each sidecar retrieveWith peer, recurse
# returns → expanded ComponentSet (requested + closure)

# 2) deploy (CLI hands the expanded set back to SDR)
componentSet.deploy(...)
└─ MetadataApiDeploy.start()
   └─ MetadataConverter.convert(components, 'metadata', { type: 'zip' })
      └─ ComponentConverter → MetadataTransformerFactory.getTransformer → D360MetadataTransformer
         └─ toMetadataFormat → getWriteInfos: payload + .meta.json sidecar
            └─ getD360Destination: trimUntil(path, 'd360') at the package root
   └─ POST zip to Core Metadata API deploy → poll
# returns → DeployResult

Call stack (resolution & conversion detail)

Resolution (retrieve):

MetadataResolver.getComponentsFromPathRecursive
└─ MetadataResolver.resolveComponent
   ├─ RegistryAccess.getTypeBySuffix('json')            # gate: suffix now indexed
   ├─ RegistryAccess.resolveTypeFromStrictFolder        # per-type directoryName
   └─ SourceAdapterFactory.getAdapter('d360')
      └─ D360SourceAdapter.getComponent
         └─ (MixedContentSourceAdapter) → D360SourceAdapter.populate
            ├─ skip when path endsWith '.meta.json'
            └─ calculateD360Name  →  '‹dataspace›.‹name›'

Conversion (both directions):

MetadataConverter.convert
└─ ComponentConverter (transform stream)
   └─ MetadataTransformerFactory.getTransformer → D360MetadataTransformer
      ├─ toSourceFormat / toMetadataFormat
      └─ getWriteInfos
         ├─ walkContent()                       # payload files
         ├─ derive '‹name›.meta.json' sidecar    # co-written when present
         └─ getD360Destination                   # trimUntil(path, 'd360') under target root

Deploy closure:

expandD360ComponentSet(full, requested)
├─ index d360 members by sidecar componentName
├─ seed worklist with requested members
└─ while worklist: add each sidecar retrieveWith peer, recurse

Testing

• yarn build (compile + lint) — green.
• yarn test (unit suite) — green.
• Manual end-to-end: sf project retrieve start writes the payload +
  .meta.json sidecar for a retrieved component and its co-emitted
  retrieveWith peers under
  force-app/main/default/d360/dataspace/‹dataspace›/‹typeDir›/, and the reverse
  deploy direction reproduces the same layout in the mdapi package.

[salesforce-cla]

Thanks for the contribution! Unfortunately we can't verify the commit author(s): Aparna Goyal <r***@A***.com>. One possible solution is to add that email to your GitHub account. Alternatively you can change your commits to another email and force push the change. After getting your commits associated with your GitHub account, sign the Salesforce Inc. Contributor License Agreement and this Pull Request will be revalidated.