Skip to content

Portable attachment provenance and exact rerun metadata #68

Description

@Evangelink

Producer context

Microsoft.Testing.Platform produces reports in local development, CI shards, multi-module runs, and retry processes. The CTRF report can then be merged and retained after the producing machine is gone.

For attachments, MTP currently emits the attachment name, MIME type, and the producer's absolute local file path:

Section 12.3 now clearly resolves one question: attachments[].path may be a URL or file path, has no prescribed URI/path semantics, and must be treated as opaque. That permits MTP to emit an artifact URI instead of a local path when one is available.

Two portability gaps remain.

Questions

Attachment provenance and integrity

  1. When both a producer-local path and a durable artifact URI exist, is path expected to contain only the durable reference, with the local path moved to namespaced extra, or is a standardized way to carry both desirable?
  2. Is there an intended representation for a content digest (algorithm plus hash) so consumers can verify, deduplicate, or retrieve content-addressed attachments? attachmentId identifies a reference instance, so it does not appear to be a content identity.
  3. Would portable artifact metadata belong directly on Attachment, or should producers use a namespaced object under attachments[].extra?

Exact rerun / reproduction metadata

A failed test is not always reproducible from name, suite, and testId alone. MTP may need the test application/module, target framework, architecture, filter or UID, working directory, relevant runner options, and a safe reference or digest for configuration/runsettings. The exact original command line can also contain secrets and machine-local paths.

  1. Does exact rerun/reproduction metadata belong in CTRF as a standardized object, or is it intentionally producer-specific extra data?
  2. If standardized, should it describe a redacted invocation (executable, arguments, workingDirectory, configuration references) or a higher-level portable recipe (module, test selector, target, environment/configuration digests)?
  3. How should this relate to the pre-v1 environment/initiator redesign being discussed in Request for feedback: Restructuring the environment object #61 and to the provenance/derivation work explicitly left out of scope by PR Add identity model #57?

Why these belong together

Both questions determine whether a CTRF document remains actionable after it leaves the worker that produced it: can a consumer retrieve and verify the evidence, and can it reproduce the failing execution without parsing framework-specific prose or logs? We are not proposing a specific schema yet; guidance on first-class fields versus namespaced extra would unblock a correct MTP mapping.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions