Route signed-binary packages through a per-artifact release path - #48966
Daniel Jurek (danieljurek) wants to merge 25 commits into
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Restores the azure-sdk-build-tools repository resource referenced by the mac/win signing templates (@azure-sdk-build-tools), pinned to tag azure-sdk-build-tools_20260702.2 to match azure-dev. This seeds a ref line that the tools-repo-versioning auto-updater will keep current. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8225ba73-f36e-4078-b21a-a7a166d261f9
Restores the ESRP publish path (isolate + esrp-publish.yml) gated on PublicFeed == 'PyPi'. Reads from packages_all_signed/<artifact>, so the signed wheels/sdist are what get published. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8225ba73-f36e-4078-b21a-a7a166d261f9
ESRP Code Signing used minimatch pattern '*.pyd' (non-recursive) against win-sign-input/, but extract_sign_inputs.py places .pyd in numbered subfolders (00000/crc64.pyd). The glob matched 0 of 4 files, the task reported success signing nothing, and repackage reinserted unsigned .pyd. Mac was unaffected because it zips the payload dir and signs the zip. Change WinPattern to '**/*.pyd' so ESRP recurses into the subfolders, matching the mac path's recursive behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8225ba73-f36e-4078-b21a-a7a166d261f9
Uncomment the esrp-publish.yml template call that was temporarily disabled during signing-pipeline testing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8225ba73-f36e-4078-b21a-a7a166d261f9
# Conflicts: # eng/pipelines/templates/stages/archetype-python-release.yml # sdk/storage/azure-storage-extensions/CHANGELOG.md
Packages that ship compiled binaries need their wheels signed by ESRP before release, but the previous approach keyed signing off ServiceDirectory, which dragged every storage package onto the signed path. Make it a per-artifact opt-in instead. An artifact sets `signBinaries: true` in its service ci.yml and gets a dedicated Sign_<safeName> stage plus a Release_<safeName> stage fed from the resulting packages_<safeName>_signed artifact. Every other artifact keeps releasing from packages_extended off the build stage exactly as before, so a service can mix both kinds of packages. The signed artifact mirrors the layout of packages_extended, so the release jobs work against it unchanged. To keep those jobs single-sourced they move verbatim into release-artifact.yml, which parameterizes only the artifact name and the upstream stage. packages_extended itself is untouched. Signing runs on release and scheduled builds and never on PR builds: the public project cannot reach internal/azure-sdk-build-tools, and ESRP must not be handed unreviewed code. Scheduled runs are included because the Integration stage now publishes signed alpha packages to the dev feed for opted-in artifacts. The signing stage deliberately ignores Skip.Release and SetDevVersion. Those belong on the consumers, which already carry them; checking them here would propagate through Integration's dependency and silently break the nightly and manual dev version alpha publishes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9079636e-8478-4697-ae82-538136cd41b5
|
Azure Pipelines: Successfully started running 1 pipeline(s). 10 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Testing the scheduled path directly, rather than simulating it with a queue-time SetDevVersion, showed the signing condition was only ever being satisfied by its 'Manual' term. Two gaps that hid behind that: The 'Schedule' term is coupled to daily-dev-build-variable.yml, which sets SetDevVersion only when Build.Reason is exactly 'Schedule'. Alpha wheels exist only when that is true, so the two must widen together. Record that, along with why the reasons left out of the list need no signing. A missing signed artifact skips one package's alpha publish by design, so a signing failure cannot take down the dev feed publish for a whole service. Raise it as a build issue rather than Write-Warning so that degradation shows up in the run summary instead of only in the task log. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9079636e-8478-4697-ae82-538136cd41b5
ENABLE_EXTENSION_BUILD gated the mac and windows builds on a hardcoded azure-storage-extensions name check, so any other package opting into signBinaries got a signing stage with no wheels to sign. Resolve it from the artifact property instead, which Save-Package-Properties already copies into each package info file as ArtifactDetails. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9079636e-8478-4697-ae82-538136cd41b5
Adds a minimal C extension package so the template service exercises both kinds of release artifact: azure-template stays pure Python and unsigned, while azure-template-two ships platform wheels routed through signing via the signBinaries artifact property. The extension exposes a single is_true() function and is compiled against the limited API, so one cp310 abi3 wheel per platform covers 3.10+. cibuildwheel is configured for x86_64 Linux, x86_64 Windows and arm64 macOS. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9079636e-8478-4697-ae82-538136cd41b5
Doc publishing resolves exactly one wheel per package, which does not hold for a package that ships one wheel per platform, so the namespace step fails converting the match list to a string. azure-storage-extensions already opts out of doc publishing for the same reason. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9079636e-8478-4697-ae82-538136cd41b5
This comment has been minimized.
This comment has been minimized.
Property lookups on a declared object parameter are case sensitive, unlike the each-loop variable this stage was extracted from, so Artifact.safename resolved to empty and every artifact compiled to a stage literally named Release_. A single artifact service still compiled, but any service with two or more artifacts failed with a duplicate stage name. The same spelling was also used to look up the auto-release output variable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9079636e-8478-4697-ae82-538136cd41b5
[Pilot] PR Pipeline Failure AnalysisWhat failedBuild 6834566 — check python - pullrequest (Build Analyze) — failed on two independent validation steps, both triggered by the new 1. CSpell spell-check (log
2. README structure validation ( The Relevant pipeline outputRecommended next steps
Automated fix: Requested
|
Packages that ship compiled binaries need ESRP-signed wheels before release, but signing was keyed off
ServiceDirectory, which pulled every storage package onto the signed path. Make it a per-artifact opt-in instead.signBinaries: truein its serviceci.ymlto get aSign_<safeName>stage and release frompackages_<safeName>_signedpackages_extendedoff the build stage as before, so a service can mix both kindsrelease-artifact.ymlso the two paths stay single-sourcedIntegrationpublishes the signed wheels for opted-in artifacts