Skip to content

chore: untrack generated Rust and iOS outputs, generate on demand - #551

Open
pgherveou wants to merge 8 commits into
mainfrom
untrack-generated-files
Open

chore: untrack generated Rust and iOS outputs, generate on demand#551
pgherveou wants to merge 8 commits into
mainfrom
untrack-generated-files

Conversation

@pgherveou

@pgherveou pgherveou commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

Generated outputs are not tracked. The Rust dispatcher, wire table and wasm bridge, the UniFFI Swift bindings, the FFI headers and modulemaps, and the lockdown container bundle are git-ignored build outputs produced from the Rust crates on demand.

Producing the outputs

scripts/codegen.sh emits the Rust outputs, including the mod.rs that truapi-server declares with pub mod generated;. make setup runs it, make codegen regenerates on its own. Every Make target that compiles truapi-server depends on check-generated, which names the missing file and points at make codegen instead of letting the failure surface as error[E0583]: file not found for module 'generated'.

In CI the Codegen job produces them once and uploads the codegen-output artifact; every job that compiles truapi-server depends on that job and downloads it. ci-android, release-android and release-ios are separate workflows with no such job, so they run scripts/codegen.sh in place with TRUAPI_SKIP_PACKAGE_BUILD=1.

The Codegen job also gates that nothing generated is committed, with git ls-files -ci --exclude-standard over the whole repo.

How iOS consumers get the sources

SwiftPM resolves a package's source targets from the git checkout and has no way to fetch them from a release asset: binaryTarget accepts only .xcframework and .artifactbundle. So an iOS release is two artifacts.

artifact carries consumer sees it as
@parity/ios-host@<version> release asset truapi_server.xcframework.zip publishedBinaryURL + checksum
plain semver tag <version> generated bindings, FFI headers, container bundle, manifest pointing at the asset the thing they pin

Consumers pin the semver tag:

.package(url: "https://github.com/paritytech/host-rust-core", exact: "0.14.0")

ios/truapi-host/scripts/tag-release.sh builds that commit. It reads the required paths out of Package.swift rather than repeating them, so the file set cannot drift from what SwiftPM looks for, and it refuses to tag an xcframework into git. When the tag already exists with an identical manifest it exits successfully, so a rerun of the release pushes the existing tag as a no-op; a tag with a different manifest is an error.

release-ios.yml publishes the asset and the tag. release.yml calls it for a release: @parity/ios-host <version> subject, and it is also dispatchable on its own, matching release-android.yml. Dispatching it from a branch with a version like 0.14.0-beta.1 cuts an asset and a tag an app can pin, which is how an unmerged host change gets tested in the app. A dispatched run passes no manifest_branch and touches no branch.

When called from the release flow, the job opens a pull request on release/ios-host-<version> that points the release branch's Package.swift at the new asset, and dispatches CI for it explicitly because events created with GITHUB_TOKEN do not start another workflow.

The job clones the tag and compiles it against the published asset before pushing, so a tag that cannot be resolved is never published.

What CI proves

iOS package (swift compile) builds the tag rather than the working tree: it runs tag-release.sh, clones the result, diffs the cloned file set against the working tree under every declared path, and compiles the package and its test target from that clean clone. Compiling in place would pass on generated files the tag does not carry, which is the exact failure being guarded.

iOS bindings (uniffi) regenerates the Swift and provider bindings. With nothing committed to diff against, what it gates is that bindgen still produces a binding for every UniFFI-exposed type.

ios-swift's path filter names every crate the bindings are generated from (truapi, truapi-platform, truapi-server, truapi-provider) plus js/container/, because a protocol change leaves no ios/ diff to key on.

Notes

  • Package.swift declares TrUAPIProvider as a source target, so its bindings have to exist even for a consumer that only imports TrUAPIHost. ios/truapi-provider/scripts/sync-bindings.sh produces them without Xcode or the iOS targets, and rebuild.sh delegates to it.
  • .gitignore lists the three generated Swift filenames rather than truapi*.swift. Under core.ignorecase, which git sets by default on macOS, the glob also matches the hand-written TrUAPIHost.swift.
  • The mod.rs emitted alongside dispatcher.rs and wire_table.rs has a golden fixture. It is the file that makes truapi-server parse, so losing it fails a test rather than the next build.
  • tag-release.sh and the manifest step both assert that Package.swift reads TRUAPI_USE_LOCAL_BINARY from the environment, so a published manifest can never pin the local binary.

Verifying the release path

GitHub only exposes workflow_dispatch for workflows present on the default branch, so release-ios.yml cannot be dispatched from this branch. The tag's content and buildability are covered on the PR by the iOS package (swift compile) job above. Cutting a pre-release to exercise the publish half (asset upload, tag push, resolution from GitHub by the app) needs this merged first.

pgherveou and others added 8 commits September 2, 2026 12:43
…pi-server

The untracking commits fixed only ci.yml's rust job. truapi-server declares
`pub mod generated;` unconditionally, so every other job that builds it fails
at parse time: ios-bindings, ios-swift, ci-android, release-android and
release.yml's publish-ios.

Also:
- ios-swift never ran sync-bindings.sh or the js/container build, so the
  package had no Swift sources and no .copy resource to resolve.
- the ios-changes filter keyed on an ios/ diff that generated bindings no
  longer produce; it now names the crates the bindings come from.
- release.yml's post-rebuild `git diff --exit-code` could no longer fail.
- .gitignore's truapi*.swift glob matched the hand-written TrUAPIHost.swift
  under core.ignorecase, which git sets by default on macOS.
- check-generated is a prerequisite of every target that compiles the crate,
  names the missing file, and covers mod.rs and wire_table.rs.
- sync-bindings.sh --check and provider-swift-check compared against files
  that are no longer committed.
SwiftPM resolves source targets from the git checkout, so a branch that
ignores the generated bindings cannot be consumed. A release is now two
artifacts: the xcframework as a release asset, and a plain semver tag whose
commit carries the generated sources plus a manifest pointing at that asset.
Consumers keep the same URL and the same exact() pin.

publish-ios moves into release-ios.yml so it is callable from release.yml and
dispatchable for a pre-release, matching release-android.yml. The tag is
verified by cloning and building it before the push.

tag-release.sh reads the required paths out of Package.swift so the file set
cannot drift from what SwiftPM looks for.

Also fixes the manifest guard: 9b708fb replaced 'let useLocalBinary = false'
with the ProcessInfo form, but release.yml still grepped for the literal, so
the follow-up commit step could never pass.
Also compiles the ios-swift CI gate from a clean clone of the tag rather than
in place, so a generated file the tag does not carry fails the PR instead of
the next consumer.
@valentinfernandez1
valentinfernandez1 marked this pull request as ready for review September 2, 2026 15:46
@valentinfernandez1
valentinfernandez1 requested a review from a team September 2, 2026 15:46
# A rerun of the release finds the tag already cut. Accept it when it records
# the same manifest, so the caller can push it again as a no-op.
if git rev-parse -q --verify "refs/tags/${VERSION}" >/dev/null 2>&1; then
if git diff --quiet "refs/tags/${VERSION}" -- Package.swift; then

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rerun check only diffs Package.swift, but the whole reason this tag exists is to carry the generated sources. If someone cuts the tag, then regenerates bindings and reruns the release at the same version, this exits zero and says nothing to do, and consumers stay pinned to the old bindings. Can we compare the manifest paths' contents against the tag too, and error when they differ?

exit 65
fi

git commit -q -m "release(ios): TrUAPIHost ${VERSION}"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This dies for anyone with commit signing on globally, it fails on my machine. CI is fine because there's no signing config there, but the manual fallback is the case that matters. Worth passing --no-gpg-sign, or -c commit.gpgsign=false.

@TarikGul TarikGul left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just two small nits

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants