Repository navigation
Separate the source-built native image #1539
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
f45b554
a4b3b48
fe04ce1
5781382
b9e1ff2
8fef1e1
37c7fab
5d3cbeb
439ccb8
8de80f2
718bfc1
24867d1
c677a8c
866c519
d96063b
94ef18e
31981ca
224540e
8d41cb0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| # Allow only build inputs, never the entire operator checkout. | ||
| ** | ||
| !Cargo.toml | ||
| !Cargo.lock | ||
| !.tool-versions | ||
| !rust-toolchain.toml | ||
| !edgezero.toml | ||
| !.cargo/ | ||
| !.cargo/config.toml | ||
| !crates/ | ||
| !crates/** | ||
| # Exclude operator TOMLs even if placed below a crate. | ||
| **/*.toml | ||
| !**/Cargo.toml | ||
| !rust-toolchain.toml | ||
| !edgezero.toml | ||
| !.cargo/config.toml | ||
| !trusted-server.example.toml | ||
| **/.git | ||
| **/target | ||
| **/node_modules | ||
| **/dist | ||
| crates/trusted-server-adapter-cloudflare/build | ||
| crates/trusted-server-adapter-cloudflare/pkg | ||
| **/.env | ||
| **/.env.* | ||
| **/*.pem | ||
| **/*.key | ||
| **/*.p12 | ||
| **/*.pfx | ||
| **/keys | ||
| **/secrets | ||
| **/.aws | ||
| **/.npmrc | ||
| **/.cargo/credentials* | ||
| **/.ts-tmp | ||
| **/.tsjs-npm-ci.lock |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| name: Native RC container | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - Dockerfile | ||
| - .dockerignore | ||
| - .tool-versions | ||
| - rust-toolchain.toml | ||
| - Cargo.toml | ||
| - Cargo.lock | ||
| - edgezero.toml | ||
| - .cargo/** | ||
| - crates/trusted-server-adapter-axum/** | ||
| - crates/trusted-server-core/** | ||
| - crates/trusted-server-js/** | ||
| - crates/trusted-server-openrtb*/** | ||
| - scripts/test-native-container.mjs | ||
| - .github/workflows/native-container.yml | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| local-provider-image: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 30 | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| persist-credentials: false | ||
| - name: Read pinned Node version | ||
| id: node | ||
| run: | | ||
| version=$(awk '$1 == "nodejs" { print $2 }' .tool-versions) | ||
| test -n "$version" | ||
| echo "version=$version" >> "$GITHUB_OUTPUT" | ||
| - uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: ${{ steps.node.outputs.version }} | ||
| - name: Build local-provider image | ||
| run: docker build --build-arg TRUSTED_SERVER__GIT_VERSION="${{ github.sha }}" -t ts-rc-local:test . | ||
| - name: Check real TS routes, startup rejection and drain | ||
| run: node scripts/test-native-container.mjs ts-rc-local:test | ||
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,64 @@ | ||||||||||
| # syntax=docker/dockerfile:1 | ||||||||||
| # Adapted from the pinned EdgeZero RC's hardened native recipe. Build at workspace root. | ||||||||||
| FROM node:24.20.0-bookworm-slim@sha256:ba849c60be29959425b8734d57b8b4b7d56f98edd9504c9af091d5281095a71e AS node | ||||||||||
| FROM rust:1.98.1-bookworm@sha256:93ce27a88655056a51dbdd8f5f2d7ddc071c7b0070fb288a37b5a285fc83971e AS builder | ||||||||||
| COPY --from=node /usr/local/ /usr/local/ | ||||||||||
| WORKDIR /src | ||||||||||
| COPY . . | ||||||||||
| # AWS providers and CloudFront Geo are default-off. Forward them explicitly. | ||||||||||
| ARG NATIVE_FEATURES="" | ||||||||||
| # Optional existing build metadata input; .git never enters the build context. | ||||||||||
| ARG TRUSTED_SERVER__GIT_VERSION="" | ||||||||||
| RUN <<'SH' | ||||||||||
| set -eu | ||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ♻️ refactor — Each image build downloads three unused wasm standard libraries.
Setting
Suggested change
|
||||||||||
| export RUSTUP_TOOLCHAIN=1.98.1 CARGO_BUILD_JOBS=2 CARGO_INCREMENTAL=0 CARGO_PROFILE_RELEASE_DEBUG=0 | ||||||||||
| test "$(rustc --version | awk '{print $2}')" = 1.98.1 | ||||||||||
| test "$(node --version)" = v24.20.0 | ||||||||||
| test "$(awk '$1 == "nodejs" {print $2}' .tool-versions)" = 24.20.0 | ||||||||||
| test "$(awk '$1 == "rust" {print $2}' .tool-versions)" = 1.98.1 | ||||||||||
| grep -Eq '^channel = "1.98.1"$' rust-toolchain.toml | ||||||||||
| feature_ifs=$IFS | ||||||||||
| IFS=, | ||||||||||
| for feature in $NATIVE_FEATURES; do | ||||||||||
| case "$feature" in | ||||||||||
| aws-appconfig-agent|aws-secrets-manager|aws-dynamodb-ec|celld-ec|geo-cloudfront) ;; | ||||||||||
| *) echo 'Only the declared native store, EC and Geo features are supported' >&2; exit 1 ;; | ||||||||||
| esac | ||||||||||
| done | ||||||||||
| IFS=$feature_ifs | ||||||||||
| unset feature feature_ifs | ||||||||||
| apt-get update | ||||||||||
| apt-get install -y --no-install-recommends cmake | ||||||||||
| rm -rf /var/lib/apt/lists/* | ||||||||||
| npm --prefix crates/trusted-server-js/lib ci --no-audit --no-fund | ||||||||||
| # No host target, node_modules or prebuilt dist enters this context. build.rs | ||||||||||
| # builds and validates new bundles in Cargo's private OUT_DIR before embedding. | ||||||||||
| host=$(rustc -vV | awk '$1 == "host:" {print $2}') | ||||||||||
| set -- | ||||||||||
| if test -n "$NATIVE_FEATURES"; then set -- --features "$NATIVE_FEATURES"; fi | ||||||||||
| cargo build -p trusted-server-adapter-axum --bin trusted-server-axum --release --locked --target "$host" "$@" | ||||||||||
| mkdir /out | ||||||||||
| cp "target/$host/release/trusted-server-axum" /out/app | ||||||||||
| strip --strip-debug /out/app | ||||||||||
| SH | ||||||||||
|
|
||||||||||
| # Retain an auditable hash list of the exact context, not host working-tree state. | ||||||||||
| RUN find . -type d \( -name target -o -name node_modules \) -prune -o -type f ! -name .tsjs-npm-ci.lock -print0 \ | ||||||||||
| | sort -z | xargs -0 sha256sum > /out/build-inputs.sha256 | ||||||||||
|
|
||||||||||
| FROM debian:bookworm-slim@sha256:7c7b2c966bc9ee8cedfeef67e0e279108992c77681fa595db4a9d65c06ccc587 AS runtime | ||||||||||
| RUN apt-get update \ | ||||||||||
| && apt-get upgrade -y --no-install-recommends \ | ||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤔 thinking — The base images are digest-pinned and the image records a hash of its build inputs. But |
||||||||||
| && apt-get install -y --no-install-recommends ca-certificates libgcc-s1 \ | ||||||||||
| && rm -rf /var/lib/apt/lists/* | ||||||||||
| WORKDIR /app | ||||||||||
| COPY --from=builder /out/app /usr/local/bin/app | ||||||||||
| COPY --from=builder /out/build-inputs.sha256 /usr/local/share/ts-build-inputs.sha256 | ||||||||||
| ENV EDGEZERO__ADAPTER__HOST=0.0.0.0 \ | ||||||||||
| EDGEZERO__ADAPTER__PORT=8787 \ | ||||||||||
| EDGEZERO__ADAPTER__CONFIG_DIR=/config \ | ||||||||||
| EDGEZERO__ADAPTER__DATA_DIR=/data | ||||||||||
| USER 10001:10001 | ||||||||||
| EXPOSE 8787 | ||||||||||
| STOPSIGNAL SIGTERM | ||||||||||
| ENTRYPOINT ["/usr/local/bin/app"] | ||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| # Native container | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📝 note — Nothing links to this page. It isn't in the |
||
|
|
||
| Build the Axum adapter as a native Linux container from the workspace root: | ||
|
|
||
| ```sh | ||
| docker build \ | ||
| --build-arg TRUSTED_SERVER__GIT_VERSION="$(git rev-parse HEAD)" \ | ||
| -t ts-native:local . | ||
| ``` | ||
|
|
||
| The multi-stage build uses pinned Rust 1.98.1 and Node 24.20.0 images. The build checks both versions against `.tool-versions`. It installs JS dependencies with `npm ci` and builds fresh bundles through Cargo. Host `target`, `node_modules` and `dist` directories never enter the build context. | ||
|
|
||
| ```mermaid | ||
| flowchart TD | ||
| A[Workspace build inputs] --> B[Rust and Node builder] | ||
| B --> C[Native binary and input hashes] | ||
| C --> D[Non-root Debian runtime] | ||
| ``` | ||
|
|
||
| The runtime image contains the application binary, CA roots and runtime libraries, without Rust or Node. It runs as UID and GID `10001`, listens on port `8787` on all container interfaces, and uses `/config` and `/data` for runtime configuration and data. Mount configuration read-only and provide writable storage for `/data` and `/tmp` when using a read-only root filesystem. Supply configuration and secrets at runtime, never as image build arguments or files in the source checkout. | ||
|
|
||
| The `.dockerignore` file limits the build context to workspace build inputs. It excludes operator TOMLs, environment files, credentials and host build outputs. The image stores an input hash list at `/usr/local/share/ts-build-inputs.sha256`. `TRUSTED_SERVER__GIT_VERSION` supplies optional build metadata because `.git` is excluded. | ||
|
|
||
| ## Optional AWS providers | ||
|
|
||
| Local providers are the default. Enable only the native AWS providers needed by the application: | ||
|
|
||
| ```sh | ||
| docker build \ | ||
| --build-arg NATIVE_FEATURES=aws-appconfig-agent,aws-secrets-manager \ | ||
| -t ts-native:aws . | ||
| ``` | ||
|
|
||
| `NATIVE_FEATURES` accepts an empty value or a comma-separated selection of `aws-appconfig-agent`, `aws-secrets-manager`, `aws-dynamodb-ec`, `celld-ec` and `geo-cloudfront`. Compiling a provider does not select it. Validated runtime configuration selects EC storage, and a separate boot profile selects protected CloudFront Geo. Credentials and provider configuration belong in the runtime environment, not the image. The local-provider smoke test does not validate AWS access. | ||
|
|
||
| ## Local smoke test | ||
|
|
||
| With Docker available, use the host Node version pinned in `.tool-versions`: | ||
|
|
||
| ```sh | ||
| node scripts/test-native-container.mjs ts-native:local | ||
| ``` | ||
|
|
||
| The test creates a private Docker network, a synthetic publisher origin and temporary local configuration. It checks startup rejection for invalid configuration and missing secrets, the published listener, HTML rewriting, embedded JS, non-root execution, CA roots, writable mounts and SIGTERM draining with a request in flight. Fixtures use only `example.com` identities and synthetic secrets. The test removes its containers, network and temporary files on completion. | ||
|
|
||
| The `Native container` workflow builds the local-provider image and runs this test on relevant pull requests or manual dispatch. It does not publish an image or deploy a service. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤔 thinking: On
pull_requesteventsgithub.shais the synthetic merge commit, not the PR head, so the image's bakedx-ts-versionnames a commit that exists on neither branch. The guide builds with$(git rev-parse HEAD), so CI should label the image with the head commit too.workflow_dispatchstill falls back togithub.sha.