Skip to content

scripts/helm-docs.sh: add --check, fail only on README diffs - #954

Open
marcleblanc2 wants to merge 2 commits into
mainfrom
marc/helm-docs-posix-sh
Open

scripts/helm-docs.sh: add --check, fail only on README diffs#954
marcleblanc2 wants to merge 2 commits into
mainfrom
marc/helm-docs-posix-sh

Conversation

@marcleblanc2

@marcleblanc2 marcleblanc2 commented Sep 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Changes

Adds --check

  • Only READMEs count: the script checksums charts/**/README.md before and after helm-docs runs, and fails listing any README helm-docs rewrote. A developer's unrelated uncommitted edits no longer fail the check locally, and a README that was regenerated but not yet committed passes. In CI the tree is clean before the run, so the result is the same as the old git status check

  • Moves the && [[ -z $(git status -s) ]] check inside the script, simplifying the "Verify helm-docs is up-to-date" Buildkite step and AGENTS.md

  • Drops git dependency, using only find, cksum, and grep -xF

Makes the script POSIX portable

  • Runs from the repository root regardless of the caller's cwd, since helm-docs scans the cwd and reads .helmdocsignore from there

  • mktemp uses a six-X template and the temp dir is removed after the download

  • The pinned helm-docs version and target/bin cache location are unchanged

Test plan

Ran with dash and macOS /bin/sh, from the repo root, from /tmp, and from inside scripts/:

  • Clean tree: --check exits 0 with nothing on stdout; a plain run regenerates all four chart READMEs and git status stays clean

  • Unrelated dirty values.yaml (comment only): --check exits 0

  • A new key added to charts/sourcegraph/values.yaml and charts/sourcegraph-migrator/values.yaml

    • --check exits 1, prints both README paths on stdout and the hint on stderr
    • A second --check after regenerating READMEs exits 0

  • Cold cache: downloads once into target/bin/helm-docs-1.14.2, no helm-docs.* dirs left in $TMPDIR

  • --version and --check --log-level=warning pass the args through to helm-docs

  • shellcheck -s sh scripts/helm-docs.sh is clean

@github-actions

github-actions Bot commented Sep 30, 2026

Copy link
Copy Markdown

Open the Diff Tour for this PR in Sourcegraph

@marcleblanc2 marcleblanc2 changed the title scripts/helm-docs.sh: POSIX sh, so it runs anywhere a shell and curl exist scripts/helm-docs.sh: Make script consistent across environments (POSIX) Sep 30, 2026
@marcleblanc2 marcleblanc2 changed the title scripts/helm-docs.sh: Make script consistent across environments (POSIX) CI: Make scripts/helm-docs.sh portable Sep 30, 2026
@marcleblanc2 marcleblanc2 changed the title CI: Make scripts/helm-docs.sh portable CI: Make helm-docs.sh portable Sep 30, 2026
@marcleblanc2
marcleblanc2 marked this pull request as draft September 30, 2026 21:38
@marcleblanc2
marcleblanc2 marked this pull request as ready for review October 2, 2026 00:05
@marcleblanc2
marcleblanc2 requested a review from DaedalusG October 2, 2026 00:05
@DaedalusG DaedalusG mentioned this pull request Oct 14, 2026
Merged
michaellzc pushed a commit that referenced this pull request Oct 21, 2026
@DaedalusG
charts: Keep chart READMEs unchanged across releases (#963)
c36d89e 
Every release rewrites the image digests and registry in `values.yaml`
and the chart version in `Chart.yaml`, and all three show up in the
generated READMEs. That forced the release worker to regenerate the
READMEs with its own copy of helm-docs. When that copy drifted from
`scripts/helm-docs.sh`, `internal/release-v8.1.0` failed ":book: Verify
helm-docs is up-to-date" and had to be fixed by hand. The regeneration
also doesn't reach customers. The publish Action packages the chart from
the first push to `promote/release-*`, before the docs commit lands, so
the published `sourcegraph-8.0.0.tgz` README lists the internal registry
as the default.

This PR makes the READMEs independent of everything a release changes,
so the release worker can drop its helm-docs step entirely
(sourcegraph/sourcegraph#17056):

- Every pinned image `defaultTag` gets `` # @default --
`<appVersion>@sha256:<digest>`, pinned by each release ``, and
`sourcegraph.image.repository` gets `# @default -- Sourcegraph's public
Docker Hub registry`, in the four charts the release updates. The two
executor images also get the `# -- ` description helm-docs needs before
it honors `@default`. The registry annotation deliberately avoids the
literal registry string, because the release replaces that string
everywhere in `values.yaml`, comments included.
- The migrator README uses a `<chart-version>` placeholder instead of
`{{ template "chart.version" . }}`.
- A new CI step fails if any chart README renders a pinned digest or
`index.docker.io/sourcegraph`. A new image added without an annotation
then fails on its own PR rather than on the next release branch. It is a
separate step so it doesn't conflict with #954, which rewrites the
existing helm-docs step.
- Published charts still list the exact values. Before `helm package`,
the release and insiders publish workflows run the new
`scripts/ci/render-published-readmes.sh`. It deletes the release-value
`@default` annotations and reruns `./scripts/helm-docs.sh`, so each
published README lists the same digests and registry as the
`values.yaml` packaged with it. Nothing is committed. Because the docs
are built from the commit being published, the README no longer depends
on a later docs commit landing first, which was the cause of the wrong
v8.0.0 README. The step is `continue-on-error`: if it fails (for
example, the helm-docs download from GitHub fails), the chart still
publishes with the committed placeholder README.

The tradeoff is that READMEs viewed in this repository show placeholders
instead of exact digests. The branch's `values.yaml` still has them, and
the README in the published chart (`helm show readme`) shows them in
full. The migrator's `helm upgrade` examples keep the `<chart-version>`
placeholder in the published chart too.

## How this was tested

- **Simulated release.** CI can't exercise a release, so I reproduced
one on a scratch worktree. A temporary test in the sourcegraph repo ran
the release worker's own `values.yaml` rewrite
(`dev/images.UpdateHelmManifestContent`) over the four charts the
release touches. It first rewrote them to the internal registry with new
tags and digests (the internal phase), then back to the public registry
(the promote phase). I bumped `Chart.yaml` `version`/`appVersion` with
the same regex as `UpdateChartVersions`, and ran
`./scripts/helm-docs.sh` after each phase.

  | | README lines changed after internal phase | after promote phase |
  |---|---|---|
  | `main` | 68 | 60 |
  | this branch | 0 | 0 |

`values.yaml` and `Chart.yaml` changed as expected in both runs. Zero
drift on this branch also shows that the release rewrite leaves the `#
@default --` comments intact.
- **Annotation coverage.** After regenerating, no chart README contains
a 64-character `@sha256:` digest or `index.docker.io/sourcegraph`. A
second `./scripts/helm-docs.sh` run leaves the tree clean.
- **New CI guard.** I ran the new step's command against both trees. It
exits 0 on this branch. On `main` it exits 1 and lists the offending
lines, for example `sourcegraph.image.repository` in
`charts/sourcegraph-executor/dind/README.md`.
- **Publish rendering, end to end.** In a scratch worktree I rebuilt a
promote branch the way a release would: internal phase, `Chart.yaml`
bump, promote phase. Buildkite's helm-docs check stays clean on that
tree. I then ran the publish job's steps with Helm v3.4.0, the version
the workflows pin: the new script, then both `helm package` loops. I
extracted each `.tgz` and compared its README with its packaged
`values.yaml`. The expected values were taken from `values.yaml`, not
from helm-docs output.

| Scenario | Digests in packaged `values.yaml` | Missing from packaged
README | Placeholders left | Registry row |
  |---|---|---|---|---|
| Release publish (4 charts) | 24 / 1 / 1 / 1 | 0 | 0 |
`"index.docker.io/sourcegraph"` |
| Insiders publish (4 charts) | 24 / 1 / 1 / 1 | 0 | 0 |
`"index.docker.io/sourcegraph"` |
| Render step forced to fail | 24 / 1 / 1 / 1 | all | all | placeholder;
packaging still succeeds |

For every chart, in all three scenarios, `helm template` of the packaged
chart matches the unmodified chart, after normalizing the migrator Job's
random name suffix. The script only touches comments and READMEs. The
`sourcegraph` chart has 25 annotated images but 24 distinct digests,
because `codeIntelDB` and `pgsql` share one. `shellcheck` is clean, and
`actionlint` reports the same three pre-existing `actions/checkout@v3`
findings as on `main`.
- **CI.** On the first commit, [Buildkite build
3049](https://buildkite.com/sourcegraph/deploy-sourcegraph-helm/builds/3049)
passed, including ":book: Verify helm-docs is up-to-date" and the new
":book: Verify READMEs do not render release-managed values" step. "Lint
and Test charts" also passed. The changes to `values.yaml` are comments
only, so rendered manifests are unchanged. The publish workflows only
run on pushes to `main`, `release/**` and `promote/release-**`, so this
PR's CI can't exercise them. The end-to-end run above stands in for
that.

The first real check of the full flow will be the next release once both
PRs are deployed.

Merge this, and backport it to every `X.Y.x` branch that still gets
patch releases (at least `8.0.x` and `8.1.x`), before the release worker
change deploys. Patch releases branch from `X.Y.x`, and a branch without
these annotations would fail the helm-docs check once the worker stops
regenerating READMEs.

Part of
[PLAT-876](https://linear.app/sourcegraph/issue/PLAT-876/find-a-permanent-fix-for-helm-docs-release-failures).
@sourcegraph-release-bot sourcegraph-release-bot mentioned this pull request Oct 21, 2026
Merged
michaellzc pushed a commit that referenced this pull request Oct 21, 2026
@sourcegraph-release-bot @DaedalusG
[Backport 8.1.x] charts: Keep chart READMEs unchanged across releases (…
6c57f47 
…#973)

Backport c36d89e from #963

Every release rewrites the image digests and registry in `values.yaml`
and the chart version in `Chart.yaml`, and all three show up in the
generated READMEs. That forced the release worker to regenerate the
READMEs with its own copy of helm-docs. When that copy drifted from
`scripts/helm-docs.sh`, `internal/release-v8.1.0` failed ":book: Verify
helm-docs is up-to-date" and had to be fixed by hand. The regeneration
also doesn't reach customers. The publish Action packages the chart from
the first push to `promote/release-*`, before the docs commit lands, so
the published `sourcegraph-8.0.0.tgz` README lists the internal registry
as the default.

This PR makes the READMEs independent of everything a release changes,
so the release worker can drop its helm-docs step entirely
(sourcegraph/sourcegraph#17056):

- Every pinned image `defaultTag` gets `` # @default --
`<appVersion>@sha256:<digest>`, pinned by each release ``, and
`sourcegraph.image.repository` gets `# @default -- Sourcegraph's public
Docker Hub registry`, in the four charts the release updates. The two
executor images also get the `# -- ` description helm-docs needs before
it honors `@default`. The registry annotation deliberately avoids the
literal registry string, because the release replaces that string
everywhere in `values.yaml`, comments included.
- The migrator README uses a `<chart-version>` placeholder instead of
`{{ template "chart.version" . }}`.
- A new CI step fails if any chart README renders a pinned digest or
`index.docker.io/sourcegraph`. A new image added without an annotation
then fails on its own PR rather than on the next release branch. It is a
separate step so it doesn't conflict with #954, which rewrites the
existing helm-docs step.
- Published charts still list the exact values. Before `helm package`,
the release and insiders publish workflows run the new
`scripts/ci/render-published-readmes.sh`. It deletes the release-value
`@default` annotations and reruns `./scripts/helm-docs.sh`, so each
published README lists the same digests and registry as the
`values.yaml` packaged with it. Nothing is committed. Because the docs
are built from the commit being published, the README no longer depends
on a later docs commit landing first, which was the cause of the wrong
v8.0.0 README. The step is `continue-on-error`: if it fails (for
example, the helm-docs download from GitHub fails), the chart still
publishes with the committed placeholder README.

The tradeoff is that READMEs viewed in this repository show placeholders
instead of exact digests. The branch's `values.yaml` still has them, and
the README in the published chart (`helm show readme`) shows them in
full. The migrator's `helm upgrade` examples keep the `<chart-version>`
placeholder in the published chart too.

## How this was tested

- **Simulated release.** CI can't exercise a release, so I reproduced
one on a scratch worktree. A temporary test in the sourcegraph repo ran
the release worker's own `values.yaml` rewrite
(`dev/images.UpdateHelmManifestContent`) over the four charts the
release touches. It first rewrote them to the internal registry with new
tags and digests (the internal phase), then back to the public registry
(the promote phase). I bumped `Chart.yaml` `version`/`appVersion` with
the same regex as `UpdateChartVersions`, and ran
`./scripts/helm-docs.sh` after each phase.

  | | README lines changed after internal phase | after promote phase |
  |---|---|---|
  | `main` | 68 | 60 |
  | this branch | 0 | 0 |

`values.yaml` and `Chart.yaml` changed as expected in both runs. Zero
drift on this branch also shows that the release rewrite leaves the `#
@default --` comments intact.
- **Annotation coverage.** After regenerating, no chart README contains
a 64-character `@sha256:` digest or `index.docker.io/sourcegraph`. A
second `./scripts/helm-docs.sh` run leaves the tree clean.
- **New CI guard.** I ran the new step's command against both trees. It
exits 0 on this branch. On `main` it exits 1 and lists the offending
lines, for example `sourcegraph.image.repository` in
`charts/sourcegraph-executor/dind/README.md`.
- **Publish rendering, end to end.** In a scratch worktree I rebuilt a
promote branch the way a release would: internal phase, `Chart.yaml`
bump, promote phase. Buildkite's helm-docs check stays clean on that
tree. I then ran the publish job's steps with Helm v3.4.0, the version
the workflows pin: the new script, then both `helm package` loops. I
extracted each `.tgz` and compared its README with its packaged
`values.yaml`. The expected values were taken from `values.yaml`, not
from helm-docs output.

| Scenario | Digests in packaged `values.yaml` | Missing from packaged
README | Placeholders left | Registry row |
  |---|---|---|---|---|
| Release publish (4 charts) | 24 / 1 / 1 / 1 | 0 | 0 |
`"index.docker.io/sourcegraph"` |
| Insiders publish (4 charts) | 24 / 1 / 1 / 1 | 0 | 0 |
`"index.docker.io/sourcegraph"` |
| Render step forced to fail | 24 / 1 / 1 / 1 | all | all | placeholder;
packaging still succeeds |

For every chart, in all three scenarios, `helm template` of the packaged
chart matches the unmodified chart, after normalizing the migrator Job's
random name suffix. The script only touches comments and READMEs. The
`sourcegraph` chart has 25 annotated images but 24 distinct digests,
because `codeIntelDB` and `pgsql` share one. `shellcheck` is clean, and
`actionlint` reports the same three pre-existing `actions/checkout@v3`
findings as on `main`.
- **CI.** On the first commit, [Buildkite build
3049](https://buildkite.com/sourcegraph/deploy-sourcegraph-helm/builds/3049)
passed, including ":book: Verify helm-docs is up-to-date" and the new
":book: Verify READMEs do not render release-managed values" step. "Lint
and Test charts" also passed. The changes to `values.yaml` are comments
only, so rendered manifests are unchanged. The publish workflows only
run on pushes to `main`, `release/**` and `promote/release-**`, so this
PR's CI can't exercise them. The end-to-end run above stands in for
that.

The first real check of the full flow will be the next release once both
PRs are deployed.

Merge this, and backport it to every `X.Y.x` branch that still gets
patch releases (at least `8.0.x` and `8.1.x`), before the release worker
change deploys. Patch releases branch from `X.Y.x`, and a branch without
these annotations would fail the helm-docs check once the worker stops
regenerating READMEs.

Part of
[PLAT-876](https://linear.app/sourcegraph/issue/PLAT-876/find-a-permanent-fix-for-helm-docs-release-failures).

Co-authored-by: Warren Gifford <warrenbruceg@gmail.com>
@marcleblanc2 @ampagent
scripts/helm-docs.sh: add --check, fail only on README diffs
586b072 
The Buildkite "Verify helm-docs is up-to-date" step and AGENTS.md ran
helm-docs and then failed on any `git status` output, so a developer's
unrelated uncommitted edits failed the check locally. `--check` moves
that logic into the script and checksums charts/**/README.md before and
after helm-docs runs, so only a README that helm-docs rewrote fails it,
and it prints those paths.

The script is POSIX sh now, runs from the repository root regardless of
the caller's cwd, uses a six-X mktemp template, and removes its temp dir.
The pinned helm-docs version and cache location are unchanged.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18eb1-b712-75fe-8da7-00b7e605508e
Co-authored-by: Amp <amp@ampcode.com>
@marcleblanc2
marcleblanc2 force-pushed the marc/helm-docs-posix-sh branch from 800d3e4 to 586b072 Compare October 30, 2026 21:58
@marcleblanc2 marcleblanc2 changed the title CI: Make helm-docs.sh portable scripts/helm-docs.sh: add --check, fail only on README diffs Oct 30, 2026
@marcleblanc2
marcleblanc2 enabled auto-merge (squash) October 30, 2026 22:17
@marcleblanc2
Update agent instructions for helm docs regeneration
e447c67 
Clarified instructions for regenerating helm docs and committing changes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@DaedalusG DaedalusG Awaiting requested review from DaedalusG

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@marcleblanc2