fix(deployer): refuse a hot upgrade built on a different toolchain - #271
Closed
thiagoesteves wants to merge 1 commit into
Closed
fix(deployer): refuse a hot upgrade built on a different toolchain#271thiagoesteves wants to merge 1 commit into
thiagoesteves wants to merge 1 commit into
Conversation
A hot upgrade replaces application code inside the running VM, it cannot replace the VM or the language runtime under it. systools:make_relup/4 needs an appup for every application whose version changes, and neither Elixir nor the OTP applications ship one. Applying the OTP 28 artifact to an installation running the OTP 27 build therefore fails partway through, after the release has been unpacked, with Could not open file /opt/deployex/lib/elixir-1.19.5/ebin/elixir.appup systools:make_relup failed, reason: :error which says nothing about the actual mistake, that the wrong file was chosen. check/1 already extracts the release before validating it, so both versions are on disk and answerable before anything is installed. It now compares the Elixir and ERTS versions in the release against the running system and refuses when either differs, naming both sides. The UI reports which versions changed rather than "invalid release". A release whose layout cannot be read is left to the checks that follow, which report it better than a guess would. Risk assessment Impact: choosing the wrong artifact is rejected at the point of choosing it, with a message naming the versions. An artifact built on the same toolchain behaves as before. Blast radius: the deployex self upgrade check and the error branch that displays it. The monitored application path is untouched, it has its own check. Regression risk: low. The new step only rejects an upgrade that would have failed at make_relup anyway, and an unreadable layout falls through rather than blocking. Rollback: plain commit revert.
Owner
Author
|
Superseded: the check was specific to Elixir and ERTS. Replaced by a single OTP comparison against the erts the release bundles. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The failure this prevents
Applying the OTP-28 artifact to an installation running the OTP-27 build:
Nothing there says the wrong file was chosen. It surfaces two steps after the mistake, as a missing file inside
systools.Why it cannot work
A hot upgrade replaces application code inside the running VM. It cannot replace the VM or the language runtime beneath it.
systools:make_relup/4demands an.appupfor every application whose version changes between the two releases, and neither Elixir nor the OTP applications ship one - so any toolchain difference is fatal by construction, not by accident.For DeployEx that maps directly onto
otp_version:The check
check/1already extracts the release before validating it, so both versions are sitting on disk. It now compares them against the running system:and refuses when either differs, naming both sides:
The UI shows
wrong artifact for this installation: elixir 1.18.4 -> 1.19.5instead ofinvalid release.Only the versions that actually differ are reported - an ERTS-only change reports ERTS alone.
Deliberately permissive in one place
A release whose layout cannot be read (no
lib/elixir-*, noerts-*) falls through rather than being rejected. The checks that follow report a malformed release far better than a guess here would, and refusing on a failedPath.wildcardwould turn an unrelated packaging problem into a confusing toolchain error.Risk assessment
Impact: choosing the wrong artifact is rejected at the point of choosing it, naming the versions. An artifact built on the same toolchain behaves exactly as before.
Blast radius: the DeployEx self-upgrade check and the error branch that displays it. The monitored-application path is untouched - it has its own check.
Regression risk: low. The new step only rejects an upgrade that would have failed at
make_relupanyway, and an unreadable layout falls through rather than blocking.Rollback: plain commit revert.
Checks
Full umbrella suite green (foundation 236 + 25 doctests, host 21, deployer 174, sentinel 84, deployex_web 177 + 6 doctests),
mix credo --strictclean,mix format --check-formattedclean, compiles with--warnings-as-errors.Two new tests: one builds a release layout with a different Elixir and the same ERTS - exactly the shape of the real failure - and asserts only
elixiris reported; the other builds a matching layout and asserts it passes through.Not addressed here
The other half of that incident:
Deployex.execute/2loggedHot upgrade in deployex installed with successthree seconds before the unpack started, because withsync_execution: falseit logs on the cast returning:okrather than on the upgrade completing. A failed self-upgrade still reports success. Worth its own PR.🤖 Generated with Claude Code