Skip to content

config: substitute with Hoplite's resolvers, not preprocessors - #74

Open
rocketraman wants to merge 1 commit into
masterfrom
rg/hoplite-resolvers
Open

rocketraman wants to merge 1 commit into
masterfrom
rg/hoplite-resolvers

Conversation

@rocketraman

Copy link
Copy Markdown
Owner

hopliteConfigBinder substituted config values with Hoplite's preprocessors, which Hoplite has deprecated in favour of resolvers since 2023, and which its maintainer has confirmed are the intended default. Preprocessors expand ${...} across the whole loaded tree and then fail on any that's left, so an exported shell variable such as PS1 broke loading (sksamuel/hoplite#602), which Bootable worked around by filtering such variables out of the environment source.

Resolvers resolve a value only when it's bound, so the filter is no longer needed.

Changes

  • hopliteConfigBinder adds Hoplite's default resolvers instead of its default preprocessors.
  • ${{ env:name }} reads the environment that hopliteConfigBinder's environmentVariables supplies, as the environment variables source does, rather than System.getenv as Hoplite's own EnvVarContextResolver does.
  • isKnownNonConfigEnvVar and its environment filter are removed.
  • Hoplite's module resolvers (Vault, AWS, GCP, Azure, Consul) can now be added in the withConfigStageN hooks. addResolver used to throw, because Bootable had added preprocessors.

Breaking: config substitution syntax

The syntax changes from ${VAR} / {{key}} to ${{ env:VAR }}, ${{ env:VAR:-default }}, ${{ sysprop:name }}, ${{ ref:key }}, and ${{ random:int }}. The old syntax isn't an error: it's left in the value as literal text. That includes ${VAR} inside a quoted HOCON string, which preprocessors used to expand.

HOCON's own substitutions (unquoted ${VAR}, ${?VAR}, ${a.b}) are unaffected, since HOCON resolves them when it parses the file. They're the recommended way to reference environment variables.

See the "Config substitutions use Hoplite's resolvers" section of CHANGELOG.adoc under 3.1.0 for the full mapping table and migration notes. Bootable's own .conf files use no substitutions, so nothing in the repo needed migrating.

Known gaps

  • Resolvers resolve only a data-class field whose value is a single string. A ${{ ... }} inside a map, list, or set (dataSourceProperties, Flyway's placeholders/locations/schemas, telemetry's otlp.headers/resourceAttributes) is left as literal text; use HOCON's ${?VAR} there.
  • There's no resolver for AWS Parameter Store's ${ssm:key} / paramstore://. Module prefixes change too: awssm:// becomes aws-secrets-manager://, azurekeyvault:// becomes azure-key-vault://, and gcpsm:// becomes gcp-secrets-manager://.
  • data class default values don't apply after resolvers sksamuel/hoplite#460: a resolver that yields nothing fails the bind instead of falling back to the data-class default, so optional values need :-fallback.
  • The config report shows the unresolved expression (${{ env:DB_HOST }}), not the value it resolves to.

Docs

  • boot-config-hoplite/README.adoc has a new Substitutions section.
  • The root README.adoc's Hoplite section points to it.

Tests

HopliteConfigBinderTest covers:

  • PS1 and BASH_FUNC_ml%% with ${...} in their values don't fail binding (the gh-602 case).
  • ${{ env:X }} resolves into a string field, an Int field, and a Masked field. Unset, it fails the bind, and with :-fallback it gives the fallback.
  • ${{ ref:key }} resolves.
  • Old ${X} in a quoted string is left as literal text.
  • HOCON's ${a.b}, ${PATH}, and ${?UNSET} resolve, in strings and in a Map<String, String>.
  • The known gap: ${{ env:X }} in a map value is passed through literally. The test fails if Hoplite starts resolving it.

Put back on preprocessors, 6 of the new tests fail. ./gradlew test passes across the build, including the MySQL and PostgreSQL Testcontainers suites.

Hoplite has deprecated preprocessors in favour of resolvers since
2023. Resolvers resolve a value only when it is bound, so `${...}` in
exported shell variables such as PS1 no longer fails loading, and the
isKnownNonConfigEnvVar filter is removed.

- `${{ env:name }}` reads the environment that hopliteConfigBinder's
  environmentVariables supplies, rather than System.getenv as
  Hoplite's EnvVarContextResolver does.
- Test resolvers, HOCON's own substitutions, and pin down that
  resolvers leave `${{ ... }}` in map values unresolved.
- Document the syntax change and its gaps in the CHANGELOG and the
  module README.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant