feat: platform SecretStack with AWS and Vault backends - #28
Conversation
Break the API group to hops.ops.com.ai and select secrets backend via spec.backend (aws|vault). Gate PodIdentity and SM stores on aws; add optional Vault Helm install plus Vault SecretStore. Gate dependents on sticky helm revision existence rather than Ready. Dogfooded locally with hops config install --path against dory (ESO + Vault Releases Ready).
📝 WalkthroughWalkthroughThe PR renames SecretStack to a platform-neutral API and adds AWS or Vault backend selection. Rendering now supports optional Vault installation, backend-specific SecretStores, expanded status data, and updated validation scenarios. ChangesSecretStack backend expansion
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant SecretStack
participant StateInit
participant VaultRelease
participant SecretStore
SecretStack->>StateInit: provide backend configuration
StateInit->>VaultRelease: render optional Vault Helm Release
StateInit->>SecretStore: provide AWS or Vault provider state
VaultRelease->>SecretStore: expose observed Vault installation state
SecretStore->>SecretStack: render backend-specific store resources
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
tests/test-render/main.k (1)
172-181: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winTest revision-based gates when Helm is not Ready.
Both cases set
revision = 1andReady=True. A readiness-based gate would also render the SecretStore. Set the HelmReadycondition toFalsewhile retainingrevision = 1, then keep the SecretStore assertions. This validates the sticky revision contract.Proposed test change
conditions = [ - {type = "Ready", status = "True"} + {type = "Ready", status = "False"} {type = "Synced", status = "True"} ]Also applies to: 244-271
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test-render/main.k` around lines 172 - 181, Update the status fixtures in the affected test cases around the status assignment so Helm retains revision = 1 but its Ready condition is set to False. Keep the existing SecretStore assertions unchanged, ensuring both cases validate revision-based rendering independently of readiness.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apis/secretstacks/definition.yaml`:
- Around line 121-125: Add conditional x-kubernetes-validations to the
SecretStack schema so AWS requires spec.aws.region when backend=aws, Vault
requires spec.vault.server unless Vault installation is enabled, and external
Vault token authentication requires vault.auth.tokenSecretRef with an existing
reference rather than relying on the default name. Add negative admission/render
tests covering each incomplete configuration and preserve valid installed-Vault
and explicitly configured cases.
In `@functions/render/000-state-init.yaml.gotmpl`:
- Around line 75-83: Extend the vaultInstall branch in the state initialization
template to bootstrap the configured Kubernetes auth method before SecretStack
becomes Ready. Render the auth method, Kubernetes API configuration, and role
using vaultAuthMountPath, vaultAuthRole, and the configured service account, and
add readiness dependencies so SecretStack waits for these resources.
In `@functions/render/230-secret-store.yaml.gotmpl`:
- Around line 61-65: Update the conditional around tokenSecretRef namespace in
the Vault SecretStore template to add the namespace only when $ss.scope is
"Cluster". Remove the normalized namespace check and preserve the existing
defaulting and merge behavior within the ClusterSecretStore branch.
---
Nitpick comments:
In `@tests/test-render/main.k`:
- Around line 172-181: Update the status fixtures in the affected test cases
around the status assignment so Helm retains revision = 1 but its Ready
condition is set to False. Keep the existing SecretStore assertions unchanged,
ensuring both cases validate revision-based rendering independently of
readiness.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 1ec0a569-8263-470e-9b56-3e005d666171
📒 Files selected for processing (27)
.github/workflows/on-pr.yaml.github/workflows/on-push-main.yaml.gitignoreMakefileREADME.mdapis/secretstacks/composition.yamlapis/secretstacks/definition.yamlexamples/secretstacks/minimal.yamlexamples/secretstacks/standard.yamlexamples/secretstacks/vault-external.yamlexamples/secretstacks/vault.yamlexamples/test/mocks/observed-resources/standard/steps/1/helm-external-secrets.yamlexamples/test/mocks/observed-resources/vault/steps/1/helm-external-secrets.yamlexamples/test/mocks/observed-resources/vault/steps/1/helm-vault.yamlexamples/test/mocks/observed-resources/vault/steps/2/helm-external-secrets.yamlexamples/test/mocks/observed-resources/vault/steps/2/helm-vault.yamlexamples/test/mocks/observed-resources/vault/steps/2/secret-store.yamlfunctions/render/000-state-init.yaml.gotmplfunctions/render/010-state-status.yaml.gotmplfunctions/render/200-helm-release-external-secrets.yaml.gotmplfunctions/render/201-helm-release-vault.yaml.gotmplfunctions/render/210-aws-pod-identity.yaml.gotmplfunctions/render/230-secret-store.yaml.gotmplfunctions/render/999-status.yaml.gotmpltests/e2etest-secretstacks/main.ktests/test-render/main.kupbound.yaml
| description: AWS configuration. Required when backend=aws (region). | ||
| type: object | ||
| properties: | ||
| region: | ||
| description: AWS region for the EKS cluster. | ||
| description: AWS region for the EKS cluster and Secrets Manager. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect existing validation patterns and backend render guards.
rg -n -C 4 \
'x-kubernetes-validations|backend:|region:|server:|tokenSecretRef:' \
apis/secretstacks tests examples
rg -n -C 4 \
'state\.aws\.region|state\.vault\.server|tokenSecretRef' \
functions/renderRepository: hops-ops/aws-secret-stack
Length of output: 13382
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'definition.yaml relevant sections:\n'
sed -n '1,260p' apis/secretstacks/definition.yaml
printf '\nstate init template relevant sections:\n'
sed -n '1,160p' functions/render/000-state-init.yaml.gotmpl
printf '\nsecret store template lines:\n'
sed -n '1,90p' functions/render/230-secret-store.yaml.gotmpl
printf '\nGo template files mentioning validation errors or requirements:\n'
rg -n 'errorf|required|required value|x-kubernetes-validations|validation' .Repository: hops-ops/aws-secret-stack
Length of output: 20048
Reject incomplete backend configuration at admission.
spec.aws.region is optional, and the AWS SecretStore is omitted when it is empty. spec.vault.server is optional, and the Vault SecretStore is omitted when it is empty while not installing Vault. For external Vault + token auth, spec.vault.auth.tokenSecretRef is optional and may default only to name: vault-token, even though the referenced Secret is missing. Add conditional x-kubernetes-validations and negative admission/render tests for these cases.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apis/secretstacks/definition.yaml` around lines 121 - 125, Add conditional
x-kubernetes-validations to the SecretStack schema so AWS requires
spec.aws.region when backend=aws, Vault requires spec.vault.server unless Vault
installation is enabled, and external Vault token authentication requires
vault.auth.tokenSecretRef with an existing reference rather than relying on the
default name. Add negative admission/render tests covering each incomplete
configuration and preserve valid installed-Vault and explicitly configured
cases.
| {{- if eq $vaultAuthMethod "" }} | ||
| {{- if $vaultInstall }} | ||
| {{- $vaultAuthMethod = "kubernetes" }} | ||
| {{- else }} | ||
| {{- $vaultAuthMethod = "token" }} | ||
| {{- end }} | ||
| {{- end }} | ||
| {{- $vaultAuthMountPath := $vaultAuth.mountPath | default "kubernetes" }} | ||
| {{- $vaultAuthRole := $vaultAuth.role | default "external-secrets" }} |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find declarative Vault Kubernetes-auth bootstrap resources.
rg -n -C 3 \
'auth enable kubernetes|auth/kubernetes/config|auth/kubernetes/role|external-secrets' \
functions examples testsRepository: hops-ops/aws-secret-stack
Length of output: 25617
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Locate and inspect the relevant Vault templates and state defaults around auth installation and SecretStore rendering.
for f in functions/render/000-state-init.yaml.gotmpl functions/render/201-helm-release-vault.yaml.gotmpl functions/render/230-secret-store.yaml.gotmpl examples/secretstacks/vault.yaml examples/secretstacks/vault-external.yaml tests/test-render/main.k; do
echo "===== $f ====="
wc -l "$f"
rg -n -C 5 'vault|auth|kubernetes|external-secrets|auth enable|auth/kubernetes/config|auth/kubernetes/role|role:|method:' "$f" || true
doneRepository: hops-ops/aws-secret-stack
Length of output: 23128
🌐 Web query:
HashiCorp Vault Kubernetes auth method Kubernetes API configuration role
💡 Result:
To configure the HashiCorp Vault Kubernetes authentication method, you must perform two primary steps: configuring the authentication backend to communicate with the Kubernetes API and defining roles that map Kubernetes service accounts to Vault policies. Configuration of the Kubernetes Auth Backend The backend configuration informs Vault how to reach the Kubernetes API and how to validate service account tokens via the TokenReview API [1][2]. 1. Mount the Kubernetes auth method (if not already done): vault auth enable kubernetes [3] 2. Configure the backend using the /auth/kubernetes/config endpoint [1][2]. - kubernetes_host: The URL of the Kubernetes API server (e.g., https://:) [1][4]. - kubernetes_ca_cert: The CA certificate used to verify the Kubernetes API server's TLS certificate. If Vault is running inside a Kubernetes pod, omitting this will cause Vault to default to the pod's service account CA certificate [1][2]. - token_reviewer_jwt: A service account token with permission to access the TokenReview API. If Vault is running in a pod, omitting this will default to the pod's service account token [1][2]. Important: Ensure the service account used by Vault (or specified as the token_reviewer_jwt) has the system:auth-delegator ClusterRole assigned to allow it to perform TokenReview requests [5]. Role Configuration Roles define the mapping between Kubernetes service accounts and Vault policies [5]. You can create a role using the /auth/kubernetes/role/:name endpoint [6]. Key parameters for role creation: - bound_service_account_names: A list of Kubernetes service account names permitted to use this role [6][7]. - bound_service_account_namespaces: A list of Kubernetes namespaces allowed to use this role [6][7]. - token_policies: The list of Vault policies to assign to the token generated upon successful authentication [6][7]. - audience: An optional audience to verify in the JWT [6][7]. Example command: vault write auth/kubernetes/role/my-role \ bound_service_account_names="my-service-account" \ bound_service_account_namespaces="default" \ token_policies="my-policy" \ ttl=1h [5] These configurations ensure that Vault can securely verify incoming Kubernetes tokens and authorize them based on your defined security policies [2][5].
Citations:
- 1: https://docs.hashicorp.com/vault/api-docs/auth/kubernetes
- 2: https://developer.hashicorp.com/vault/docs/auth/kubernetes
- 3: https://developer.hashicorp.com/vault/docs/deploy/kubernetes/helm/examples/kubernetes-auth
- 4: https://registry.terraform.io/providers/hashicorp/Vault/latest/docs/resources/kubernetes_auth_backend_config
- 5: https://github.com/hashicorp/web-unified-docs/blob/main/content/vault/v1.21.x/content/docs/auth/kubernetes.mdx
- 6: https://developer.hashicorp.com/vault/api-docs/auth/kubernetes
- 7: https://registry.terraform.io/providers/hashicorp/vault/latest/docs/resources/kubernetes_auth_backend_role
🌐 Web query:
HashiCorp Vault Helm chart kubernetes auth enable auto_unseal kubernetes auth backend role
💡 Result:
To manage HashiCorp Vault on Kubernetes using the Helm chart, you must configure three distinct areas: Auto-Unseal, the Kubernetes Auth Method, and the Kubernetes Auth Backend Role [1][2][3]. Auto-Unseal Configuration Auto-unseal is configured by providing a seal stanza in the Vault configuration block within your values.yaml file [4][3]. This requires you to specify the cloud provider KMS or HSM service (e.g., AWS KMS, Google Cloud KMS) [1][4]. Example for values.yaml: server: ha: enabled: true config: | seal "gcpckms" { project = "your-project" region = "global" key_ring = "your-key-ring" crypto_key = "your-key" } # Other config... Kubernetes Auth Method You must enable and configure the Kubernetes Auth Method within Vault after the cluster is running [5][6]. This is typically done via the Vault CLI or API [6]. 1. Enable the method: vault auth enable kubernetes 2. Configure the connection to Kubernetes: vault write auth/kubernetes/config \ kubernetes_host=https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT You can also use the Helm chart's server.authDelegator.enabled=true setting to automatically bind the necessary ClusterRole for the Vault service account to perform token reviews [7][8]. Kubernetes Auth Backend Role Once the auth method is enabled, you create roles that map Kubernetes service accounts to Vault policies [6][9]. This can be managed via the CLI or Terraform [6][9]. Example using the Vault CLI: vault write auth/kubernetes/role/my-app-role \ bound_service_account_names=my-app-sa \ bound_service_account_namespaces=default \ policies=my-app-policy \ ttl=1h For automated infrastructure management, you can use the vault_kubernetes_auth_backend_role resource in Terraform [9]. The role defines which Kubernetes service accounts and namespaces are authorized to authenticate with Vault and what policies they receive [6][9].
Citations:
- 1: https://developer.hashicorp.com/vault/docs/deploy/kubernetes/helm/run
- 2: https://developer.hashicorp.com/vault/docs/deploy/kubernetes/helm
- 3: https://developer.hashicorp.com/vault/tutorials/kubernetes/kubernetes-raft-deployment-guide
- 4: https://github.com/hashicorp/vault-helm/blob/main/values.yaml
- 5: https://developer.hashicorp.com/vault/docs/deploy/kubernetes/helm/examples/kubernetes-auth
- 6: https://developer.hashicorp.com/vault/docs/auth/kubernetes
- 7: https://developer.hashicorp.com/vault/docs/platform/k8s/helm/configuration
- 8: https://docs.hashicorp.com/vault/docs/deploy/kubernetes/helm/configuration
- 9: https://registry.terraform.io/providers/hashicorp/vault/latest/docs/resources/kubernetes_auth_backend_role
Bootstrap the Kubernetes auth method when rendering the installed Vault.
vault.install=true renders Vault before 230-secret-store.yaml.gotmpl creates a Vault ClusterSecretStore with Kubernetes auth. Add the Vault Kubernetes auth method, Kubernetes API configuration, and role for the configured service account, and keep the SecretStack from becoming Ready until those resources exist.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@functions/render/000-state-init.yaml.gotmpl` around lines 75 - 83, Extend the
vaultInstall branch in the state initialization template to bootstrap the
configured Kubernetes auth method before SecretStack becomes Ready. Render the
auth method, Kubernetes API configuration, and role using vaultAuthMountPath,
vaultAuthRole, and the configured service account, and add readiness
dependencies so SecretStack waits for these resources.
| {{- /* ClusterSecretStore tokenSecretRef needs namespace */}} | ||
| {{- if or (eq $ss.scope "Cluster") $state.vault.auth.tokenSecretRef.namespace }} | ||
| {{- $tokenNs := $state.vault.auth.tokenSecretRef.namespace | default $state.namespace }} | ||
| {{- $tokenRef = merge $tokenRef (dict "namespace" $tokenNs) }} | ||
| {{- end }} |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== locate file =="
fd -a '230-secret-store.yaml.gotmpl' . || true
echo "== file snippet =="
file="$(fd '230-secret-store.yaml.gotmpl' . | head -n1)"
if [ -n "${file:-}" ]; then
wc -l "$file"
sed -n '1,140p' "$file" | nl -ba
fi
echo "== state/init references to vault.auth.tokenSecretRef =="
rg -n "tokenSecretRef|vault\.auth|namespace" -S . --glob '!node_modules' --glob '!dist' --glob '!build' | head -n 200Repository: hops-ops/aws-secret-stack
Length of output: 358
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== file snippet =="
sed -n '1,180p' functions/render/230-secret-store.yaml.gotmpl
echo "== state/init references to vault.auth.tokenSecretRef =="
rg -n "tokenSecretRef|vault\.auth|namespace" -S . --glob '!node_modules' --glob '!dist' --glob '!build' | sed -n '1,240p'Repository: hops-ops/aws-secret-stack
Length of output: 12436
🌐 Web query:
External Secrets Operator Vault provider tokenSecretRef namespace ClusterSecretStore SecretStore tokenSecretRef schema
💡 Result:
In the External Secrets Operator (ESO), the tokenSecretRef field is used within the Vault provider configuration to authenticate using a static Vault token stored in a Kubernetes Secret [1][2]. The schema and requirements for this field differ depending on whether you are using a namespaced SecretStore or a cluster-wide ClusterSecretStore [2][3]. Schema for tokenSecretRef The tokenSecretRef block requires the following fields: - name: The name of the Kubernetes Secret containing the Vault token [1][4]. - key: The specific key within that Kubernetes Secret that holds the token value [1][4]. - namespace: (Mandatory for ClusterSecretStore) The namespace where the Kubernetes Secret resides [1][2]. For a namespaced SecretStore, this field is typically omitted because the operator looks for the secret in the same namespace as the SecretStore [2][3]. Usage Examples For a SecretStore (Namespaced): In a namespaced SecretStore, the operator expects the referenced secret to exist in the same namespace as the SecretStore [2][3]. spec: provider: vault:... auth: tokenSecretRef: name: "my-vault-token-secret" key: "token" For a ClusterSecretStore (Cluster-wide): In a ClusterSecretStore, you must explicitly define the namespace where the secret is located, as the store is cluster-scoped and cannot rely on implicit namespace matching [1][2]. spec: provider: vault:... auth: tokenSecretRef: name: "my-vault-token-secret" namespace: "secret-admin" key: "token" Key Considerations: - ClusterSecretStore: Because ClusterSecretStore is global, failing to specify the namespace for tokenSecretRef will cause the authentication to fail, as the operator will not know where to look for the secret [2][5]. - Security: Always ensure that the namespace containing the secret is appropriately restricted to maintain secure access to the Vault token [3].
Citations:
- 1: https://external-secrets.io/latest/api/clustersecretstore/
- 2: https://external-secrets.io/latest/provider/hashicorp-vault/
- 3: https://external-secrets.io/v2.6.0/introduction/overview/
- 4: https://external-secrets.io/latest/api/secretstore/
- 5: https://external-secrets.io/main/api/clustersecretstore/
Omit tokenSecretRef.namespace for a namespaced SecretStore.
State initialization always normalizes vault.auth.tokenSecretRef.namespace to external-secrets. That causes this branch to add tokenSecretRef.namespace to every Vault SecretStore token reference. ESO uses tokenSecretRef.namespace for ClusterSecretStore; namespaced SecretStore references are scoped to the SecretStore namespace.
Only add namespace when $ss.scope is Cluster.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@functions/render/230-secret-store.yaml.gotmpl` around lines 61 - 65, Update
the conditional around tokenSecretRef namespace in the Vault SecretStore
template to add the namespace only when $ss.scope is "Cluster". Remove the
normalized namespace check and preserve the existing defaulting and merge
behavior within the ClusterSecretStore branch.
Published Crossplane PackageThe following Crossplane package was published as part of this PR: Package: ghcr.io/hops-ops/aws-secret-stack:pr-28-c66547c7731fe39477be6bb7b404f58bf50a51b9 |
1 participant
Summary
hops.ops.com.aiand package name tosecret-stackspec.backend: aws|vaultwith optional Vault Helm installrevisionexistence (not Ready)Dogfood
Installed via
hops config install --pathon dory; claim withbackend: vaultproduced Ready Helm Releases for external-secrets + vault and ClusterSecretStore Object.Test plan
Summary by CodeRabbit
New Features
Documentation
Tests