Skip to content
19 changes: 18 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,23 @@ hops validate --help
hops xr --help
```

## Local workbench (happy path)

Multi-workspace local GitOps on the laptop control plane:

```bash
# once
hops local start

# daily
hops local up ./gitops/env/local

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Correct the worktree path.

This line uses ./gitops/env/local, while the documented layout and skills/claude/references/local-workbench.md use ./gitops/envs/local. Change the command to use ./gitops/envs/local.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@README.md` at line 93, Update the documented “hops local up” command to use
the correct worktree path “./gitops/envs/local” instead of “./gitops/env/local”,
matching the documented layout and local-workbench reference.

hops local status
hops local open
hops local down
```

Use `--name` for concurrent worktrees (`<name>` namespaces). Full guide: [skills/claude/references/local-workbench.md](skills/claude/references/local-workbench.md).

## Command Areas

`hops-cli` is organized into a few command groups:
Expand Down Expand Up @@ -593,7 +610,7 @@ Notes:
- Source-build mode intended for a local control plane because it depends on the local registry flow
- Runs `up project build` in `PATH` (defaults to current directory)
- Loads generated `.uppkg` artifacts from `<PATH>/_output`
- Pushes package images to the registry exposed at `localhost:30500`
- Pushes package images to the registry exposed at `127.0.0.1:30500` (IPv4; avoids Docker's `localhost` → `[::1]` path)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Document the backend-specific registry endpoint.

This line presents 127.0.0.1:30500 as the endpoint for every backend. The Dory section states that Docker runs inside Dory and uses {dory-k8s-ip}:30500 instead. Qualify this endpoint for the applicable backends, or state that the CLI selects it per backend.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@README.md` at line 613, Update the package-image registry description to
clarify that 127.0.0.1:30500 applies only to backends where Docker runs locally,
while the Dory backend uses {dory-k8s-ip}:30500; alternatively state that the
CLI selects the endpoint based on the backend.

- Applies Crossplane `Configuration` resources pointing at `registry.crossplane-system.svc.cluster.local:5000/...`
- Supports `--skip-dependency-resolution`
- `config install --repo <org/repo> [--reload]`
Expand Down
6 changes: 3 additions & 3 deletions bootstrap/drc/helm.yaml → bootstrap/providers/helm-drc.yaml
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Per-provider DeploymentRuntimeConfig for provider-helm (local dev).
# Per-provider DeploymentRuntimeConfig for local dev.
# Colocated with bootstrap/providers/helm.yaml (runtimeConfigRef: local-dev-helm).
#
# Each provider gets its OWN uniquely-named DRC + cluster-admin ServiceAccount.
# Providers must NOT share a DRC: a shared runtime config lets one provider's
# image/SA silently clobber the other's pod (and makes drift impossible to
# attribute). This mirrors the per-provider DRCs the
# crossplane-helm-provider-stack composes for remote clusters.
# attribute).
apiVersion: pkg.crossplane.io/v1beta1
kind: DeploymentRuntimeConfig
metadata:
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Per-provider DeploymentRuntimeConfig for provider-kubernetes (local dev).
# Per-provider DeploymentRuntimeConfig for local dev.
# Colocated with bootstrap/providers/kubernetes.yaml (runtimeConfigRef: local-dev-kubernetes).
#
# Each provider gets its OWN uniquely-named DRC + cluster-admin ServiceAccount.
# Providers must NOT share a DRC: a shared runtime config lets one provider's
# image/SA silently clobber the other's pod (and makes drift impossible to
# attribute). This mirrors the per-provider DRCs the
# crossplane-kubernetes-provider-stack composes for remote clusters.
# attribute).
apiVersion: pkg.crossplane.io/v1beta1
kind: DeploymentRuntimeConfig
metadata:
Expand Down
56 changes: 55 additions & 1 deletion renovate.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,60 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": [
"config:recommended"
"config:recommended",
":dependencyDashboard"
],
"labels": ["dependencies"],
"packageRules": [
{
"description": "Automerge patch updates that pass checks",
"matchUpdateTypes": ["patch"],
"automerge": true,
"matchManagers": ["custom.regex"]
},
{
"description": "Group local-start bootstrap Crossplane pins",
"groupName": "local-start-crossplane-bootstrap",
"matchManagers": ["custom.regex"],
"matchPackageNames": [
"crossplane/crossplane",
"crossplane-contrib/provider-helm",
"crossplane-contrib/provider-kubernetes"
]
},
{
"description": "Conventional commit prefix for bootstrap pin bumps",
"matchManagers": ["custom.regex"],
"matchDatasources": ["github-releases"],
"semanticCommitType": "chore",
"semanticCommitScope": "deps"
}
],
"customManagers": [
{
"customType": "regex",
"description": "Bootstrap Provider package tags (hops local start)",
"managerFilePatterns": [
"bootstrap/providers/**/*.yaml"
],
"matchStrings": [
"package:\\s*xpkg\\.crossplane\\.io/(?<depName>[^:]+):(?<currentValue>v?[0-9][^\\s]+)"
],
"datasourceTemplate": "github-releases",
"packageNameTemplate": "{{depName}}"
},
{
"customType": "regex",
"description": "Crossplane Helm chart version pin in hops local start",
"managerFilePatterns": [
"src/commands/local/start.rs"
],
"matchStrings": [
"const CROSSPLANE_CHART_VERSION: &str = \"(?<currentValue>[^\"]+)\";"
],
"datasourceTemplate": "github-releases",
"packageNameTemplate": "crossplane/crossplane",
"extractVersionTemplate": "^v(?<version>.*)$"
}
]
}
90 changes: 83 additions & 7 deletions skills/claude/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,24 @@
name: hops
description: |
Hops CLI and Crossplane platform toolkit. Use when working with hops commands,
Crossplane configuration packages, XR lifecycle workflows (observe/adopt/manage),
secrets management (SOPS + AWS Secrets Manager), or local control plane setup.
Crossplane configuration packages, provider install (local source or published),
local control plane / gitops workbench, XR lifecycle (observe/adopt/manage),
secrets (SOPS + AWS SM), or iterating on stacks from source with
`hops config install --path` / `hops provider install --path` and `--gitops`.
---

# Hops CLI

`hops` is a CLI for Crossplane development and XR lifecycle workflows. It manages local
control planes, configuration packages, secrets, and live infrastructure adoption.
control planes, configuration packages, providers, secrets, and live infrastructure adoption.

## Quick Reference

| Command area | Purpose |
|-------------|---------|
| `hops local` | Local Colima-based control plane setup |
| `hops config` | Build, install, and uninstall Crossplane configuration packages |
| `hops local` | Local CP (dory/colima/kind), gitops cluster/worktree, workbench |
| `hops config` | Install Configuration packages (published **or** source + gitops) |
| `hops provider` | Install/patch Providers (published **or** source + SemVer-safe tags + gitops) |
| `hops secrets` | SOPS encrypt/decrypt, sync to AWS Secrets Manager or GitHub |
| `hops vars` | Declarative non-secret config (e.g. GitHub Actions repo variables) |
| `hops xr` | Observe/adopt/manage/orphan existing infrastructure |
Expand All @@ -26,14 +29,87 @@ control planes, configuration packages, secrets, and live infrastructure adoptio

For detailed reference on each area, see the bundled references:

- [Config install modes and local/published switching](references/config-install.md)
- **[Local source packages & providers](references/local-source-packages.md)** — **read this** when developing configs/providers on a laptop CP
- [Config install modes and gitops](references/config-install.md)
- [Local workbench (gitops cluster / worktree)](references/local-workbench.md)
- [Local control plane setup](references/local-setup.md)
- [XR observe → adopt → manage workflow](references/xr-workflow.md)
- [Secrets management](references/secrets.md)
- [Vars management (declarative GH Actions variables)](references/vars.md)
- [Local control plane setup](references/local-setup.md)
- [Available stacks and XRs](references/stacks-and-xrs.md)
- [Debugging with kubectl](references/debugging.md)

---

## Local control plane + platform packages (dogfood)

```bash
hops local start --backend dory --gitops ./gitops/cluster
# bootstrap writes helm/k8s providers + ProviderConfigs (default) into the tree,
# then runs cluster gitops (apply + watch)

hops config install --repo hops-ops/psql-stack --version v0.9.1 \
--gitops ./gitops/cluster --local
hops config install --repo hops-ops/auth-stack --version v1.6.0 \
--gitops ./gitops/cluster --local

hops local gitops cluster ./gitops/cluster # watches by default; --once for CI
hops local gitops worktree ./gitops/envs/local --name dogfood
```

- **`--gitops`** materializes pins under the cluster tree (not one-shot-only kubectl)
- **`--local`** on config install scaffolds stack XRs with ProviderConfig **`default`**
- Backend preference is **user-local** (`~/.hops/local/backend`), not `.hops.yaml`

---

## Developing packages from source (must know)

When changing XRDs/compositions or a provider implementation on the local CP:

### Configuration packages (stacks)

```bash
hops config install --path xrs/stacks/k8s/auth --gitops ./gitops/cluster --local
# or --watch for rebuild-on-save
```

| Mode | Gitops writes |
|------|----------------|
| Published `--repo --version` | `packages/*.yaml` only |
| Source `--path` | `packages/*.yaml` **+** `imageconfigs/*` (required) |

Without ImageConfigs, source function packages will not pull from the local registry on gitops re-apply.

### Providers

```bash
# Cluster must already have the upstream Provider (e.g. from start)
hops provider install --path /path/to/provider-helm --gitops ./gitops/cluster
```

| Mode | `spec.package` | Gitops |
|------|----------------|--------|
| Published | real tag `v1.3.0` | `providers/` + `runtime/` |
| Source | **upstream URL** + **`vMAJOR.999.N`** (not bare `dev-sha`) | + `imageconfigs/` |

**Why `v1.999.N`:** Configuration deps use `>=vMAJOR`. Masterminds/semver **excludes prereleases**, so `dev-abc` or `v1.999.1-dev-sha` break dep resolution. ImageConfig rewrites **fetch** only; Lock source stays the upstream package path.

Optional: `--version-prefix v1` to force major for the 999 scheme.

### Auth-stack XR surface

| XR | When |
|----|------|
| **AuthStack** | Platform Zitadel install |
| **MachineUser** | Machine identity + optional PAT |
| **Grant** | Roles on a project (same/cross-org) — prefer over raw grant MRs |
| Provider MRs | Project, HumanUser, Oidc app — no thin 1:1 hops HumanUser wrapper |

Full detail: [local-source-packages.md](references/local-source-packages.md).

---

## Crossplane Conventions

- **Crossplane 2+**: Use `managementPolicies`, never `deletionPolicy` on managed resources
Expand Down
109 changes: 106 additions & 3 deletions skills/claude/references/config-install.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
# Config Install Reference

> **Developing stacks or providers from source on a local CP?** Start with
> [local-source-packages.md](./local-source-packages.md) — published vs source,
> ImageConfigs, and the provider `vMAJOR.999.N` SemVer rule.

## Two Install Modes

### Source-build mode (`--path` or `--repo`)
### Source-build mode (`--path` or `--repo` without `--version`)

Builds an Upbound-format XRD project locally and pushes through the local registry.
Intended for local control planes started with `hops local start`.
Expand Down Expand Up @@ -35,7 +39,7 @@ hops config install --path /path/to/project --watch
**What happens:**
1. Runs `up project build` to create `.uppkg` artifacts
2. Loads images via `docker load`
3. Pushes render function images to `localhost:30500` (local registry)
3. Ensures registry **host access** the same way workbench/gitops exposes services: map-mode `kubectl port-forward -n crossplane-system svc/registry 30500:5000` on `127.0.0.1` (PID under `~/.hops/local/`). Then pushes images to `127.0.0.1:30500`. No dory fork / NodePort publish required.
4. Creates ImageConfig rewrites so Crossplane pulls from the in-cluster registry
5. Patches the configuration package metadata with local render digests
6. Applies the Configuration resource
Expand All @@ -54,6 +58,101 @@ hops config install --repo hops-ops/aws-auto-eks-cluster --version v0.11.0
3. Deletes inactive ConfigurationRevisions pointing at the local registry
4. Applies Configuration with `ghcr.io/<org>/<repo>:<version>`

## Cluster gitops: `--gitops` (+ `--local`)

Prefer this when dogfooding a **local control plane** so package pins live in
gitops (not only as one-shot kubectl applies).

```bash
# 1. Bootstrap CP (creates helm/k8s ProviderConfigs named "default")
hops local start --backend dory

# 2. Install published stacks + write package YAML under gitops/cluster
hops config install --repo hops-ops/psql-stack --version v0.9.1 \
--gitops ./gitops/cluster --local

hops config install --repo hops-ops/auth-stack --version v1.6.0 \
--gitops ./gitops/cluster --local

# 3. Day-to-day: apply/watch the tree (packages + XRs)
hops local gitops cluster ./gitops/cluster
# or: hops local start --backend dory --gitops ./gitops/cluster
```

| Flag | Effect |
|------|--------|
| `--gitops PATH` | After install, write package gitops under `PATH` (see modes below) |
| `--local` | Requires `--gitops`. Also scaffolds local XR YAMLs when known (e.g. `psql/stack.yaml`, `auth/stack.yaml`) with **`helmProviderConfigRef` / `kubernetesProviderConfigRef` → `name: default`**. Does **not** overwrite existing XR files. |

### Published vs source under `--gitops`

| Mode | Writes |
|------|--------|
| **`--repo --version`** (published) | `packages/<name>.yaml` only (ghcr pin) |
| **`--path` / source** | `packages/<name>.yaml` (local registry `dev-*` ref, `packagePullPolicy: Always`) **and** `imageconfigs/*.yaml` (ImageConfig rewrites so Crossplane pulls render functions from the in-cluster registry) |

Source packages **need** those ImageConfigs in gitops. Without them, `hops local gitops cluster` re-apply cannot resolve function packages. Switching back to published: remove stale `imageconfigs/hops-local-rewrite-*` files (or re-run `--repo --version --gitops` and delete them manually for now).

### Why `--local` ProviderConfigs?

`hops local start` installs Helm + Kubernetes ProviderConfigs named **`default`**.
Stack XRs default provider config names from `spec.clusterName` (e.g. `dory`), which
does **not** exist unless you create matching ProviderConfigs.

Local XR scaffolds therefore pin:

```yaml
helmProviderConfigRef:
name: default
kubernetesProviderConfigRef:
name: default
```

`clusterName` remains a logical label (and may match your backend name).

### What gets written

```text
gitops/cluster/
packages/psql-stack.yaml # Configuration (published or local registry)
packages/auth-stack.yaml
imageconfigs/hops-local-rewrite-….yaml # source builds only
psql/stack.yaml # only with --local, if missing
auth/stack.yaml # only with --local, if missing
```

Re-apply anytime with `hops local gitops cluster ./gitops/cluster` (watches by default).

```bash
# Develop auth-stack XRs from source + keep gitops coherent
hops config install --path xrs/stacks/k8s/auth --gitops ./gitops/cluster --local
hops local gitops cluster ./gitops/cluster --once
```

## Provider install (`hops provider install`) + gitops

Same published vs source idea. **Source builds use a SemVer-compatible tag** so
Configuration deps like `>=v1` keep resolving:

| Mode | `spec.package` tag | Gitops extras |
|------|-------------------|---------------|
| **Published** `--repo --version` | real release (`v1.3.0`) | `providers/` + `runtime/` only |
| **Source** `--path` | **`vMAJOR.999.<N>`** (auto-increment) on the **upstream** URL prefix | + `imageconfigs/` rewrite to local registry |

Crossplane’s dep manager records the **upstream** package URL in the Lock;
ImageConfig only rewrites **fetch**. Never write a bare `dev-<sha>` tag into
gitops for providers that Configurations depend on — Masterminds/semver treats
prereleases specially and `>=v1` will not match.

```bash
# Bootstrap first so an upstream Provider exists to patch
hops local start --gitops ./gitops/cluster

# Iterate on provider-helm from source (preserves v1.999.N + ImageConfig in gitops)
hops provider install --path /path/to/provider-helm --gitops ./gitops/cluster
# or force major: --version-prefix v1
```

## Switching Between Local and Published

The CLI handles cleanup automatically when switching modes:
Expand All @@ -67,7 +166,9 @@ The CLI handles cleanup automatically when switching modes:
## Configuration Naming

Configurations are named `<org>-<repo>`, e.g. `hops-ops-aws-secret-stack`.
This matches both local and published installs.
This matches both local and published installs. Gitops package **filenames** use
the short package name (`psql-stack.yaml`); `metadata.name` matches the applied
Configuration.

## Uninstall

Expand All @@ -92,6 +193,8 @@ Functions, Providers) and ImageConfig rewrites.
| `--path` | Source build | Path to local XRD project |
| `--repo` | Both modes | GitHub `<org>/<repo>` |
| `--version` | Remote mode | Version tag (e.g. `v0.11.0`) |
| `--gitops PATH` | All modes | Write Configuration YAML under cluster gitops tree |
| `--local` | With `--gitops` | Scaffold local XRs using ProviderConfig `default` |
| `--watch` | Source build | Re-run install on filesystem changes |
| `--debounce` | Used with `--watch` | Quiet interval in seconds before rebuild (default 15) |
| `--skip-dependency-resolution` | All modes | Set `spec.skipDependencyResolution=true` |
Expand Down
Loading
Loading