Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/env.md
Original file line number Diff line number Diff line change
Expand Up @@ -462,7 +462,7 @@ global Hack home, in its "secrets hygiene" section. It names files, never values
- an encrypted store whose key file is missing: "store present, key missing, cannot be
read".

`hack doctor --fix` sets the files it reports to mode 0600, adds missing ignore
`hack doctor --fix` removes group and other permission bits from eligible files it reports, adds missing ignore
entries, and removes Hack's own leftovers, and prints each change. It never deletes a
store, a key or a file you wrote, and never untracks a file.

Expand Down
61 changes: 45 additions & 16 deletions docs/guides/upgrade-to-v5.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,9 @@ Keep these manual checks too:

- Check ownership and permissions of `.hack/hack.env.local.yaml` and
`.hack/hack.env.<name>.local.yaml`. New local layers are created mode 0600,
but existing layers keep their modes. Keep local layers owner-only and do
not assume an upgrade tightened old files.
but next.16 keeps existing modes. The merged #764 change, intended for
next.17, tightens a local layer on a write and lets Doctor fix loose files
without waiting for that write. Keep local layers owner-only.
- Check Git's tracked-file list for `.hack/.env.<name>` and
`hack.env*.local.yaml`. next.16's managed ignore block omitted legacy overlay
dotenv names; newer ignore rules still cannot untrack an existing file.
Expand All @@ -106,9 +107,10 @@ keys, config and YAML inputs are not cleanup targets, including filesystem
aliases, and cleanup refuses a request that names one. It cannot recover a key already removed by an older
build. Back up original keys and env state before migration on any version.

The broader Doctor secrets-hygiene report and repairs (HACK-2165, #764) are a
separate change; check the chosen release's availability. next.16 already
reports `env-key-weak-permissions` and suggests `chmod 600` for an old loose key.
Merged #764 (HACK-2165), intended for next.17, adds the
[Doctor secrets-hygiene report and fixes](#after-upgrading-check-secrets-at-rest)
below. next.16 already reports `env-key-weak-permissions` with a `chmod 600`
remedy, but does not include the broader hygiene report.

See [Env and secrets](../env.md#legacy-repos-and-migration) and the
[threat model](../security/env-secrets-threat-model.md). These are manual checks,
Expand All @@ -120,23 +122,50 @@ Earlier versions could leave env values and keys more exposed than v5 does:
machine-local env files created by hand or by an older release with mode 0644,
staging copies of `.hack/.env` or the project key under names git does not
ignore, env values staged for a terminal session that never ran, or an
encrypted secret store whose key file is gone. After upgrading, run this once
encrypted secret store whose key file is gone. In a build containing #764, run this once
in each project, and once outside any project for the global Hack home:

```bash
hack doctor # read the "secrets hygiene" section
hack doctor --fix # apply its safe fixes
hack doctor --fix # apply the fixes after reviewing the report
```

The "secrets hygiene" section names each file and what it holds, never a value.
`hack doctor --fix` sets loose env layers, `.hack/.env` files, keys and stores to
mode 0600, adds a missing ignore entry for a secret-bearing file, and removes
staging copies and terminal-session payloads that Hack left behind. It prints
each change. It never deletes a store, a key or a file you wrote, and it does not
untrack a file git already tracks: the section names the `git rm --cached`
command, and what that file held should be treated as exposed. A store reported
as "store present, key missing, cannot be read" stays where it is: restore its
key file to read it, or remove the store yourself once you no longer need it.
The "secrets hygiene" section reports loose modes, links, tracked or unignored
secret-bearing files, staging leftovers and stale terminal-session payloads.
It names files and what they hold, never values. `hack doctor --fix` removes
group and other permission bits from eligible files owned by this user, adds
missing ignore entries inside the project, and removes Hack's staging leftovers
and terminal payloads older than ten minutes. It prints each change.

Removal never includes a live input: the configured store or key, project/shared
keys, config and env layers are protected by their paths and resolved paths.
Doctor rechecks the configuration before each removal and leaves the file when
it cannot prove removal is allowed. Retired stores are kept too. It never deletes
a store, a key or a file you wrote, and never untracks a file git already tracks.
Review any suggested `git rm --cached` and treat earlier shared copies as exposed.
For a store whose key is lost, use the restore-or-retire choices below, not hygiene
cleanup. Keep these repairs separate from `--migrate-env-config` and its backup step.

### An old global secret store whose key is gone

Merged #779, intended for next.17, makes an old store with a missing or wrong
key a Doctor error only where the project's runtime reads it. Other projects
receive a warning. Choose one: restore the original key from a private backup,
or retire the store if the key is gone for good. A new key cannot open the old
entries.

`hack secrets retire-store --path <project>` selects the store for that project
and moves it into a timestamped directory beside it. It deletes nothing, leaves
the key file alone and prints how to put the encrypted file back. Retirement
does not decrypt or print entries. Doctor then lists the retired file as
information.

Retiring requires `--allow-unchecked` because another shell may hold the key in
`HACK_SECRETS_FILE_KEY`, which Hack cannot check. Pass it only after confirming
that no shell for a project selecting the store has that key; it also acknowledges
registered projects Hack could not check. Without a terminal, add `--yes` to
confirm the move. A store that an available key opens is refused. See
[the recovery and retirement details](../env.md#compatibility-notes).

## The daemon after an upgrade

Expand Down
26 changes: 21 additions & 5 deletions docs/guides/v5-release-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ its original behavior.
| **Env listings hide values by default, including migrated plaintext.** Ordinary `env` / `env list` text and JSON output are metadata. JSON omits `value` and reports `has_value`. | Use `has_value` for availability checks. For a value a script actually needs, call `hack env get KEY` with the same `--env` and `--service`; that command prints plaintext. Bulk `--show-values` requires interactive stdin and stdout; `--show-secrets` remains an alias. |
| **Migration cleanup preserves live keys (HACK-2172).** A legacy store's `keyPath` can no longer make the current project/shared key a cleanup target. Current config and YAML inputs are protected too, including filesystem aliases. | Keep key and env backups before migration. On next.16 or an earlier prerelease, do not run `doctor --migrate-env-config` without a private readable copy of `.hack.secret.key`. The fix does not recover a key already removed. |
| **Doctor fails when an env layer cannot be used by the runtime.** Unreadable, invalid or undecryptable layers no longer get an apparent all-clear. | Run `hack doctor` after upgrading. Fix the reported layer or recover the original key before starting; keep the encrypted file rather than generating a replacement key over it. |
| **Doctor reports secrets hygiene (#764).** The report covers loose modes, staging leftovers and stale terminal payloads, naming files rather than values. | Review `hack doctor`, then use `hack doctor --fix` to tighten eligible files, add missing project ignore entries and remove stale Hack artifacts. Removal excludes live inputs and retired stores; it rechecks configuration before each removal. |
| **An old global secret store with a lost key no longer fails Doctor in unrelated projects (HACK-2177).** It is an error where the runtime reads it and a warning elsewhere. | Restore the original key, or deliberately retire the store with `hack secrets retire-store --path <project>`. Retirement preserves the encrypted file and prints how to put it back; see the confirmation requirements below. |
| **Doctor checks the Compose proxy's loopback ports.** A reachable container address no longer hides a broken published loopback path. | Read the `proxy ports` warning, check competing listeners and DNS/bind settings, then retry. HTTP success on port 80 is not HTTPS success on 443; the check applies no automatic repair. |
| **Native route readiness can be inferred.** A complete standard Caddy route with no authored healthcheck gets a reviewed HTTP probe and 240 seconds of initial grace. A complete HTTP 200–499 response establishes endpoint readiness, not application correctness. | Review the route port and startup logs. Explicit healthchecks are not replaced, and inferred readiness does not satisfy `depends_on: condition: service_healthy`. The grace does not extend the whole graph's deadline. |
| **Native start waits for bridge sockets after a stopped pool resumes.** An immediate start no longer fails merely because provider readiness preceded the bridge listener. | Use normal native `up`; startup still requires a successful bridge connection within its deadline. Do not replace readiness checks with a fixed sleep. |
Expand Down Expand Up @@ -77,11 +79,25 @@ proof that nothing changed. See [cleanup and reconciliation](../env.md#legacy-re
[#768](https://github.com/hack-dance/hack/pull/768) and
[#783](https://github.com/hack-dance/hack/pull/783).

An old global secret store whose key is lost still needs its original key for
recovery. Keep the encrypted store and key backups; do not delete it or provision
a new key just to clear Doctor. A separate retire path is being added in
[#779](https://github.com/hack-dance/hack/pull/779); no retire command is promised
by these notes until it merges.
### An old global store whose key is lost

Merged [#779](https://github.com/hack-dance/hack/pull/779), intended for next.17,
makes a missing or wrong store key a Doctor error only in projects whose runtime
reads that store. Other projects receive a warning. The finding names registered
projects that read it; without the key, entries cannot be counted or read.

Choose one: restore the original key, or, if it is gone for good, run
`hack secrets retire-store --path <project>` for a project that selects the store.
Retirement moves the encrypted file into a timestamped directory beside it,
deletes nothing, leaves the key file alone and prints the command to put it back.
It does not recover the values; a new key cannot decrypt them.

The command refuses a store that an available key opens. It also needs
`--allow-unchecked`: another shell may set `HACK_SECRETS_FILE_KEY` to the key,
and Hack cannot inspect that shell. Pass the flag only after confirming that no
shell for a project selecting the store has that key; it also acknowledges
registered projects Hack could not check. Without a terminal, add `--yes` for
the move confirmation. See [store recovery and retirement](../env.md#compatibility-notes).

## Highlights

Expand Down
50 changes: 35 additions & 15 deletions docs/versions/v5/guides/upgrade-from-v4.md
Original file line number Diff line number Diff line change
Expand Up @@ -365,12 +365,16 @@ HACK_RUNTIME_BACKEND=compose "$V5" doctor --path /absolute/project
An existing project key can retain group or other access from an older version.
next.16 Doctor already reports `env-key-weak-permissions` and suggests
`chmod 600` for the key. Review that finding and keep the backup private too.
The broader secrets-hygiene report and fixes are tracked in
[#764](https://github.com/hack-dance/hack/pull/764); those are not a next.16
guarantee. Once using a release
that contains them, review Doctor's findings before applying its fixes. The
hygiene repair is separate from `--migrate-env-config` and does not remove the
key-backup requirement above.
Merged [#764](https://github.com/hack-dance/hack/pull/764), intended for next.17,
adds Doctor's `secrets hygiene` report: loose modes, tracked or unignored files,
staging leftovers and terminal payloads older than ten minutes. It names files,
never values. After reviewing the report, `doctor --fix` tightens eligible files'
permissions, adds missing project ignore entries and removes only Hack leftovers
that are not live inputs. It rechecks configuration before removal; current
stores, keys, config and env layers, and retired stores, stay in place. It never
untracks files. The hygiene repair is separate from `--migrate-env-config` and
does not remove the key-backup requirement above. next.16 does not have this
broader report. Source: [hygiene inspection and repair](https://github.com/hack-dance/hack/blob/25bd63c19f4d6aa152ecb9bda88086364a1bd4fd/src/lib/secrets-hygiene.ts).

In the next.16 trials, most real env entries were stored as plain values.
`env list --json` still prints those values: JSON is formatting, not redaction.
Expand All @@ -387,8 +391,9 @@ output contract or assume the protection command exists in next.16.
#### An old global secret store whose key is gone

A machine upgraded from v4 may still have an encrypted store such as
`~/.hack/secrets.enc.json` whose key file was lost. From next.17 (HACK-2169,
HACK-2177), Doctor reports that store as a `secret integrity` finding:
`~/.hack/secrets.enc.json` whose key file was lost. Merged
[#779](https://github.com/hack-dance/hack/pull/779), intended for next.17,
changes Doctor's `secret integrity` finding for that store:

- an **error** in a project whose legacy env contract reads the store, and a
**warning** in any other project;
Expand All @@ -402,8 +407,9 @@ entries.
600), from a backup or another machine, or set `HACK_SECRETS_FILE_KEY` to the
same key.
2. **Retire the store** when the key is gone for good:
`hack secrets retire-store`, run in a project that selects the store or
outside any project for the global one. Without a terminal, add `--yes`.
`hack secrets retire-store --path <project>` selects that project's store;
outside any project, omitting `--path` selects the global one. Without a
terminal, add `--yes`.
- It moves the store into a new directory `<store>.retired-<UTC time>/`
beside it, keeping its name. It deletes nothing, reads and prints no
entry, and leaves the key file alone. It only retires a file that is a
Expand All @@ -418,6 +424,21 @@ entries.
- Afterwards Doctor lists the retired file as information, and the project
check no longer fails.

For example, after those key and project checks, use the selected candidate
with the same Compose home as the Doctor finding:

```sh
HACK_RUNTIME_BACKEND=compose "$V5" secrets retire-store --path /absolute/project --allow-unchecked
```

Review the store path in the confirmation prompt. Retirement keeps the encrypted
entries for recovery; it does not make them readable with a replacement key.
It is separate from migration cleanup and is not available in next.16.

Sources: [Doctor's store finding](https://github.com/hack-dance/hack/blob/91c311e4d9540ea3ca32113a61458a26db90c704/src/lib/doctor-secret-integrity.ts),
[retirement command and confirmation](https://github.com/hack-dance/hack/blob/91c311e4d9540ea3ca32113a61458a26db90c704/src/commands/secrets.ts),
[retirement checks](https://github.com/hack-dance/hack/blob/91c311e4d9540ea3ca32113a61458a26db90c704/src/lib/secret-store-retire.ts).

Sources for next.16 behavior: [env list and get](https://github.com/hack-dance/hack/blob/b21c3b189841c1a3cbb8af2b76414fa347dc7057/src/commands/env.ts),
[project key checks](https://github.com/hack-dance/hack/blob/b21c3b189841c1a3cbb8af2b76414fa347dc7057/src/lib/doctor-env-key.ts).

Expand All @@ -444,10 +465,9 @@ With [#783](https://github.com/hack-dance/hack/pull/783), a confirmed store
removal drops its legacy config pointer even if a later cleanup step fails.
If the store remains, the pointer stays. A config-write failure is reported too,
so follow the remaining reconciliation guidance rather than assuming rollback.
An old global store with a missing key should stay intact until its key is
recovered or the separate retirement path in
[#779](https://github.com/hack-dance/hack/pull/779) is available. Do not discard
the store or replace its only key to silence Doctor.
For an old global store with a missing key, use the separate
[restore-or-retire choices](#an-old-global-secret-store-whose-key-is-gone) above.
Do not discard the store or replace its only key to silence Doctor.

Source: [current env migration and cleanup guidance](https://github.com/hack-dance/hack/blob/af91049f30d7ec272c461d009253faf6800507bc/docs/env.md#legacy-repos-and-migration).

Expand Down Expand Up @@ -743,7 +763,7 @@ next.16 bundle.
| next.16 candidate Doctor suggests `hack daemon start` | That command names the stable CLI. Keep the stable daemon and its launchd service stopped during the shared-home candidate run. Merged [#776](https://github.com/hack-dance/hack/pull/776) treats the daemon as optional for newer candidate Compose trials. |
| `env list --json` reveals plain values | This is next.16 behavior. Review the breaking listing changes in [#767](https://github.com/hack-dance/hack/pull/767) before updating scripts for the next prerelease. |
| next.16 or earlier explicit Doctor migration cleanup can remove `.hack.secret.key` | Keep a private readable key backup before any migration, including `doctor --migrate-env-config`. The merged [#769](https://github.com/hack-dance/hack/pull/769) live-input protection is intended for next.17; an older bundle retains the bug, and upgrading does not recover an already removed key. |
| An old project key has group or other access | next.16 Doctor reports `env-key-weak-permissions` with a `chmod 600` remedy. The broader secrets-hygiene report and fixes are tracked in [#764](https://github.com/hack-dance/hack/pull/764); check the chosen release's availability. |
| An old project key has group or other access | next.16 Doctor reports `env-key-weak-permissions` with a `chmod 600` remedy. Merged [#764](https://github.com/hack-dance/hack/pull/764), intended for next.17, adds `secrets hygiene`; review its findings, then use `doctor --fix` to tighten eligible files and remove only stale Hack artifacts that are not live inputs. |

Sources for startup failures: [next.16 lifecycle and startup handling](https://github.com/hack-dance/hack/blob/b21c3b189841c1a3cbb8af2b76414fa347dc7057/src/commands/project.ts).
The three-project result, credential refresh, daemon-service behavior and HTTPS
Expand Down
Loading