Skip to content

fix(auth): report an expired MFA enrollment as expired and pin the MFA write guards - #1284

Merged
IAmJSD merged 2 commits into
LeanerCloud:mainfrom
IAmJSD:fix/issue-568
Oct 26, 2026
Merged

IAmJSD merged 2 commits into
LeanerCloud:mainfrom
IAmJSD:fix/issue-568

Conversation

@IAmJSD

@IAmJSD IAmJSD commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

This works through the follow-ups listed in the #567 auth review. Each item was checked again against current main (a14dc8f). One item, (d), needed a code fix. The others needed a comment or a test that pins existing behavior, and (f) needed a test fix.

Item Status on main What this PR does
(a) mfa_enabled / mfa_secret guards in DisableMFA and ReplaceMFARecoveryCodes each survive being dropped alone Still open. No test isolated a single conjunct. Fixed (tests). TestIntegration_MFAWriteGuardsArePinnedIndividually puts the row in a state where exactly one of mfa_enabled, mfa_secret or mfa_recovery_codes differs from the caller's read, for both writes. The write must be refused and the row left untouched. The issue's suggested subtest stale-regenerate-after-disable-with-zero-recovery-codes is added too. Both guards are kept rather than dropping one, because they are cheap and each pins a separate invariant.
(b) Nullable secret comparison Closed in the issue as "checked and dropped". Not part of the acceptance criteria. No change.
(c) slices.Clone in MFADisable needs a comment Still open. There was no comment, and the doc comment on consumeRecoveryCode still said "persist via UpdateUser". Fixed. Comments added on the clone and on consumeRecoveryCode. A new unit test, TestMFADisable_WithRecoveryCodePassesCodesAsRead, spends a code that is not the last one. The existing unit test used a single code, and with one code the in-place shift is invisible.
(d) An EnableMFA refused by the SQL expiry check returns ErrUserChanged (409) instead of "enrollment expired" (400) Still reproduces. TestIntegration_MFAEnableExpiryEnforcedInWrite asserted ErrUserChanged. Fixed. See below.
(e) No test pins that DisableMFA clears mfa_pending_secret and mfa_pending_secret_expires_at Still open Fixed (test). Subtest disable-clears-every-mfa-column.
(f) TestIntegration_LoginLockoutExpiry compares DB timestamps against Go-clock bounds Still open Fixed. The bounds are now read from the database's own clock. See below.

No item was deferred. I checked every in-flight auth branch for overlap with the lines this PR edits (internal/auth/store_postgres_credentials.go EnableMFA, service_mfa.go around consumeRecoveryCode and MFADisable, service_mfa_cas_db_test.go, service_login_db_test.go TestIntegration_LoginLockoutExpiry, service_mfa_test.go). For the PRs I used gh pr diff; for the branches, git diff origin/main...fork/<branch>.

Root cause and fix

(d) The EnableMFA write rejects for three different reasons: MFA already on, the pending secret was replaced or cleared, or the pending secret has expired. All three collapsed into ErrUserChanged. The service checks expiry with the Go clock before the write, so the SQL expiry check only fires at the boundary. When it does, the user got 409 "account changed concurrently" when the real problem was an expired enrollment that needs a fresh setup.

When the write affects no rows, EnableMFA now runs a read-only follow-up on the database clock (enableMFARefusal):

SELECT (mfa_pending_secret_expires_at <= NOW()) IS TRUE FROM users
WHERE id = $1 AND mfa_enabled = false AND mfa_pending_secret = $2

true maps to ErrMFAEnrollmentExpired, which the API already maps to 400. No row, or not expired, maps to ErrUserChanged. If the check itself errors, the result is still ErrUserChanged, with the cause wrapped. The write itself is unchanged, so this cannot accept anything that was refused before. It only changes which error a refused call reports.

Why the separate read is race-safe. The read can only report "expired" if, after the refused write, the row still has MFA off, the same pending secret, and a passed expiry. Pending secrets are random. They leave the row only by being cleared (enable, disable, cancel) or replaced (a new setup). A row that still holds this secret with MFA off therefore held it at write time too. The other two refusal reasons are ruled out, which leaves the expiry. NOW() does not go backwards between the two statements, so an expiry that refused the write is still past at the read. Concurrent enables of one live enrollment: the winner flips mfa_enabled, so every loser's read finds no row and gets ErrUserChanged. A real-Postgres race test covers this: 8 racers × 5 rounds, exactly one winner, every loser gets ErrUserChanged and none gets "expired".

(f) last_login_at and locked_until are stamped with the server's NOW(), but the test bounded them with time.Now() from the test process. Two clocks within a few ms of each other can disagree. The test now reads SELECT clock_timestamp() before and after Login. The stamped NOW() comes from a transaction that starts between those two statements, on the same clock and at the same microsecond precision, so the inclusive bounds always hold. There is no tolerance window left to tune.

Verification

All runs used export PATH=$HOME/.local/go/bin:$HOME/go/bin:$HOME/.local/bin:$PATH AWS_CONFIG_FILE=/dev/null AWS_SHARED_CREDENTIALS_FILE=/dev/null AWS_EC2_METADATA_DISABLED=true on Linux, with Postgres 16 testcontainers.

Static checks and unit tests:

  • go build ./..., go vet ./..., go vet -tags integration ./...: clean
  • go test ./internal/auth/ -count=1: ok
  • go test ./internal/mocks/ -count=1: ok
  • go test ./internal/api/ -count=1 -run 'MFA|Mfa': ok
  • golangci-lint run --allow-parallel-runners ./internal/auth/ and the same with --build-tags integration: 0 issues
  • gocyclo -over 10 -ignore "_test\.go" internal/auth/store_postgres_credentials.go internal/auth/service_mfa.go: nothing over 10

Fail-before/pass-after. "Before" means the origin/main store_postgres_credentials.go swapped into this tree; "mutation" means one SQL clause removed by hand.

  • (d), unit (pgxmock): go test ./internal/auth/ -count=1 -run TestPGXMock_EnableMFA -v
    • Before: TestPGXMock_EnableMFA_RefusalReason fails in all 4 subtests.
    • After: pass.
  • (d), integration on real Postgres: go test -tags integration -count=1 -timeout 20m -run 'TestIntegration_MFAEnableExpiryEnforcedInWrite|TestIntegration_EnableMFARefusalReason' ./internal/auth/
    • Before: TestIntegration_MFAEnableExpiryEnforcedInWrite fails (expected: "MFA enrollment expired"), and so does TestIntegration_EnableMFARefusalReason/expired. The other refusal cases and the concurrent-enable race already passed before, so the fix does not change them.
    • After: pass.
  • (d), mutations of the follow-up read:
    • Dropping mfa_enabled = false fails TestIntegration_EnableMFARefusalReason/mfa-on-with-expired-pending.
    • Dropping mfa_pending_secret = $2 fails .../replaced-by-expired-setup.
  • (a) and (e) pin behavior that is already correct on main, so the fail-before evidence is one mutation run per clause against TestIntegration_MFAWriteGuardsArePinnedIndividually. Each mutation fails exactly the matching subtest:
    • DisableMFA drop mfa_enabled → disable-refused-on-mfa-enabled-false
    • DisableMFA drop mfa_secret → disable-refused-on-mfa-secret-replaced
    • DisableMFA drop mfa_recovery_codes → disable-refused-on-mfa-recovery-codes-changed
    • Same three for ReplaceMFARecoveryCodes → the matching regenerate-refused-on-* subtests
    • DisableMFA SET without mfa_pending_secret = '', or without mfa_pending_secret_expires_at = NULL → disable-clears-every-mfa-column
    • ReplaceMFARecoveryCodes guarded by mfa_recovery_codes alone → TestIntegration_MFADisableAndRegenerateAreColumnScoped/stale-regenerate-after-disable-with-zero-recovery-codes
  • (c), mutation: removing slices.Clone in MFADisable makes TestMFADisable_WithRecoveryCodePassesCodesAsRead fail (unit). The existing TestMFADisable_WithRecoveryCode still passes under that mutation, because it uses a single code.
  • (f): no deterministic fail-before. The flake comes from skew between two clocks, and I could not reproduce that skew on this host, where Docker shares the kernel clock. The fix removes the second clock by construction. After the change, TestIntegration_LoginLockoutExpiry passes.
  • Final tree: go test -tags integration -count=1 -timeout 20m -run 'TestIntegration_MFAEnableExpiryEnforcedInWrite|TestIntegration_EnableMFARefusalReason|TestIntegration_MFAWriteGuardsArePinnedIndividually|TestIntegration_MFADisableAndRegenerateAreColumnScoped|TestIntegration_LoginLockoutExpiry' ./internal/auth/: ok. TestIntegration_EnrollmentCodeIsSpent, which also exercises EnableMFA, passes as well. One run of it failed before any test code ran, because the container did not start ("wait until ready ... context deadline exceeded") while the machine was loaded. It passed on rerun.

Whole integration packages, -race and git-secrets were not run here. They run on the combined tree below.

Combined-tree verification. This branch was tested together with the other queued branches on a local, never-pushed tree: queue/platform @ 936996ad = origin/main a14dc8f2 plus these heads, merged in this order:

Branch Tested head
fix/issue-337 ff3420c
fix/issue-169 7d75eb5
fix/issue-342 66b221f
fix/issue-310 5e11c21
fix/issue-326 5f0ea59
fix/issue-243 365e959
fix/issue-343 6c94036
fix/issue-312 d2848a8
fix/issue-316 acd60dd
fix/issue-339 75ea9b9
fix/issue-339-commitment-type 73bdb19
fix/issue-353 5d25fcd
fix/issue-313 3156bc4
fix/issue-357 53d9e58
fix/issue-368 0ba727d
fix/issue-419 61648c9
fix/issue-111 b04c372
fix/issue-686 0ad90b7
fix/issue-674 793095d
fix/issue-1055 4ec96f1
fix/issue-559 dfa9525
fix/issue-558 034eed4
fix/issue-874 1720539
fix/issue-666 1e120da
fix/issue-668 fb26345
fix/issue-568 ce04173
fix/issue-311 e54f250
fix/issue-177 0cbb88a
fix/issue-151 c21377c
fix/issue-166 41fb77b
fix/issue-27 2d67622
fix/issue-323 69288ae
fix/issue-59 36463e8
fix/issue-43 35e4069
fix/issue-158 6f76f8a
fix/issue-104 bb00491
fix/issue-55 971ef1b
fix/issue-361 89ac220
fix/issue-120 b458719
fix/issue-36 42bac6e
fix/issue-525 57c97dd
fix/issue-712 e5bcda0
fix/issue-72 6cc63ed

fix/issue-72 and fix/issue-312 conflict on one line in internal/api/handler_dashboard.go (both change the calculateCommitmentMetrics call). The tree used #72's side with #312's validated provider variable; whichever PR merges second takes that same resolution.

Linux, Go 1.26.9, AWS_CONFIG_FILE=/dev/null AWS_SHARED_CREDENTIALS_FILE=/dev/null AWS_EC2_METADATA_DISABLED=true; Docker Postgres via testcontainers:

  • Passed at 936996ad: go build ./..., go vet ./..., go vet -tags integration ./... and the full go test -race -short -count=1 ./....
  • Static checks at 936996ad: golangci-lint v2.10.1 found 0 issues on every changed package, with and without --build-tags integration. tests/e2e is a separate module, so it was vetted under its own e2e tag. gocyclo -over 10 is silent on the changed non-test files.
  • Integration tests, every changed package, one package at a time (-tags integration -count=1, 60-minute local timeout because the VM was shared):
    • At 936996ad, these pass: internal/server, internal/credentials, internal/scheduler, internal/database/postgres/migrations, internal/purchase and internal/api.
    • internal/config passes except TestPostgresStoreDB_ResolveStaleRIExchange_ValidatesCloudCheck, which fails the same way on unmodified main (fixed by fix(config): compare resolve checked_at at the database's microsecond precision #1252).
    • These were run on the first build of this tree (26795376, same branches at earlier heads): internal/analytics, internal/auth, internal/commitmentopts, internal/database and cmd/lambda/clear-rate-limit all pass. The later fixes don't touch those packages.
  • What the combined run caught, each fixed on its own branch before this PR opened:
    • a compile break between fix/issue-104's test and fix/issue-326's httpToLambdaRequest signature change;
    • fix/issue-316 rejecting legacy-keyed plans;
    • fix/issue-120 versus main's global-filter test.
  • Secrets: git-secrets, using the repo's scripts/setup-git-secrets.sh patterns in a fresh repo with an isolated HOME, found nothing in the changed files, and a planted fake key was flagged. scripts/test-git-secrets-allowlist.sh: 137 PASS, 0 FAIL.
  • Nothing was applied to any cloud.

Out of scope / notes

  • The subtests that isolate one guard write synthetic row states directly with SQL, for example mfa_enabled = false with the secret and codes still set. Normal flows don't produce those states, because disable clears all three columns together. That is exactly why a single guard could not be isolated through the service API. The realistic zero-codes scenario is covered separately at the service level.
  • MFAEnable losers of a concurrent enable could arguably get ErrMFAAlreadyEnabled instead of ErrUserChanged. Both are 409, and this PR leaves that as it is.
  • No frontend change is needed. The UI already receives 400 "MFA enrollment expired" from the pre-write check in this case. It now gets the same response at the boundary instead of 409.

Closes #568

Maintainers: please mirror the issue's triage labels onto this PR: impact/internal, type/chore, triaged, priority/p3, severity/low, urgency/eventually, effort/s.

🤖 Generated with Claude Code

…A write guards

Follow-ups from the LeanerCloud#567 review:

- (d) When EnableMFA's guarded write is refused, re-check the row on the
  database clock: if MFA is still off under the same pending secret and
  its expiry has passed, return ErrMFAEnrollmentExpired (400) instead of
  ErrUserChanged (409). Every other refusal, including the losers of
  concurrent enables, stays ErrUserChanged.
- (a) Pin each conjunct of the DisableMFA and ReplaceMFARecoveryCodes
  guards on its own, and add the zero-recovery-codes stale
  regenerate-after-disable subtest.
- (c) Say why MFADisable clones the recovery codes before spending one,
  and pin it with a unit test that spends a non-last code.
- (e) Pin that DisableMFA clears the pending secret and its expiry.
- (f) Bound LoginLockoutExpiry's timestamps on the database clock
  (clock_timestamp()) instead of the test process clock.

Closes LeanerCloud#568

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@IAmJSD IAmJSD added impact/internal Team-internal only type/chore Maintenance / non-user-visible triaged Item has been triaged priority/p3 Polish / idea / may never ship severity/low Minor harm urgency/eventually No deadline effort/s Hours labels Oct 23, 2026
@coderabbitai

coderabbitai Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

  • Run on-demand review

This review includes 6 billable files and costs up to $1.50.

Or wait 59 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
Commits

Reviewing files that changed from the base of the PR and between 503eb2e and bf63bd5.

Files selected for processing: 6
  • internal/auth/service_login_db_test.go
  • internal/auth/service_mfa.go
  • internal/auth/service_mfa_cas_db_test.go
  • internal/auth/service_mfa_test.go
  • internal/auth/store_postgres_credentials.go
  • internal/auth/store_postgres_credentials_pgxmock_test.go
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

Brings in main's CI fixes (LeanerCloud#1236, LeanerCloud#1252) and CI changes since, so this PR gets a fresh CI run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@IAmJSD

IAmJSD commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

Independent review (IAmJSD's agent, fresh context, given only the diff, the issue and the PR text): VERDICT: APPROVE bf63bd58450124b6e2277e00a7525dbe2646a500

Blocking:

  • none

Non-blocking:

  • internal/auth/store_postgres_credentials.go enableMFARefusal: the follow-up read is a second statement outside the write's transaction. I worked through the interleavings and the error is only reported the safe way round. "Expired" needs the same random pending secret, MFA still off and a past expiry on the DB clock, which is exactly the write's remaining refusal reason. Every other interleaving falls back to ErrUserChanged (409), as before. The write itself is unchanged, so nothing that was refused before is accepted now. A short note in the doc comment that a re-setup reusing the same secret could only ever turn "expired" into 409, never the reverse, would help the next reader.
  • internal/auth/service_login_db_test.go (f): no deterministic fail-before, as the body admits. I checked that last_login_at and locked_until are both stamped with NOW() in store_postgres_login.go, so bounds from clock_timestamp() on the same server are sound.
  • The (a) subtests write synthetic row states directly. That is the only way to isolate one guard, and the body explains it, but they pin SQL shape rather than reachable states. The realistic zero-codes subtest covers the reachable case.

Attacked:

  • Head checked: bf63bd58. I read issue chore(auth): follow-ups from the #567 review #568, whose acceptance criteria are (a), (c), (d), (e) and (f). (b) is closed in the issue. All five are addressed, so Closes #568 is right.
  • Error mapping: ErrMFAEnrollmentExpired maps to 400 in mapMFAServiceError. If the diagnosis query fails, the error wraps ErrUserChanged and still maps to 409 with the fixed message, so no DB error text leaks.
  • Fail-before, with the merge-base store_postgres_credentials.go restored: all 4 subtests of TestPGXMock_EnableMFA_RefusalReason fail. On real Postgres, TestIntegration_MFAEnableExpiryEnforcedInWrite and TestIntegration_EnableMFARefusalReason/expired fail.
  • (c) mutation: replacing slices.Clone with the raw slice makes TestMFADisable_WithRecoveryCodePassesCodesAsRead fail. Everything was restored afterwards.
  • Commands run: go build ./...; go vet on ./internal/auth/, with and without -tags integration; go test -race -short -count=1 ./internal/auth/ ./internal/api/ (ok); go test -race -tags integration -count=1 ./internal/auth/ on the whole package (ok, 89s, including the 8x5 concurrent-enable race test); golangci-lint v2.10.1 on internal/auth, with and without the integration tag (0 issues).

Body-stale:

  • none found. "main (a14dc8f)" is just older than main now. The combined-tree table is the author's own record, and I did not re-run it.

Reviewer: rB19. Bar while the maintainers are away (set by IAmJSD as acting owner): one approving independent review on the head, two for money, auth and security paths.

@IAmJSD

IAmJSD commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

Independent review (IAmJSD's agent, fresh context, given only the diff, the issue and the PR text): VERDICT: APPROVE bf63bd58450124b6e2277e00a7525dbe2646a500

Blocking:

  • none

Non-blocking:

  • internal/auth/store_postgres_credentials.go:159: when the follow-up read fails, the returned error wraps the raw DB error (%w (checking enrollment expiry: %w)). That's fine today because handler_auth.go builds the 409 from auth.ErrUserChanged.Error() and not from err.Error(). If any caller ever logs or returns err.Error() to the client, the pgx text goes with it. Worth keeping in mind; no change needed.
  • The race-safety argument holds only if a pending secret can never be the empty string while an expiry is set. That's true today: SetPendingMFASecret always writes a random secret, and the clear paths set the expiry to NULL. Nothing enforces it at the schema level, though. A CHECK (mfa_pending_secret <> '' OR mfa_pending_secret_expires_at IS NULL) would make it structural. This is optional, for a later PR.

Attacked:

  • (d) race reasoning: I read the guarded UPDATE and the follow-up SELECT side by side. "Expired" needs MFA off, the same random pending secret, and expires_at <= NOW() on a later statement's clock. A concurrent enable/disable/setup either clears or replaces the secret, so those all land in ErrUserChanged. Under READ COMMITTED, enable losers re-evaluate after the winner commits and then read mfa_enabled = true, so they get ErrUserChanged. The write itself is unchanged, so nothing that was refused before can now be accepted. The API maps both errors with errors.Is, so the double-%w wrap still gives 409.
  • (f) all three stamps (last_login_at, locked_until) are server-side NOW() (store_postgres_login.go:12-29), so the clock_timestamp() bounds use the same clock.
  • Fail-before: I swapped the merge-base store_postgres_credentials.go into the head tree. All 4 TestPGXMock_EnableMFA_RefusalReason subtests fail. TestIntegration_MFAEnableExpiryEnforcedInWrite and TestIntegration_EnableMFARefusalReason/expired fail on real Postgres. I also replaced slices.Clone with slices.Clip in MFADisable, and TestMFADisable_WithRecoveryCodePassesCodesAsRead fails. I restored both files afterwards.
  • Checks run: go build ./..., go vet (with and without -tags integration) on internal/auth, go test -race -short -count=1 ./internal/auth/ ./internal/api/, and go test -race -tags integration -count=1 ./internal/auth/ (whole package, testcontainers). I repeated the auth unit and integration tests on a local merge of the head with current main (5a2e31e, which touched the same file), and that merge is clean. golangci-lint v2.10.1 on internal/auth, with and without --build-tags integration: 0 issues.

Body-stale:

  • none. "current main (a14dc8f)" is out of date now, but the head merges cleanly with main 5a2e31e and still passes there.

Reviewer: rB20. Bar while the maintainers are away (set by IAmJSD as acting owner): one approving independent review on the head, two for money, auth and security paths.

@IAmJSD

IAmJSD commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

Local verification at bf63bd58450124b6e2277e00a7525dbe2646a500 merged onto main (eb9e4802), together with #1277, #1284, #1288, #1290, #1294, #1306 in that order (combined tree 6bb85247, Linux, Go 1.26.10): go build ./..., go vet with and without -tags integration, the full go test -race -short -count=1 ./..., and go test -race -tags integration -count=1 on every changed package, against Docker Postgres. All pass. CI Success and pre-commit are green on this head, with 2 approving independent review(s) on it. Nothing was run against any cloud.

@IAmJSD
IAmJSD marked this pull request as ready for review October 26, 2026 00:00
@IAmJSD
IAmJSD merged commit fb2f3a4 into LeanerCloud:main Oct 26, 2026
34 checks passed
@IAmJSD

IAmJSD commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

Merged by IAmJSD's agent at IAmJSD's request (IAmJSD is acting owner while the maintainers are away), under the bar IAmJSD set: 2 approving independent review(s) on bf63bd58 (two required: money, auth or security path), green CI, and the combined-tree verification above. Squash-merged at exactly bf63bd58450124b6e2277e00a7525dbe2646a500.

IAmJSD added a commit to IAmJSD/cloud-commitments-platform that referenced this pull request Oct 26, 2026
Brings the branch up to date with main so the PR is mergeable again.
The branch sat on the pre-squash LeanerCloud#1202 commit cad1a4b, so a plain merge
conflicted in store_postgres.go and the two admin-lock test files. Those
three files take the result of replaying only c1d39b0 onto main
(git merge-tree --merge-base cad1a4b), which is conflict-free.
service_mfa.go takes main's version: LeanerCloud#1284 already reworded the
consumeRecoveryCode comment so it no longer names the store UpdateUser,
which is all this PR's change to that file did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/s Hours impact/internal Team-internal only priority/p3 Polish / idea / may never ship severity/low Minor harm triaged Item has been triaged type/chore Maintenance / non-user-visible urgency/eventually No deadline

Projects

None yet

Development

Successfully merging this pull request may close these issues.

chore(auth): follow-ups from the #567 review

1 participant