Skip to content

fix(credentials): pin stored GCP credential JSON types, drop deprecated CredentialsFromJSON - #1308

Merged
IAmJSD merged 2 commits into
LeanerCloud:mainfrom
IAmJSD:fix/issue-80
Oct 25, 2026
Merged

IAmJSD merged 2 commits into
LeanerCloud:mainfrom
IAmJSD:fix/issue-80

Conversation

@IAmJSD

@IAmJSD IAmJSD commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Root cause

loadStoredGCPTokenSource (internal/credentials/resolver.go) turned stored GCP credential JSON into a token source with the deprecated google.CredentialsFromJSON. It was //nolint:staticcheck (SA1019) on the grounds that the replacement needed a GCP SDK upgrade and that the bytes are operator-controlled.

Neither reason holds:

  • No upgrade is needed. google.CredentialsFromJSONWithType is already in the pinned golang.org/x/oauth2 v0.36.0.
  • The function accepted any Google credential type in either stored slot. That covers authorized_user, impersonated_service_account, external_account_authorized_user, external_account in the service-account slot, and an OAuth client config ({"installed": …}, which has no type). The client config became an interactive auth-code token source. The API's validateCredentialPayload only lets service_account into gcp_service_account and external_account into gcp_workload_identity_config, but the read path never checked that.

Fix

  • A small map, gcpStoredJSONTypes, ties each stored slot to the one JSON type the API accepts for it:
    • gcp_service_account → google.ServiceAccount
    • gcp_workload_identity_config → google.ExternalAccount (the legacy stored-JSON WIF path needs external_account, so that slot has to keep accepting it)
  • google.CredentialsFromJSONWithType parses with that pinned type, and the //nolint is gone. Any other type fails with parse gcp credentials …: google: expected credential type "…", found "…".
  • A slot with no pinned type fails closed before the store is read. It does not fall back to accepting any type.
  • No go.mod / go.sum change.

Accepted types are only ever narrowed, never widened. Nothing is accepted that wasn't before. The only new rejections are types the API never let into those slots.

Closes #80

Merge-order and overlap notes

When it takes effect

This is a code-only change to the GCP credential resolver used by the scheduler and the purchase manager. It needs no deploy to merge. It takes effect in a deployment once that deployment runs a build that includes it. GCP deploys are paused right now, so GCP-hosted instances keep the old behavior until the pause lifts and they are redeployed. AWS/Azure-hosted deployments that manage GCP accounts pick it up on their next deploy.

Verification

All evidence below is unit tests with mock-store fixtures. No GCP call was made: building a token source is lazy, and no test calls Token().

New tests are in internal/credentials/resolver_gcp_type_test.go:

  • TestResolveGCPTokenSource_ServiceAccountSlot_AcceptsServiceAccountJSON: service-account JSON is accepted (auth mode "" and service_account_key).
  • TestResolveGCPTokenSource_ServiceAccountSlot_RejectsOtherTypes: external_account, authorized_user, impersonated_service_account, external_account_authorized_user and an untyped installed client config are rejected.
  • TestResolveGCPTokenSource_WIFSlot_AcceptsExternalAccountJSON: external_account JSON is accepted on the legacy stored-WIF path.
  • TestResolveGCPTokenSource_WIFSlot_RejectsOtherTypes: service_account plus the four types above are rejected.
  • TestResolveGCPTokenSource_StoredSlots_RejectMalformedJSON: both slots reject non-JSON, truncated JSON, a JSON array, a non-string type and {}. This guards behavior that was already correct.
  • TestLoadStoredGCPTokenSource_RejectsUnknownSlot: a slot with no pinned type fails closed.

Fail-before, on origin/main (a14dc8f2) with only the new test file added:

go test ./internal/credentials/ -count=1 -v -run 'TestResolveGCPTokenSource_(ServiceAccountSlot|WIFSlot|StoredSlots)|TestLoadStoredGCPTokenSource_RejectsUnknownSlot'
--- PASS: TestResolveGCPTokenSource_ServiceAccountSlot_AcceptsServiceAccountJSON
--- FAIL: TestResolveGCPTokenSource_ServiceAccountSlot_RejectsOtherTypes   (5/5 subtests: "An error is expected but got nil")
--- PASS: TestResolveGCPTokenSource_WIFSlot_AcceptsExternalAccountJSON
--- FAIL: TestResolveGCPTokenSource_WIFSlot_RejectsOtherTypes             (5/5 subtests: "An error is expected but got nil")
--- PASS: TestResolveGCPTokenSource_StoredSlots_RejectMalformedJSON        (10/10)
--- FAIL: TestLoadStoredGCPTokenSource_RejectsUnknownSlot

Pass-after, at 63405f9a: the same command passes all 6 tests and all 20 subtests.

Other checks run at 63405f9a, with AWS_CONFIG_FILE=/dev/null AWS_SHARED_CREDENTIALS_FILE=/dev/null AWS_EC2_METADATA_DISABLED=true:

go build ./...                                   # ok
go vet ./...                                     # ok
go vet -tags integration ./...                   # ok
go test ./internal/credentials/ -count=1         # ok
go test ./internal/mocks/ -count=1               # ok
go test ./internal/scheduler/ -count=1           # ok (caller of ResolveGCPTokenSourceWithOpts)
go test ./internal/purchase/ -count=1            # ok (caller of ResolveGCPTokenSourceWithOpts)
golangci-lint v2.10.1 run --allow-parallel-runners ./internal/credentials/   # 0 issues
gocyclo -over 10 -ignore "_test\.go" internal/credentials/resolver.go        # none (loadStoredGCPTokenSource = 6)

Not verified: a real GCP token exchange end to end. GCP calls are off-limits here, and GCP deploys are paused.

Combined-tree verification. This branch was tested together with the other queued branches on a local, never-pushed tree: queue/platform @ b56afb2b = origin/main a14dc8f2 plus these heads, merged in this order:

Branch Tested head
fix/issue-204 76f85a4
fix/issue-438 78fe011
fix/issue-838 120b07f
fix/issue-143 aef1256
fix/issue-349 cfbb551
fix/issue-80 63405f9
fix/issue-199 36b9f8f
fix/issue-41 96fc237
fix/issue-299 6af14ed
fix/issue-324 3ab6729

fix/issue-324 and fix/issue-204 both edit .github/workflows/ci.yml next to the azure-kv-access-policy job. The tree kept #324's new "Run Function URL reader self-tests" step and #204's updated job comment; whichever PR merges second takes that same resolution.

Linux, Go 1.26.9, AWS_CONFIG_FILE=/dev/null AWS_SHARED_CREDENTIALS_FILE=/dev/null AWS_EC2_METADATA_DISABLED=true; Docker Postgres via testcontainers. The full suite ran on ed0c7174, which differs from b56afb2b only by one .gitallowed line on fix/issue-324:

  • Passed: go build ./..., go vet ./..., go vet -tags integration ./... and the full go test -race -short -count=1 ./....
  • Static checks: golangci-lint v2.10.1 found 0 issues on every changed package, with and without --build-tags integration. gocyclo -over 10 is silent on the changed non-test files.
  • Integration tests, every changed package that has them, one package at a time (-tags integration -count=1, 60-minute local timeout): internal/api, internal/purchase, internal/scheduler and internal/server all pass.
  • Secrets: the first run caught a bare placeholder account ID in fix/issue-324's new tftest, now allowlisted with an anchored .gitallowed line. Re-run at b56afb2b: git-secrets, using the repo's scripts/setup-git-secrets.sh patterns in a fresh repo with an isolated HOME, found nothing in the changed files, and a planted fake key was flagged. scripts/test-git-secrets-allowlist.sh: 137 PASS, 0 FAIL.
  • Nothing was applied to any cloud.

Out of scope

  • Validation on the other credential write paths, and of workload-identity config contents, is handled in a separate change.

Maintainer: please mirror the issue's triage labels onto this PR: impact/internal, type/chore, triaged, priority/p3, severity/low, urgency/eventually, effort/m.

🤖 Generated with Claude Code

…ed CredentialsFromJSON

loadStoredGCPTokenSource parsed stored GCP JSON with the deprecated
google.CredentialsFromJSON (SA1019, nolint'd), which accepts any Google
credential type: authorized_user, impersonated_service_account,
external_account_authorized_user, an OAuth client config with no type,
and external_account in the service-account slot.

Use google.CredentialsFromJSONWithType, already in golang.org/x/oauth2
v0.36.0 (no go.mod change), and pin each slot to the one type the API
accepts into it: gcp_service_account -> service_account,
gcp_workload_identity_config -> external_account. A slot with no pinned
type fails closed before the store is read.

Closes LeanerCloud#80

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@IAmJSD IAmJSD added impact/internal Team-internal only type/chore Maintenance / non-user-visible triaged Item has been triaged priority/p3 Polish / idea / may never ship severity/low Minor harm urgency/eventually No deadline effort/m Days labels Oct 23, 2026
@coderabbitai

coderabbitai Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

  • Run on-demand review

This review includes 3 billable files and costs up to $0.75.

Or wait 58 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
Commits

Reviewing files that changed from the base of the PR and between 1c5787a and 4bed9c0.

Files selected for processing: 3
  • internal/credentials/resolver.go
  • internal/credentials/resolver_extra_test.go
  • internal/credentials/resolver_gcp_type_test.go
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

Brings in main's CI fixes (LeanerCloud#1236, LeanerCloud#1252) and CI changes since, so this PR gets a fresh CI run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@IAmJSD

IAmJSD commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

Independent review (IAmJSD's agent, fresh context, given only the diff, the issue and the PR text): VERDICT: APPROVE 4bed9c056b81a45eee82a48e2c0ed9fbef6b8483

Blocking:

  • none

Non-blocking:

  • internal/api/handler_registrations.go:237-249 (not touched by this PR): storeRegistrationCredentials saves the registrant-supplied payload into whatever credential_type the registrant named, with no validateCredentialPayload call (submit-time validateRegistrationRequest doesn't check it either). So, unlike the PR body says, validateCredentialPayload is not the only way into the GCP slots. An approved public registration can put any JSON into gcp_service_account / gcp_workload_identity_config. This PR is still a strict improvement, because the read path now rejects anything but the pinned type no matter how it got stored. But it has two consequences. (a) A GCP account onboarded by registration with a non-service_account JSON (e.g. authorized_user) that resolved before will now fail to resolve. That's the intended behavior, but it isn't "types the API never let in". (b) The WIF slot still accepts any external_account config, including credential_source.file / credential_source.url with an arbitrary token_url, from registrants as well as admins. That is the actual risk behind SA1019, and pinning the type doesn't close it. The PR scopes this out ("validation on the other credential write paths, and of workload-identity config contents, is handled in a separate change"), so please make sure that change exists and covers the registration path.
  • The issue's last work item ("verify GCP credential resolution still works end-to-end") remains unverified, as the body states. Closes #80 is still fair, because the deprecated call and the //nolint are gone and the replacement is the non-deprecated API in the pinned oauth2 v0.36.0 (checked google/default.go: CredentialsFromJSONWithType has no Deprecated marker, while CredentialsFromJSON/WithParams do).

Attacked:

  • Checked every reader of the stored GCP slots: loadStoredGCPTokenSource is the only JSON-to-token-source path. ResolveGCPCredentials has no non-test callers, and there's no WithCredentialsJSON anywhere. The federated (secret-free) WIF path is untouched and still taken before the stored-JSON fallback. The unknown-slot guard fails closed before the store read.
  • Fail-before: git checkout origin/main -- internal/credentials/resolver.go, then ran the new tests. ServiceAccountSlot_RejectsOtherTypes, WIFSlot_RejectsOtherTypes and LoadStoredGCPTokenSource_RejectsUnknownSlot FAIL. Restored, and they pass.
  • go build ./...; go vet on ./internal/credentials/ with and without -tags integration; go test -race -short -count=1 ./internal/credentials/ ok; golangci-lint v2.10.1 on the package with and without --build-tags integration: 0 issues. The package has no integration tests.

Body-stale:

  • "The only new rejections are types the API never let into those slots" and "the read path never checked that [the API validates]": the account-registration approval path writes unvalidated payloads into those slots (see above), so other types could be stored, and this PR now rejects them on read.

Reviewer: rB13. Bar while the maintainers are away (set by IAmJSD as acting owner): one approving independent review on the head, two for money, auth and security paths.

@IAmJSD

IAmJSD commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

Independent review (IAmJSD's agent, fresh context, given only the diff, the issue and the PR text): VERDICT: APPROVE 4bed9c056b81a45eee82a48e2c0ed9fbef6b8483

Blocking:

  • none

Non-blocking:

  • Rollout note, not a defect: validateCredentialPayload is not the only way into these slots. submitRegistration (internal/api/handler_registrations.go:56-103, the public POST /api/register) encrypts credential_payload with whatever credential_type the submitter sends and never calls validateCredentialPayload; storeRegistrationCredentials (:238-250) saves it on admin approval. So a GCP account onboarded through self-registration can hold e.g. authorized_user JSON in gcp_service_account today, which resolved before this PR and will fail with expected credential type "service_account" after it. That is the correct outcome (and this PR is what now protects the read side of that unvalidated write path), but operators should know such accounts will start failing on the next scheduler/purchase run. Consider a one-off query or log line to find them before deploy. The write-path gap itself belongs to the separate change the body mentions; worth checking that it covers the registration path.
  • The WIF slot still accepts any external_account config, including credential_source.file/url/executable and an arbitrary token_url. That is the SA1019 risk the deprecation is actually about, and pinning the type does not reduce it for this slot. The body puts content validation out of scope, which is fine, but the new gcpStoredJSONTypes comment could say that the type pin is not content validation.

Attacked:

  • Checked every reader and writer of gcp_service_account / gcp_workload_identity_config (grep for the constants, SaveCredential, CredentialsFromJSON*, WithCredentialsJSON). Only loadStoredGCPTokenSource builds credentials from them, and ResolveGCPCredentials has no non-test callers. The federated (no stored JSON) path is unchanged, and an unknown slot fails closed before the store is read. Confirmed in oauth2 v0.36.0 that CredentialsFromJSONWithType is not deprecated, and that ServiceAccount/ExternalAccount are "service_account"/"external_account", which match the API's payloadTypeMatches strings.
  • Fail-before: with resolver.go reverted to origin/main and the new tests kept, ServiceAccountSlot_RejectsOtherTypes, WIFSlot_RejectsOtherTypes and RejectsUnknownSlot fail. They pass at head.
  • Ran go build ./..., go vet (with and without -tags integration) on internal/credentials, go test -race -short -count=1 on internal/credentials, internal/scheduler and internal/purchase (all ok), and golangci-lint run ./internal/credentials/ (0 issues). The package has no integration tests.

Body-stale:

  • "Nothing is accepted that wasn't before. The only new rejections are types the API never let into those slots": the first sentence holds. The second is false because of the registration path above. Non-service_account / non-external_account JSON can be in those slots through POST /api/register + approval.

Reviewer: rB16. Bar while the maintainers are away (set by IAmJSD as acting owner): one approving independent review on the head, two for money, auth and security paths.

@IAmJSD

IAmJSD commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

Local verification at 4bed9c056b81a45eee82a48e2c0ed9fbef6b8483 merged onto main (5f717aa4), together with #1260, #1261, #1263, #1264, #1269, #1271, #1273, #1308, #1416 in that order (combined tree 62127c84, Linux, Go 1.26.10): go build ./..., go vet with and without -tags integration, the full go test -race -short -count=1 ./..., and go test -race -tags integration -count=1 on every changed package, against Docker Postgres. All pass. CI Success and pre-commit are green on this head, with 2 approving independent review(s) on it. Nothing was run against any cloud.

@IAmJSD
IAmJSD marked this pull request as ready for review October 25, 2026 21:56
@IAmJSD
IAmJSD merged commit f964316 into LeanerCloud:main Oct 25, 2026
36 checks passed
@IAmJSD

IAmJSD commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

Merged by IAmJSD's agent at IAmJSD's request (IAmJSD is acting owner while the maintainers are away), under the bar IAmJSD set: 2 approving independent review(s) on 4bed9c05 (two required: money, auth or security path), green CI, and the combined-tree verification above. Squash-merged at exactly 4bed9c056b81a45eee82a48e2c0ed9fbef6b8483.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/m Days impact/internal Team-internal only priority/p3 Polish / idea / may never ship severity/low Minor harm triaged Item has been triaged type/chore Maintenance / non-user-visible urgency/eventually No deadline

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tech-debt: migrate off deprecated google.CredentialsFromJSON (SA1019) via GCP SDK upgrade

1 participant