Skip to content

PER-14389: Migrate from Permit Cloud — self-service export + on-prem import guide#647

Open
EliMoshkovich wants to merge 8 commits into
masterfrom
PER-14389-migrate-from-cloud-import-guide
Open

PER-14389: Migrate from Permit Cloud — self-service export + on-prem import guide#647
EliMoshkovich wants to merge 8 commits into
masterfrom
PER-14389-migrate-from-cloud-import-guide

Conversation

@EliMoshkovich

@EliMoshkovich EliMoshkovich commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a customer-facing guide to the On-Premises Deployment section for migrating from Permit Cloud (SaaS) to a self-hosted deployment. The customer runs the whole migration themselves, end to end:

  • Export their organization's data from Permit Cloud with their own Organization API keyPOST /v2/data-export starts a background job, they poll GET /v2/data-export/{task_id} until it's ready, then download the tar.gz package from a short-lived link.
  • Import that package into their own on-prem Permit Platform (single-transaction, FK-deferred, from the package manifest).

No Permit-side tooling or internal process is exposed — the page is scoped to the customer's self-service flow and their own cluster.

Linear Issue

PER-14389

Changes

  • docs/how-to/deploy/on-prem/migrate-from-cloud.mdx (new page, picked up by the autogenerated on-prem sidebar):
    • Step 0 — Export Your Data from Permit Cloud: generate an Organization API key with management access (Settings → API Keys; read-only keys are rejected), then start → poll → download: POST /v2/data-export returns a task id, GET /v2/data-export/{task_id} polls until success and returns a short-lived presigned download_url. Notes that the link is short-lived + secret, plus write-freeze, version alignment (manifest.saas_alembic_version), and the too-large case (surfaces as a poll failure).
    • Package layout: data/ (one CSV per table) + manifest.json; v2_identity.csv is reference-only and never imported; policy-as-code lives in Git, not the package.
    • Import (steps 1–9): checksum verification against the manifest, pre-import DB backup, write-freeze (verified chart deployment names), single-transaction FK-deferred import driven by manifest.import.tables_in_order, dangling-member cleanup, cloud-specific config review (SSO/webhooks/PDP), admin superuser grant.
    • Verify (org-filtered row counts vs manifest.json + UI checks), Team member access (Keycloak email-linking, verified-email requirement), Point applications at on-prem (SDK api_url cutover), Troubleshooting.
  • One-line companion fix in management.mdx: backend health check /api/v2/health/v2/healthy (the backend route is /healthy under the /v2 prefix).

Scope / boundary

  • Self-service export — the customer exports their own org with their own API key; the org is derived from the key, so a key can only ever export its own org.
  • No internal content — the page contains none of Permit's internal migration tooling (the sysadmin export endpoint, Bytebase, prod-US/Aurora, RDS reader endpoints, pgbouncer). Only the public api.permit.io/v2/data-export endpoint and the customer's own permit-platform-namespace kubectl commands appear.
  • RDS data only — no audit-log / OpenSearch history content (the page does not reference exporting or requesting historical logs).

Review

  • Multi-agent deep review with a customer-facing lens (2026-07-23): verdict READY, no blocking findings.
    • Zero internal leaks (grepped Bytebase / prod-US/EU / Aurora / RDS / sysadmin / pgbouncer / staging — none present).
    • Zero audit-log / OpenSearch references remain.
    • MDX valid (frontmatter, balanced admonitions, no raw <…> outside code fences), all internal anchors and sibling-file links resolve.
    • Export instructions verified against the backend code (data_export.py): endpoint path, Bearer API-key auth, org-from-key scoping, management-access gate (read-only → 403), 413 path, and every manifest field the import steps consume.
  • Earlier rounds: api_url /api-prefix fix (Zeev), kubectl scale portability (Copilot), and verification of chart deployment names / postgres pod label / DB name+user / session_replication_role bootstrap against the platform Helm chart and backend source.

Test plan

  • Multi-agent deep review, customer-facing lens — READY (no internal leaks, MDX valid, anchors/links resolve)
  • Export instructions cross-checked against the live /v2/data-export endpoint + manifest schema
  • Import commands verified against the on-prem Helm chart (deployment names, postgres pod label, DB name/user, namespace) and backend source
  • No references to internal tooling (sysadmin endpoint, Bytebase, Permit infra) and no audit-log/OpenSearch content

🤖 Generated with Claude Code

Customer-facing guide for importing a Permit-delivered migration data
package into an on-prem deployment: package request/verification, DB
backup, write-freeze, ordered table import (with the same-session
FK-disable form for api_key/pdp_config), post-import cleanup and
superuser grant, org-filtered verification against the manifest,
Keycloak email-linking for member access, and troubleshooting.

Import side only — the export is performed by Permit and delivered as a
secure download link; no internal tooling is referenced.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 8, 2026 19:44
@linear-code

linear-code Bot commented Jul 8, 2026

Copy link
Copy Markdown

PER-14389

@netlify

netlify Bot commented Jul 8, 2026

Copy link
Copy Markdown

Deploy Preview for permitio-docs ready!

Name Link
🔨 Latest commit 80288b6
🔍 Latest deploy log https://app.netlify.com/projects/permitio-docs/deploys/6a625af316d3830008eecad7
😎 Deploy Preview https://deploy-preview-647--permitio-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new customer-facing, enterprise-only on-prem guide that walks users through importing a Permit-provided SaaS → on-prem migration package (requesting the package, integrity verification, safe/ordered DB import, post-import cleanup, verification, and app cutover guidance).

Changes:

  • Introduces a new on-prem migration guide page covering package request/process and detailed import steps.
  • Documents operational safety steps (write-freeze, pre-import DB backup, fail-fast import loops) and post-import cleanup/verification.
  • Adds team-member re-access guidance (Keycloak email linking + verified-email requirement) and an SDK endpoint cutover snippet.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/how-to/deploy/on-prem/migrate-from-cloud.mdx Outdated
Comment thread docs/how-to/deploy/on-prem/migrate-from-cloud.mdx Outdated
EliMoshkovich and others added 3 commits July 8, 2026 14:55
Critical: the SDK cutover snippet omitted the api_url override, so a
migrated app's management calls would silently keep writing to the old
Permit Cloud workspace; it also pointed pdp= at the platform domain
instead of an on-prem PDP deployment. Now shows both endpoints with a
do-not-skip caution and links PDP Deployment.

Over-disclosure removed:
- 41-table schema list replaced with a manifest-driven loop
  (manifest.import.tables_in_order / fk_disabled_tables — contract added
  to the export in the backend PR), so public docs carry no schema dump
  and can't drift per installer version
- avp_policy_store_id cleanup SQL (cloud-provider internals) moved to
  the package README; docs keep only the customer-relevant review bullets
- unilateral commitments softened: audit retention period, download-link
  expiry, "without code changes"/"automatically regain access" guarantees

Friendliness: "At a glance" orientation (phases, downtime window, done
criteria, Permit assistance), pg_restore rollback snippet, jq/tar/sha256
prerequisites, replica-count note before scale-down, FK-bypass scoping
caution, platform-wide superuser caution, dependency-order gloss, and
the standard Support footer.

Build: 0 bad links, 0 bad anchors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
FK-graph verification against the schema found the documented ordered
import cannot work for all customers: v2_project <-> v2_policy_repo
reference each other (a genuine FK cycle, hit by any GitOps org), plus
two more order violations masked in the original validation by empty
tables. The import now runs as one FK-deferred transaction
(--single-transaction + ON_ERROR_STOP): all-or-nothing, safe because the
package is a single consistent snapshot, and simpler for customers -
any failure rolls back cleanly and the step is just rerun.

Troubleshooting updated accordingly (partial-import scenario no longer
exists; added a guard entry for packages predating manifest.import).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Replace --single-transaction with explicit BEGIN/COMMIT: on psql <= 14 a
  client-side \COPY failure under --single-transaction COMMITs the
  already-loaded tables (rollback-on-client-error landed in psql 15),
  breaking the all-or-nothing promise; explicit BEGIN/COMMIT is atomic on
  every version (verified empirically on psql 14 and 16).
- Admonition: "turned off" not "deferred" (no DEFERRABLE semantics), and
  forward-reference the step-6 member cleanup as the one known exception
  to snapshot consistency.
- Step 8: log out/in only works after the step-9 restart.
- Step 9: per-deployment scale commands (one --replicas flag can't express
  three different counts).
- Troubleshooting: "prints no COPY lines" (psql still prints SET/BEGIN).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@zeevmoney zeevmoney left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an automated review. See comments.

Comment thread docs/how-to/deploy/on-prem/migrate-from-cloud.mdx Outdated
- api_url must not carry an /api suffix: on-prem routes only /v2/ through
  the frontend nginx to the backend (verified against the platform chart
  ingress and permit-backend APP_PREFIX); with /api every SDK management
  call lands on the SPA fallback and returns HTML (Zeev's finding)
- scale-down command uses the explicit deployment/<name> resource form
  with flags first (portable across kubectl versions; Copilot's finding)
- management.mdx backend health check: /api/v2/health -> /v2/healthy
  (the actual backend route; same /api routing hole as above)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@EliMoshkovich
EliMoshkovich requested a review from zeevmoney July 13, 2026 20:13

@zeevmoney zeevmoney left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Review - the PR looks clean, found nothing.

EliMoshkovich and others added 2 commits July 23, 2026 10:26
Reframes the migrate-from-cloud guide so customers run the whole migration
themselves. Adds "Step 0: Export Your Data from Permit Cloud":

- generate an Organization API key (management access) in Settings -> API Keys
- call GET /v2/data-export with the Bearer token to download the package
- freeze-writes, version-alignment, and audit-log notes; 413 -> ask Permit to
  run the offline export for very large orgs

Reconciles the rest of the page with the self-service package (data/ +
manifest.json only): package-contents tree, policy-repo guidance (ask Permit
for the default repo clone), self-contained cloud-config cleanup (no package
README), prerequisites and filenames, and the "no COPY lines" troubleshooting.
The import steps (single-transaction FK-deferred) are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…guide

The customer migration page no longer mentions audit-log history export
(dropped the "not included / request from your account team" notes in Step 0
and in "What is not migrated"). The page is dedicated to the RDS data
migration: self-serve export via the API, then import into the customer's own
on-prem deployment.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@EliMoshkovich EliMoshkovich changed the title PER-14389: Migrate from Permit Cloud — on-prem import guide PER-14389: Migrate from Permit Cloud — self-service export + on-prem import guide Jul 23, 2026
@EliMoshkovich
EliMoshkovich requested a review from omer9564 July 23, 2026 16:40
Step 0 now documents the export as start -> poll -> download: POST
/v2/data-export to start, poll GET /v2/data-export/{task_id}, then download
from the short-lived presigned URL in result.download_url. Notes the link is
short-lived + secret, and the too-large case surfaces as a poll failure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants