Merge dev into main: releases/symlink prod deploy pipeline + accumulated dev work - #456
Merged
Conversation
- Added AGENTS.md to document breaking changes in the Next.js version. - Updated .gitignore to include new generated files and directories. - Refactored ESLint configuration to use new imports and improved ignore patterns. - Enhanced i18n request handling to default to a specified locale if the incoming locale is invalid. - Updated Next.js configuration to resolve alias for next-intl and adjusted experimental settings. - Upgraded package dependencies, including Next.js and TypeScript-related packages. - Introduced a new proxy middleware for handling collaborative routes and authentication. - Added skills-lock.json to manage skills configuration. - Updated TypeScript configuration to include new paths and improved JSX handling. - Cleaned up layout.tsx to use stable API for setting request locale.
Update project configuration and add new features
… gate - npm ci (not npm install) so the build always matches package-lock.json, plus a guard that aborts if the installed Next.js version drifts from package.json - ship a versioned release.tar.gz, extract into releases/<release>/ on EC2, install deps there, verify the Next.js version again, then flip the DataExFrontend symlink atomically - boot health check after activation; auto-rolls back to the previous release on failure - dev: promote-dev/rollback-dev jobs make smoke-tests an actual deploy gate — release is only marked last-known-good on a smoke-test pass, and reverted on a smoke-test failure - prod: no smoke-test job exists yet, so the boot health check is its only gate; release is marked good immediately after - add deploy/ec2-migrate-to-releases.sh, the one-time manual migration from a plain DataExFrontend directory to the releases/ layout
The app's actual runtime secrets file on EC2 is .env.local (inside DataExFrontend itself), not .env one level up in DataExchange/ - that one belongs to a separate docker-compose stack (DataExAuth/DataExBackend/ DataExKeycloak). Confirmed via `pm2 env` showing no secrets in PM2's own captured environment, and .env.local's size/presence matching .env.local.example. Without this fix the new releases/ workflow would have symlinked the wrong file and every release would boot with no runtime config.
…ality-gate # Conflicts: # .github/workflows/deploy-Dataspace.yml
…-gate ci: releases/symlink deploy with health-check rollback and smoke-test gate
PR #441's deploy just failed: appleboy/ssh-action's non-interactive shell never sources nvm, leaving PATH pointed at the ancient system node (v10). npm and pm2 are both scripts with a '#!/usr/bin/env node' shebang, so invoking them by absolute path wasn't enough - env still re-resolved node via PATH and picked v10, which can't parse npm v24's node: imports ('Cannot find module node:path'). Failed before touching the DataExFrontend symlink, so the previous release stayed live throughout - no downtime from this. Fix: export NODE_BIN onto PATH before any npm/pm2 invocation, in both the deploy workflow (build-and-deploy activation step, rollback-dev) and the migration script. Verified live on dev-cds that node/npm/pm2 all resolve to v24 with this PATH export.
- Added BhashiniTranslation component to support multilingual capabilities across the dashboard. - Updated CSS to ensure proper styling for the translation plugin and maintain UI integrity. - Enhanced error handling in GraphQL functions to validate document structure.
Confirmed via a live failed deploy: next.config.mjs runs jiti('./env')
synchronously every time `next start` boots (not just at build time),
so env.ts has to physically exist in the release directory or the
server crash-loops with 'Cannot find module ./env' - the exact failure
the boot health check just caught, correctly triggering the auto-rollback
with zero downtime.
i18n.ts is also referenced in next.config.mjs, but only as a build-time
webpack alias target for next-intl - it's already resolved into .next's
bundled output during npm run build, so it doesn't need to ship
standalone (the crash trace only ever mentioned env.ts, not i18n.ts).
Confirmed via a live deploy: the app was actually healthy and correctly activated (release/health-check both succeeded, dev.civicdataspace.in was serving 200s), but the whole job still reported failure. Root cause: 'grep -v' exits 1 when it selects zero lines, which happens whenever there are fewer than 6 releases to prune. Under 'set -o pipefail' that non-zero code killed the script right after printing the success message, so GitHub Actions reported this deploy as failed despite it having actually succeeded - which also meant smoke-tests/promote-dev never ran, since they need build-and-deploy to report success. Pruning old releases is best-effort housekeeping, never something that should fail the deploy - added '|| true' to make 'nothing to prune' a non-error.
Confirmed via a live run: build-and-deploy and all three smoke-tests suites passed, but promote-dev still failed with 'missing server host' from appleboy/ssh-action. vars.EC2_HOST is an environment-scoped variable (Settings -> Environments -> development), only visible to a job that declares environment: - build-and-deploy has it, but promote-dev/rollback-dev never did, so vars.EC2_HOST silently resolved to empty for them. Both jobs are already gated by if: github.ref_name == 'dev', so a fixed environment: development (rather than build-and-deploy's ternary) is correct here.
…components - Removed unused ESLint rules for cleaner configuration. - Updated fetch functions to use specific types instead of 'any' for better type safety. - Enhanced type definitions in collaborative components and dataset handling. - Refactored layout component to utilize useSyncExternalStore for state management. - Improved metadata and details components with stricter type definitions. - General cleanup of type usage in various components for consistency and clarity.
Refactor dashboard components and enhance translation support
Privacy-policy
run-smoke.yml now preflights its Keycloak configuration instead of letting the authenticated API tests skip silently and the run go green on nothing. It needs the client secret to do that. `dataspace` is a confidential client, so the ROPC token request returns 401 unauthorized_client without it. The reusable workflow declares the secret optional so this repo kept parsing before this change, but api-smoke fails its preflight until the secret is passed and set.
…smoke ci: pass KEYCLOAK_CLIENT_SECRET to the smoke workflow
One client issued 228,211 requests in a day, peaking at 6,230 per
minute: 107,612 session fetches, 36,565 csrf, 35,685 signouts. It
exhausted the backend's per-IP rate limit (1000/hour for non-GET) and
returned 429s to every other user behind the same NAT address. 429s went
0 -> 7 -> 19,991 over three days.
Two pieces combined into an unbounded cycle:
1. The jwt callback returned {...token, error: 'RefreshAccessTokenError'}
while keeping the stale expires_at, so the next session fetch saw an
expired token and retried the same refresh, which failed the same way.
2. SessionGuard reacted to that error by calling signOut({redirect:
false}) with no guard - so useSession refetched, the error was still
present, and the effect fired again. Nothing bounded it.
It was also self-sustaining: once over the limit, the signOut call
itself returned 429, so the session was never cleared and the condition
could not resolve.
Fixes both halves. The jwt callback short-circuits when the token
already carries the error rather than retrying a refresh that cannot
succeed - the session is unrecoverable at that point and hammering
Keycloak helps nobody. SessionGuard tracks whether it has already acted,
so cleanup runs once per error rather than once per render, and swallows
a failed signOut instead of spinning on it. The flag clears when the
session recovers, so a later expiry is still handled.
Also declares `error` on the JWT type. It compiled without that because
JWT extends Record<string, unknown>, but the read was typed `unknown`.
Trigger, for the record: the backend row-lock fix took requests from
3-60s to 0.2-0.5s. The loop pre-existed; the slow backend had been
throttling it below the rate limit.
fix: stop the NextAuth signout loop that rate-limited every user behind one IP
refactor: update ESLint configuration and improve type safety across components
Added Privacy in Footer
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
dev's releases/symlink deploy pipeline (deploy-Dataspace.yml, PR ci: releases/symlink deploy with health-check rollback and smoke-test gate #441) onmain, givingmainrealenvironment: productionbranching for the first time.mainup to date with everything merged todevsince the last sync.Notes
productionGitHub Environment has a required-reviewer gate, so the deploy job will sit pending approval until manually approved.deploy/ec2-migrate-to-releases.shmigration run before that pending deployment should be approved — tracked separately, not part of this PR.Test plan
productiondeployment shows as pending/awaiting approval, not auto-run