release: prepare 17.0.0 - #833
Open
Ngwerume wants to merge 19 commits into
Open
Conversation
Up to standards ✅🟢 Issues
|
![codacy-production[bot]](https://avatars.githubusercontent.com/in/56611?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull Request Overview
This pull request updates the Helm chart configuration for release preparation. The modifications are compliant with automated quality standards; however, the pull request lacks a description. Providing context for configuration changes is recommended to assist with long-term maintainability.
About this PR
- The PR description is empty. Please provide context regarding the reasons for the configuration updates to improve documentation and traceability.
Test suggestions
- Verify that the Helm chart values file is valid YAML after manual indentation changes.
- Validate that the installation and documentation strings render correctly in templates.
- Confirm the availability and integrity of the worker image.
Prompt proposal for missing tests
Consider implementing these tests if applicable:
1. Verify that the Helm chart values file is valid YAML after manual indentation changes.
2. Validate that the installation and documentation strings render correctly in templates.
3. Confirm the availability and integrity of the worker image.
TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback
| global: | ||
| imagePullSecrets: | ||
| - docker-credentials | ||
| - docker-credentials |
There was a problem hiding this comment.
⚪ LOW RISK
Nitpick: The indentation of this list item is inconsistent with the 2-space convention used elsewhere in the file.
The deploy_to_doks_release job (job 30732) timed out because the CircleCI no_output_timeout (15m) matched the helm --timeout (900s). Helm produces no output while waiting for pods to roll out, so CircleCI killed the process just as helm was about to hit its own timeout. Fix: raise no_output_timeout to 30m and helm --timeout to 1500s (25m) so helm has room to complete or fail cleanly before CircleCI intervenes. AI-Generated: true Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com>
…ded (#836) * Clean up stale Kubernetes jobs before helm upgrade in release deployment Previous failed deployments leave hook jobs (e.g. DB migrations) behind in the codacy-release namespace. When helm tries to run the same post-upgrade hook on the next attempt it finds an already-failed job and immediately reports BackoffLimitExceeded, causing the deploy_to_doks_release CI job to fail (see job 30795). Add a clean_jobs Makefile target that deletes all jobs in the namespace before the helm upgrade runs. The codacy-release namespace is a dedicated Helm release namespace, so any lingering jobs are stale hook artifacts from prior failed runs and are safe to remove. AI-Generated: true * Apply suggestion from @codacy-production[bot] Co-authored-by: codacy-production[bot] <61871480+codacy-production[bot]@users.noreply.github.com> * Reduce the delete jobs scope --------- Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com> Co-authored-by: Ricardo Bernardino <ricardo.bernardino@codacy.com> Co-authored-by: codacy-production[bot] <61871480+codacy-production[bot]@users.noreply.github.com>
…837) The clean_jobs Makefile target ran kubectl delete jobs without specifying the --namespace flag, so it searched the default namespace instead of the release namespace (e.g. codacy-release). This meant stale hook jobs (DB migrations etc.) from prior failed deployments were never cleaned up, and helm's next attempt would find the existing BackoffLimitExceeded job and immediately fail again. Add --namespace ${NAMESPACE} to the kubectl delete command so it targets the correct namespace for every environment (dev, sandbox, release). AI-Generated: true Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com>
…838) When a post-upgrade helm hook job fails with BackoffLimitExceeded, the failed job lingers in the namespace. The --atomic flag causes helm to roll back, leaving the hook job behind. On the next CI run, clean_jobs removes it before the upgrade, but the hook can fail again transiently. This change converts deploy_to_doks_from_chartmuseum from a prerequisite-chain target to a recipe that cleans up and retries the helm install once if the first attempt fails, covering transient hook failures (e.g. delete-rabbitmq-queues running before RabbitMQ is ready). AI-Generated: true Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com>
…ness wait (#839) The post-upgrade hook job was spawning 100 pods (backoffLimit: 100, restartPolicy: Never) each failing in ~9s because MinIO isn't ready immediately after a Helm upgrade, totalling ~15 minutes before BackoffLimitExceeded and causing helm --atomic to roll back. Changes: - Add a wait loop in the hook container that polls MinIO (aws s3 ls) every 10s for up to 10 minutes before attempting put-bucket-lifecycle-configuration - Reduce backoffLimit from 100 to 3 since retries are now handled within the container - Add before-hook-creation to hook-delete-policy so any stale failed job is always cleaned up before the next upgrade attempt creates a new one AI-Generated: true Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com>
… pre-push delete (#840) Two fixes for the persistent deploy_to_doks_release failure (job 31035): 1. codacy/templates/fluentd/lifecycle-police-job.yaml: Add "Filter":{} to the lifecycle configuration JSON. Newer MinIO versions strictly follow the AWS S3 spec which requires a Filter element in each lifecycle rule, even when empty. Without it, put-bucket-lifecycle-configuration returns a MalformedXML error immediately, causing 3 fast pod failures and BackoffLimitExceeded (the ~15 min total time is main-deployment rollout; the hook itself fails in seconds). 2. .circleci/config.yml: Delete the existing chart version from chart museum before pushing. Chart museum rejects re-uploads of the same version with 409, which failed the first pipeline run (job 31005). Adding a pre-push DELETE via the chart museum REST API (with || true to tolerate missing charts) ensures subsequent CI runs can always push the updated chart. AI-Generated: true Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com>
…IO (#841) The previous fix (PR #839) waited for MinIO to respond to `aws s3 ls` before calling put-bucket-lifecycle-configuration. However, MinIO can be accessible while the specific bucket doesn't exist yet (bucket creation is a separate hook). This caused rapid failures of the lifecycle config command, hitting BackoffLimitExceeded (with backoffLimit:3) long before Helm's 1500s timeout. Changes: - Wait for the specific bucket (`aws s3api head-bucket`) instead of just MinIO (`aws s3 ls`), so the lifecycle config only runs once the bucket actually exists - Extend MAX_WAIT to 1200s (fits within Helm's 1500s timeout as a single container, no Kubernetes backoff overhead) - Set backoffLimit:0 so Kubernetes doesn't add exponential backoff delays between retries — all retry logic is in-shell - Add an in-shell retry loop (up to 5 attempts) for put-bucket-lifecycle-configuration to handle transient API errors after the bucket is confirmed ready AI-Generated: true Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com> Co-authored-by: Ricardo Bernardino <ricardo.bernardino@codacy.com>
The stable/minio chart (v5.0.33) uses old-style Helm labels (release: <name>, app: minio-make-bucket-job) and has hook-delete-policy: hook-succeeded only. When the job fails, it is not deleted by Helm on rollback. clean_jobs was filtering by app.kubernetes.io/instance which doesn't exist on old-style chart resources, so the stuck job was never cleaned up. The second helm upgrade attempt would then fail with "failed to deploy codacy-minio-make-bucket-job" because the job already existed. Add a second kubectl delete with the legacy release=<name> label selector so clean_jobs catches both new- and old-style Helm-labelled jobs. AI-Generated: true Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.