This project deploys a Python Flask digital library microservices application on AWS using a DevOps pipeline. Infrastructure is provisioned with Terraform, Docker images are stored in Amazon ECR, and the application is deployed to Amazon EKS through Jenkins CI/CD. Helm, ArgoCD, Prometheus, and Grafana were added to make the project closer to a production-style Kubernetes workflow.
The application contains multiple Python Flask services connected to a MySQL database:
- Frontend service for user interaction
- Auth service for signup and signin
- Book service for book listing
- Borrow service for borrowing records
- MySQL database for persistent application data
The project demonstrates cloud infrastructure provisioning, containerisation, CI/CD automation, Kubernetes deployment, GitOps, and monitoring.
Note: AWS resources were destroyed after testing to avoid ongoing cloud costs. Deployment proof is included in the screenshots section.
This was built in stages rather than all at once. Each stage added one layer:
| Stage | What was added |
|---|---|
| 1. Docker Compose | Containerised the four Flask services and MySQL on a custom Docker network |
| 2. Kubernetes | Moved to raw manifests — Deployments, Services, ConfigMaps and Secrets |
| 3. Terraform + EKS | VPC, subnets, IGW, NAT gateway, EKS cluster, managed node group and ECR, all as code |
| 4. CI/CD | Jenkins pipeline with a SonarQube quality gate and Trivy image scanning |
| 5. Helm + GitOps | Packaged the manifests as a chart, then had Argo CD sync it from Git |
| 6. Observability | Prometheus and Grafana via kube-prometheus-stack |
The stage 1 images are still published on Docker Hub as
dinesh78900/pyapp — one tag
per service (frontend, auth_image, book_image, borrow_image, db_image)
from before the move to ECR.
The same application also runs on Azure AKS — see digital-library-azure-aks for the provider-by-provider comparison.
| Area | Tools |
|---|---|
| Cloud | AWS |
| Infrastructure as Code | Terraform |
| Container Registry | Amazon ECR |
| Containerisation | Docker |
| Orchestration | Amazon EKS, Kubernetes |
| CI/CD | Jenkins |
| Code Quality | SonarQube |
| Image Scanning | Trivy |
| Packaging | Helm |
| GitOps | ArgoCD |
| Monitoring | Prometheus, Grafana |
| Application | Python Flask |
| Database | MySQL |
The architecture uses AWS EKS to run the microservices application. Users access the frontend through an AWS Load Balancer. The frontend communicates with backend Flask services running inside the EKS cluster, and the backend services connect to a MySQL pod for application data.
The Jenkins pipeline pulls code from GitHub, runs SonarQube code analysis, builds Docker images, scans images using Trivy, pushes images to Amazon ECR, and deploys the application to Amazon EKS.
ArgoCD watches the GitHub repository and synchronises the Helm chart with the EKS cluster. Prometheus collects Kubernetes metrics, and Grafana visualises node health, pod status, deployment availability, CPU usage, and memory usage.
GitHub
-> Jenkins CI/CD
-> SonarQube code analysis
-> Docker image build
-> Trivy image scan
-> Amazon ECR
-> Amazon EKS
-> Kubernetes services and pods
-> Prometheus/Grafana monitoring
| Service | Port | Purpose |
|---|---|---|
| Frontend | 5000 | Web UI |
| Auth Service | 5001 | Signup and signin |
| Book Service | 5002 | Book listing |
| Borrow Service | 5003 | Borrow records |
| MySQL | 3306 | Database |
The Jenkins pipeline performs:
- Clean workspace
- Checkout code from GitHub
- SonarQube code quality analysis
- Quality gate validation
- Docker image build for all services
- Trivy vulnerability scan
- Login to Amazon ECR
- Push images to ECR
- Connect to EKS
- Deploy Kubernetes manifests
- Verify pods, services, and deployments
Terraform provisions the AWS infrastructure:
- VPC
- Public and private subnets
- Internet Gateway
- NAT Gateway
- EKS cluster
- EKS managed node group
- ECR repositories for all services
cd terraform
terraform init
terraform plan
terraform applyThe application was first deployed using raw Kubernetes manifests in the k8s/ directory.
kubectl apply -f k8s/namespace.yaml
kubectl apply -f k8s/database/
kubectl apply -f k8s/application/
kubectl apply -f k8s/auth/
kubectl apply -f k8s/book/
kubectl apply -f k8s/borrow/
kubectl apply -f k8s/frontend/A Helm chart was added to package the Kubernetes deployment.
helm upgrade --install digital-library ./helm/digital-library \
-n library \
--create-namespaceArgoCD was installed on EKS and configured to sync the Helm chart from GitHub.
kubectl get applications -n argocdThe ArgoCD application reached:
Synced
Healthy
Monitoring was added using kube-prometheus-stack, which installs:
- Prometheus
- Grafana
- kube-state-metrics
- node-exporter
Grafana was used to view Kubernetes cluster, node, pod, and workload metrics.
├── architecture/
├── Jenkinsfile
├── terraform/
├── k8s/
├── helm/
├── argocd/
├── monitoring/
├── database/
├── auth-service/
├── book-service/
├── borrow-service/
├── frontend/
├── screenshots/
└── README.md
- Provisioning AWS EKS and ECR using Terraform
- Building CI/CD pipelines with Jenkins
- Building and pushing Docker images to ECR
- Deploying Python microservices to Kubernetes
- Using ConfigMaps, Secrets, Services, Deployments, and LoadBalancer
- Packaging Kubernetes resources with Helm
- Syncing Kubernetes deployments using ArgoCD
- Monitoring EKS workloads using Prometheus and Grafana
- Debugging IAM, ECR, Kubernetes, and database deployment issues
AWS resources were removed after testing to avoid unnecessary cost.
cd terraform
terraform destroy- Add AWS Load Balancer Controller and Ingress
- Add HTTPS using ACM and Route 53
- Store secrets using AWS Secrets Manager
- Use remote Terraform backend with S3 and DynamoDB locking
- Add namespace-level RBAC for Jenkins instead of cluster admin
- Add Prometheus alerts for pod restarts and high resource usage
- Add GitHub Actions alternative pipeline