Overview
Architecture, evolution and continuous operation of an Amazon EKS platform used to run SaaS workloads in a complex SaaS environment. The platform operated +15 clusters distributed globally, sustaining +1,700 containers in 24x7 operations.
Context
A SaaS platform growing in workload volume, deployment complexity and availability requirements. Multiple Amazon EKS clusters distributed geographically, each with its own capacity, networking and AWS service integration characteristics. Operations needed to support continuous releases, horizontal scalability and troubleshooting across interdependent layers.
Challenge
Balance compute, persistence, networking, observability, automation and cost across a platform with +15 clusters and hundreds of workloads, maintaining stability in 24x7 operations. Every architectural decision simultaneously impacted security, performance, cost and platform evolution capacity.
My role
- Amazon EKS cluster architecture and evolution — node groups, compute capacity and scaling
- Kubernetes networking design and maintenance: VPC, subnets, routing, ingress and load balancing
- IAM, EFS, AWS services and identity integration
- Troubleshooting issues spanning application, Kubernetes, networking, storage and AWS services
- Observability with CloudWatch, Grafana, Loki and Datadog — dashboards, alerts and log pipelines
- Terraform, Helm and CI/CD automation for repeatable and traceable changes
- Performance, capacity and compute/storage cost analysis
Architecture
The architecture separated cloud infrastructure from Kubernetes workloads. Application traffic moved through controlled protection layers (WAF), load balancing and ingress before reaching cluster services. Each cluster operated with segmented node groups by workload profile, enabling independent capacity and cost decisions. Shared persistence via EFS was used only where workload architecture justified it.
Ingress Architecture Migration
The platform's ingress layer relied on Nginx Ingress Controller as a critical component in the application traffic path. As the platform evolved and demanded greater routing flexibility, traffic control and observability, a migration to an Envoy-based architecture was initiated.
The migration involved analyzing existing routing rules, validating application compatibility, configuring the new ingress layer, regression testing, comparative observability and controlled rollout across clusters. Each step was conducted incrementally, with rollback capability at any point.
Replacing an ingress controller in production required work across critical components: HTTP/HTTPS routing, TLS, load balancing, Kubernetes services, health checks, headers, timeouts, logging and security. The migration was completed without impacting platform operations.
Technical decisions
- Clear separation between Kubernetes infrastructure and application workload responsibilities
- Compute choices driven by workload profile, availability and cost — not convention
- Shared persistence only where workload architecture warranted the overhead
- Controlled service exposure with ingress layer and segmented security groups
- Observability as part of the platform, not a retrospective tool
Security & governance
Identity integrations, network segmentation and service exposure were treated as architectural decisions from the start. Least privilege and defense in depth principles guided security group definitions, per-workload IAM roles and inter-cluster, inter-account network policies.
Automation
Terraform managed Kubernetes infrastructure and dependent AWS services. Helm standardized workload deployment with environment-specific values. CI/CD pipelines automated build, test and deploy with change review before production application. Traceable, auditable and repeatable changes across distributed environments.
Engineering challenges
Incident investigations required correlating application, Kubernetes, networking, storage and AWS service behavior. An apparently application-level problem could be caused by network restrictions, storage contention or node group limitations. Investigating without isolating layers was essential for efficient resolution.
Results
- +15 clusters operating in a standardized and sustainable manner
- Infrastructure changes with complete traceability via IaC and pipelines
- Troubleshooting with multi-layer correlation capability
- Foundation prepared for workload growth and new cluster additions