π― Introduction
Kubernetes Ingress is being deprecated! With Ingress NGINX scheduled for decommissioning in March 2026, it's time to migrate to the Gateway API - the official successor that offers better role separation, portability, and expressiveness.
In this comprehensive guide, I'll walk you through deploying a production-ready Gateway API setup on AWS EKS using Envoy Gateway, demonstrating the modern approach to Kubernetes traffic management.
π Table of Contents
- Why Gateway API?
- Architecture Overview
- Prerequisites
- Understanding Gateway API Components
- Step-by-Step Implementation
- Testing and Verification
- Troubleshooting
- Cleanup
- Conclusion
π Why Gateway API?
The Problem with Ingress
Traditional Kubernetes Ingress has several limitations:
- Monolithic design: Single resource controls everything
- Vendor-specific annotations: Not portable across providers
- Limited expressiveness: Can't handle advanced routing scenarios
- No role separation: Cluster admins and developers share the same resource
- Being deprecated: Ingress NGINX decommissioning in March 2026
Gateway API Benefits
Gateway API solves these problems with:
β
Role-based model: Clear separation between infrastructure, cluster admin, and developer concerns
β
Portable & consistent: Standard API across all providers (AWS, GCP, Azure, on-prem)
β
More expressive: Header-based routing, query parameters, traffic splitting, request/response modification
β
Future-proof: Official Kubernetes successor, graduated to GA (v1.0.0)
β
Better security: Fine-grained RBAC and policy attachment
ποΈ Architecture Overview
Ingress vs Gateway API Flow
Traditional Ingress:
Client β External LB (Ingress Controller) β Ingress Resource β Service β Pods
Modern Gateway API:
Client Request
β
External Load Balancer (NLB)
β
Gateway Controller Pod (Envoy)
β
GatewayClass (Infra Provider)
β
Gateway (Cluster Admin)
β
HTTPRoute (Developers)
β
Service β Pods
Role Separation
| Role | Responsibility | Resources |
|---|---|---|
| Infrastructure Provider | Provides Gateway implementation | GatewayClass |
| Cluster Administrator | Configures listeners, TLS, load balancers | Gateway |
| Application Developer | Defines routing rules for their apps | HTTPRoute, TCPRoute, etc. |
π¦ Prerequisites
Before starting, ensure you have:
- AWS Account with appropriate permissions
-
AWS CLI configured (
aws configure) - kubectl (v1.28+)
- eksctl (v0.150+)
- Helm (v3.0+)
- Basic understanding of Kubernetes concepts
Install Required Tools
# Install eksctl (macOS)
brew tap weaveworks/tap
brew install weaveworks/tap/eksctl
# Install kubectl
brew install kubectl
# Install Helm
brew install helm
# Verify installations
eksctl version
kubectl version --client
helm version
π§© Understanding Gateway API Components
1. GatewayClass
What it is: Defines the controller implementation (like a StorageClass for storage)
Who manages it: Infrastructure provider (platform team)
Purpose: Specifies which Gateway controller will handle Gateway resources
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: eg
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
parametersRef:
group: gateway.envoyproxy.io
kind: EnvoyProxy
name: internet-facing-proxy
namespace: envoy-gateway-system
Key Points:
- One GatewayClass per controller type
- Can have multiple GatewayClasses (e.g., internal, external, premium)
- Immutable once created
2. Gateway
What it is: Defines the load balancer configuration and listeners
Who manages it: Cluster administrator
Purpose: Configures how traffic enters the cluster (ports, protocols, TLS)
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: demo-gateway
spec:
gatewayClassName: eg
listeners:
- name: http
protocol: HTTP
port: 80
- name: https
protocol: HTTPS
port: 443
tls:
certificateRefs:
- name: my-cert
Key Points:
- Creates the actual load balancer (NLB in AWS)
- Defines listeners (HTTP, HTTPS, TCP, UDP)
- Manages TLS certificates
- Can have multiple listeners on different ports
3. HTTPRoute
What it is: Defines routing rules for HTTP traffic
Who manages it: Application developers
Purpose: Routes traffic from Gateway to backend services
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: backend-route
spec:
parentRefs:
- name: demo-gateway
rules:
- matches:
- path:
type: PathPrefix
value: /api
backendRefs:
- name: backend-service
port: 80
Key Points:
- Attached to a Gateway via
parentRefs - Supports path, header, query parameter matching
- Can split traffic between multiple backends
- Supports request/response modification
4. EnvoyProxy (Envoy Gateway Specific)
What it is: Configuration for Envoy Gateway's data plane
Who manages it: Infrastructure provider
Purpose: Customizes how Envoy Gateway creates load balancers
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
name: internet-facing-proxy
namespace: envoy-gateway-system
spec:
provider:
type: Kubernetes
kubernetes:
envoyService:
type: LoadBalancer
annotations:
service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
Key Points:
- Envoy Gateway specific (not part of standard Gateway API)
- Controls load balancer type (internal vs internet-facing)
- Configures resource limits, replicas, etc.
π οΈ Step-by-Step Implementation
Step 1: Create EKS Cluster
Create a new EKS cluster with Kubernetes 1.31:
eksctl create cluster \
--name eks-gateway-demo \
--region us-east-1 \
--version 1.31 \
--nodegroup-name gateway-nodes \
--node-type t3.medium \
--nodes 2 \
--nodes-min 2 \
--nodes-max 4 \
--managed
What this does:
- Creates a new EKS cluster named
eks-gateway-demo - Uses Kubernetes version 1.31 (latest stable)
- Creates 2 t3.medium nodes in a managed node group
- Sets up VPC, subnets, security groups automatically
Time: ~15-20 minutes
Verify:
kubectl get nodes
# Should show 2 nodes in Ready state
Step 2: Install Gateway API CRDs
Install the Gateway API Custom Resource Definitions:
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/standard-install.yaml
What this installs:
- GatewayClass CRD
- Gateway CRD
- HTTPRoute CRD
- ReferenceGrant CRD
- TCPRoute, UDPRoute, TLSRoute CRDs
Verify:
kubectl get crd | grep gateway
# Should show gateway-related CRDs
Step 3: Install Envoy Gateway
Install Envoy Gateway as the Gateway controller:
kubectl apply -f https://github.com/envoyproxy/gateway/releases/download/v1.0.0/install.yaml
What this installs:
- Envoy Gateway controller deployment
- Required RBAC (ServiceAccount, ClusterRole, ClusterRoleBinding)
- Envoy Gateway configuration
- Webhook configurations
Wait for it to be ready:
kubectl wait --timeout=2m -n envoy-gateway-system \
deployment/envoy-gateway --for=condition=Available
Verify:
kubectl get pods -n envoy-gateway-system
# Should show envoy-gateway pod running
Step 4: Configure Internet-Facing Load Balancer
Create EnvoyProxy configuration for internet-facing NLB:
kubectl apply -f - << 'EOF'
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
name: internet-facing-proxy
namespace: envoy-gateway-system
spec:
provider:
type: Kubernetes
kubernetes:
envoyService:
type: LoadBalancer
annotations:
service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
EOF
Why this is needed:
- By default, Envoy Gateway creates internal load balancers
- This configuration makes the NLB internet-facing
- AWS-specific annotation controls the load balancer scheme
Step 5: Create GatewayClass
Create a GatewayClass that references the EnvoyProxy configuration:
kubectl apply -f - << 'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: eg
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
parametersRef:
group: gateway.envoyproxy.io
kind: EnvoyProxy
name: internet-facing-proxy
namespace: envoy-gateway-system
EOF
Verify:
kubectl get gatewayclass
# Should show 'eg' with ACCEPTED=True
Step 6: Deploy Sample Applications
Deploy backend and frontend applications:
# Backend Application
kubectl apply -f - << 'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
name: backend
spec:
replicas: 2
selector:
matchLabels:
app: backend
template:
metadata:
labels:
app: backend
spec:
containers:
- name: backend
image: hashicorp/http-echo
args:
- "-text=Backend API Response"
ports:
- containerPort: 5678
---
apiVersion: v1
kind: Service
metadata:
name: backend-service
spec:
selector:
app: backend
ports:
- port: 80
targetPort: 5678
EOF
# Frontend Application
kubectl apply -f - << 'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
name: frontend
spec:
replicas: 2
selector:
matchLabels:
app: frontend
template:
metadata:
labels:
app: frontend
spec:
containers:
- name: frontend
image: hashicorp/http-echo
args:
- "-text=Frontend UI Response"
ports:
- containerPort: 5678
---
apiVersion: v1
kind: Service
metadata:
name: frontend-service
spec:
selector:
app: frontend
ports:
- port: 80
targetPort: 5678
EOF
Verify:
kubectl get pods
kubectl get svc
# Should show 4 pods (2 backend + 2 frontend) and 2 services
Step 7: Create Gateway
Create the Gateway resource that provisions the NLB:
kubectl apply -f - << 'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: demo-gateway
spec:
gatewayClassName: eg
listeners:
- name: http
protocol: HTTP
port: 80
EOF
What happens:
- Envoy Gateway controller sees the new Gateway
- Creates an Envoy proxy deployment
- Creates a LoadBalancer Service
- AWS provisions an internet-facing NLB
- NLB targets are registered (takes 2-3 minutes)
Monitor Gateway creation:
kubectl get gateway demo-gateway -w
# Wait for ADDRESS to be populated and PROGRAMMED=True
Get the NLB DNS:
kubectl get gateway demo-gateway -o jsonpath='{.status.addresses[0].value}'
Step 8: Create HTTPRoutes
Create routing rules for backend and frontend:
# Backend Route
kubectl apply -f - << 'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: backend-route
spec:
parentRefs:
- name: demo-gateway
rules:
- matches:
- path:
type: PathPrefix
value: /api
backendRefs:
- name: backend-service
port: 80
EOF
# Frontend Route
kubectl apply -f - << 'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: frontend-route
spec:
parentRefs:
- name: demo-gateway
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: frontend-service
port: 80
EOF
How HTTPRoute works:
- parentRefs: Attaches the route to a Gateway
- matches: Defines conditions for routing (path, headers, query params)
- backendRefs: Specifies which Service to route to
Path matching order:
- More specific paths are matched first
-
/apimatches before/ - This is why backend-route works correctly
Verify:
kubectl get httproute
kubectl describe httproute backend-route
# Should show ACCEPTED=True and ResolvedRefs=True
π§ͺ Testing and Verification
Wait for NLB to be Ready
# Check NLB status in AWS
aws elbv2 describe-load-balancers --region us-east-1 \
--query 'LoadBalancers[?contains(LoadBalancerName, `envoy`)].{Name:LoadBalancerName,State:State.Code,Scheme:Scheme}' \
--output table
# Check target health
TG_ARN=$(aws elbv2 describe-target-groups --region us-east-1 \
--query 'TargetGroups[?contains(LoadBalancerArns[0], `envoy`)].TargetGroupArn' \
--output text | head -1)
aws elbv2 describe-target-health --region us-east-1 \
--target-group-arn "$TG_ARN" \
--query 'TargetHealthDescriptions[*].{Target:Target.Id,State:TargetHealth.State}' \
--output table
Wait until:
- NLB State =
active - At least one target State =
healthy
Test the Endpoints
# Get NLB DNS
NLB_DNS=$(kubectl get gateway demo-gateway -o jsonpath='{.status.addresses[0].value}')
# Test backend
curl http://$NLB_DNS/api
# Expected: Backend API Response
# Test frontend
curl http://$NLB_DNS/
# Expected: Frontend UI Response
Verify Gateway API Resources
# Check all Gateway API resources
kubectl get gatewayclass,gateway,httproute
# Check Gateway details
kubectl describe gateway demo-gateway
# Check HTTPRoute details
kubectl describe httproute backend-route
# Check Envoy Gateway logs
kubectl logs -n envoy-gateway-system deployment/envoy-gateway --tail=50
# Check Envoy Proxy logs
ENVOY_POD=$(kubectl get pods -n envoy-gateway-system -l app.kubernetes.io/component=proxy -o jsonpath='{.items[0].metadata.name}')
kubectl logs -n envoy-gateway-system $ENVOY_POD -c envoy --tail=50
π How It Works: Deep Dive
Request Flow
Let's trace a request from client to pod:
1. Client sends: GET http://nlb-dns.amazonaws.com/api
2. DNS resolves to NLB IP addresses
3. NLB forwards to Envoy proxy pod (NodePort 31735)
4. Envoy proxy receives request on port 10080
5. Envoy matches request against HTTPRoutes:
- Checks backend-route: path=/api β MATCH
- Routes to backend-service:80
6. backend-service (ClusterIP) load balances to backend pods
7. Backend pod responds: "Backend API Response"
8. Response flows back through Envoy β NLB β Client
Gateway Controller Reconciliation Loop
Envoy Gateway continuously watches for changes:
1. Watch Gateway API resources (Gateway, HTTPRoute, etc.)
2. On change detected:
a. Validate resource
b. Translate to Envoy configuration (xDS)
c. Update Envoy proxy pods
d. Update Gateway/HTTPRoute status
3. Watch Envoy proxy pods:
a. Ensure desired replicas running
b. Monitor health
c. Update Gateway status
4. Watch LoadBalancer Service:
a. Get external IP/DNS
b. Update Gateway status with address
Envoy Proxy Configuration
Envoy Gateway translates Gateway API resources to Envoy xDS configuration:
Gateway β Listener
Gateway:
listeners:
- name: http
port: 80
protocol: HTTP
Translates to Envoy Listener:
name: default/demo-gateway/http
address: 0.0.0.0:10080
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
HTTPRoute β Route Configuration
HTTPRoute:
matches:
- path:
type: PathPrefix
value: /api
backendRefs:
- name: backend-service
port: 80
Translates to Envoy Route:
match:
prefix: /api
route:
cluster: httproute/default/backend-route/rule/0
Service β Cluster & Endpoints
Service: backend-service
selector: app=backend
port: 80
Translates to Envoy Cluster:
name: httproute/default/backend-route/rule/0
type: EDS
endpoints:
- 192.168.15.16:5678 (backend pod 1)
- 192.168.38.106:5678 (backend pod 2)
Load Balancing
NLB Level:
- Distributes traffic across Envoy proxy pods
- Uses NodePort (31735 in our case)
- Health checks on
/healthzendpoint
Envoy Level:
- Distributes traffic across backend pods
- Default: Round-robin
- Supports weighted, least-request, ring-hash, etc.
Service Level:
- Kubernetes Service provides ClusterIP
- kube-proxy manages iptables/IPVS rules
- Distributes to healthy endpoints only
π Troubleshooting
Issue 1: Gateway Not Getting Address
Symptoms:
kubectl get gateway demo-gateway
# ADDRESS column is empty, PROGRAMMED=False
Causes:
- Envoy Gateway controller not running
- LoadBalancer service not created
- AWS quota limits reached
Solutions:
# Check controller
kubectl get pods -n envoy-gateway-system
# Check controller logs
kubectl logs -n envoy-gateway-system deployment/envoy-gateway --tail=50
# Check for LoadBalancer service
kubectl get svc -n envoy-gateway-system
# Restart controller if needed
kubectl rollout restart deployment/envoy-gateway -n envoy-gateway-system
Issue 2: HTTPRoute Not Accepted
Symptoms:
kubectl describe httproute backend-route
# Conditions show Accepted=False
Causes:
- Gateway doesn't exist
- parentRefs incorrect
- Backend service doesn't exist
Solutions:
# Verify Gateway exists
kubectl get gateway demo-gateway
# Check parentRefs match Gateway name
kubectl get httproute backend-route -o yaml | grep -A 5 parentRefs
# Verify backend service exists
kubectl get svc backend-service
Issue 3: 404 Not Found
Symptoms:
curl http://$NLB_DNS/api
# Returns 404
Causes:
- HTTPRoute path doesn't match
- HTTPRoute not attached to Gateway
- Backend service selector wrong
Solutions:
# Check HTTPRoute status
kubectl describe httproute backend-route
# Test backend service directly
kubectl run test-curl --image=curlimages/curl --rm -it --restart=Never -- \
curl backend-service/api
# Check service endpoints
kubectl get endpoints backend-service
Issue 4: Connection Timeout
Symptoms:
curl http://$NLB_DNS/api
# Hangs and times out
Causes:
- NLB targets unhealthy
- Security group blocking traffic
- Network ACLs blocking traffic
Solutions:
# Check target health
TG_ARN=$(aws elbv2 describe-target-groups --region us-east-1 \
--query 'TargetGroups[?contains(LoadBalancerArns[0], `envoy`)].TargetGroupArn' \
--output text | head -1)
aws elbv2 describe-target-health --region us-east-1 --target-group-arn "$TG_ARN"
# Check security groups
aws ec2 describe-security-groups --region us-east-1 \
--filters "Name=tag:kubernetes.io/cluster/eks-gateway-demo,Values=owned"
# Check if NLB is internet-facing
aws elbv2 describe-load-balancers --region us-east-1 \
--query 'LoadBalancers[?contains(LoadBalancerName, `envoy`)].Scheme'
π§Ή Cleanup
To avoid AWS charges, clean up all resources:
# Delete HTTPRoutes
kubectl delete httproute backend-route frontend-route
# Delete Gateway (this deletes the NLB)
kubectl delete gateway demo-gateway
# Delete GatewayClass
kubectl delete gatewayclass eg
# Delete EnvoyProxy configuration
kubectl delete envoyproxy internet-facing-proxy -n envoy-gateway-system
# Uninstall Envoy Gateway
kubectl delete -f https://github.com/envoyproxy/gateway/releases/download/v1.0.0/install.yaml
# Delete Gateway API CRDs
kubectl delete -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/standard-install.yaml
# Delete applications
kubectl delete deployment backend frontend
kubectl delete service backend-service frontend-service
# Delete EKS cluster (this deletes everything)
eksctl delete cluster --name eks-gateway-demo --region us-east-1
Verify cleanup:
# Check no NLBs remain
aws elbv2 describe-load-balancers --region us-east-1 \
--query 'LoadBalancers[?contains(LoadBalancerName, `envoy`)]'
# Check cluster deleted
aws eks list-clusters --region us-east-1
π Comparison: Ingress vs Gateway API
| Feature | Ingress | Gateway API |
|---|---|---|
| Role Separation | β Single resource | β GatewayClass, Gateway, HTTPRoute |
| Portability | β Vendor annotations | β Standard API |
| Expressiveness | β Basic path/host | β Headers, query params, weights |
| Protocol Support | β HTTP/HTTPS only | β HTTP, HTTPS, TCP, UDP, gRPC |
| Traffic Splitting | β Limited | β Native support |
| Request Modification | β Controller-specific | β Standard filters |
| Multi-tenancy | β Difficult | β Built-in with ReferenceGrant |
| Status | β οΈ Deprecated (March 2026) | β GA (v1.0.0) |
π Production Readiness Checklist
Before going to production:
-
[ ] High Availability
- [ ] Multiple Gateway replicas
- [ ] Multi-AZ deployment
- [ ] Pod disruption budgets
-
[ ] Security
- [ ] TLS enabled on all listeners
- [ ] RBAC configured
- [ ] Network policies in place
- [ ] Secrets encrypted at rest
-
[ ] Monitoring
- [ ] Prometheus metrics scraped
- [ ] Alerts configured
- [ ] Dashboards created
- [ ] Logging aggregated
-
[ ] Performance
- [ ] Load testing completed
- [ ] Resource limits set
- [ ] Autoscaling configured
- [ ] Connection limits tuned
-
[ ] Disaster Recovery
- [ ] Backup strategy defined
- [ ] Restore procedure tested
- [ ] Runbooks documented
- [ ] On-call rotation established
πΈ Screenshots
π― Conclusion
Gateway API represents a significant evolution in Kubernetes networking:
β
Better separation of concerns between infrastructure, cluster admins, and developers
β
Portable and consistent across all cloud providers and on-premises
β
More expressive with advanced routing capabilities
β
Future-proof as the official successor to Ingress
With Ingress NGINX being decommissioned in March 2026, now is the perfect time to migrate to Gateway API. Envoy Gateway provides a production-ready implementation that's easy to deploy and manage.
Key Takeaways
- Gateway API is not just a replacement - it's a complete redesign with better architecture
- Role separation makes it easier to manage in large organizations
- Envoy Gateway is a solid choice for AWS EKS deployments
- Migration from Ingress is straightforward with proper planning
- Advanced features like traffic splitting and header routing are built-in
Next Steps
- Explore other Gateway controllers (Istio, Kong, Traefik)
- Implement advanced routing scenarios
- Add observability with Prometheus and Grafana
- Integrate with service mesh for mTLS
- Automate deployments with GitOps (ArgoCD, Flux)
Resources
- Gateway API Documentation
- Envoy Gateway Documentation
- AWS Load Balancer Controller
- Kubernetes SIG Network
Happy Learning
Prithiviraj Rengarajan
DevOps Engineer





Top comments (0)