Amazon EKS

Overview

Amazon Elastic Kubernetes Service (EKS) is AWS's managed Kubernetes offering. It handles the control plane (API server, etcd, scheduler) while you manage worker nodes, networking, storage, and application deployments. This reference covers the AWS-specific integrations: CSI drivers, VPC CNI, load balancer ingress, and IAM roles.

EBS CSI driver

The AWS EBS CSI driver provisions persistent block storage for pods using EBS volumes.

Installation (via EKS add-on or Helm)

The EBS CSI driver is the standard way to provide persistent block storage to pods on EKS. Installing it through the managed add-on is recommended over Helm because AWS keeps the driver updated and wires up the required IAM role for you.

# Via EKS add-on (recommended)
aws eks create-addon --cluster-name my-cluster \
  --addon-name aws-ebs-csi-driver \
  --service-account-role-arn arn:aws:iam::123456789:role/AmazonEKS_EBS_CSI_DriverRole

# Check status
aws eks describe-addon --cluster-name my-cluster --addon-name aws-ebs-csi-driver

IAM role for the CSI driver

The driver needs permission to call EBS APIs (CreateVolume, AttachVolume, etc.). This trust policy lets the driver's ServiceAccount assume an IAM role through IRSA, scoped to the cluster's OIDC provider so only that ServiceAccount can use it.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Federated": "arn:aws:iam::123456789:oidc-provider/oidc.eks.us-east-2.amazonaws.com/id/..." },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc.eks.us-east-2.amazonaws.com/id/...:sub": "system:serviceaccount:kube-system:ebs-csi-controller-sa"
        }
      }
    }
  ]
}

StorageClass and PVC

With the driver installed, define a StorageClass that maps to an EBS volume type and a PVC that requests storage. WaitForFirstConsumer delays volume creation until a pod actually uses the PVC, which avoids provisioning volumes that never get attached.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ebs-gp3
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
  encrypted: "true"
  iops: "3000"
  throughput: "125"
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: app-data
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: ebs-gp3
  resources:
    requests:
      storage: 50Gi

Using in a pod

Finally, mount the PVC into a pod by referencing it in a volume. The CSI driver attaches the EBS volume to the node and mounts it at the container's mountPath before the container starts.

apiVersion: v1
kind: Pod
metadata:
  name: app
spec:
  containers:
    - name: app
      image: myapp:latest
      volumeMounts:
        - name: data
          mountPath: /data
  volumes:
    - name: data
      persistentVolumeClaim:
        claimName: app-data

EFS CSI driver

For shared (ReadWriteMany) filesystem access across pods:

The EFS CSI driver lets many pods mount the same filesystem simultaneously — something EBS block volumes can't do. Install it via the EKS add-on, then define a StorageClass that points at it:

# Install via add-on
aws eks create-addon --cluster-name my-cluster \
  --addon-name aws-efs-csi-driver \
  --service-account-role-arn arn:aws:iam::123456789:role/AmazonEKS_EFS_CSI_DriverRole

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: efs-sc
provisioner: efs.csi.aws.com
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: shared-data
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: efs-sc
  resources:
    requests:
      storage: 5Gi

AWS VPC CNI

The VPC CNI plugin assigns native VPC IP addresses to pods, enabling direct communication with other AWS services.

Key features

• Each pod gets an IP from the VPC CIDR range (not an overlay)
• Pods can directly access RDS, ElastiCache, and other VPC services without NAT
• Secondary IP addresses are attached to ENIs on worker nodes

Common configurations

The VPC CNI is tuned through environment variables on the aws-node DaemonSet. These settings trade off IP consumption, pod density per node, and pod startup latency — adjust them to match your subnet sizes and workload patterns.

# Enable prefix delegation (more pods per node)
kubectl set env daemonset aws-node -n kube-system ENABLE_PREFIX_DELEGATION=true

# Configure minimum and maximum IPs per ENI
kubectl set env daemonset aws-node -n kube-system WARM_IP_TARGET=2 MINIMUM_IP_TARGET=10

# Custom networking (pod traffic uses different subnets than node traffic)
kubectl set env daemonset aws-node -n kube-system AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG=true
kubectl set env daemonset aws-node -n kube-system ENI_CONFIG_LABEL_DEF=topology.kubernetes.io/zone

IP exhaustion troubleshooting

Because every pod consumes a VPC IP address, large or dense clusters can exhaust a subnet's address space and leave pods stuck in ContainerCreating. These commands show how many addresses remain per node and how to reclaim or conserve them.

# Check available IPs per node
kubectl describe nodes | grep -A5 "Allocatable"

# View CNI metrics
kubectl logs -n kube-system daemonset/aws-node | grep "ip address"

# Enable SNAT for pod traffic leaving the VPC (use this to conserve
# pod IPs when running with custom networking / prefix delegation)
kubectl set env daemonset aws-node -n kube-system AWS_VPC_K8S_CNI_EXTERNALSNAT=true

AWS Load Balancer Controller

Replaces the in-tree cloud provider for provisioning ALBs and NLBs from Kubernetes Ingress and Service resources.

Installation

The AWS Load Balancer Controller replaces the legacy in-tree cloud provider and provisions ALBs (from Ingress resources) and NLBs (from LoadBalancer Services). It needs its own IAM role, which eksctl creates alongside a dedicated ServiceAccount:

# Create IAM policy and role (using eksctl or manual)
eksctl create iamserviceaccount \
  --cluster my-cluster \
  --namespace kube-system \
  --name aws-load-balancer-controller \
  --attach-policy-arn arn:aws:iam::123456789:policy/AWSLoadBalancerControllerIAMPolicy \
  --approve

# Install via Helm
helm repo add eks https://aws.github.io/eks-charts
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=my-cluster \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller

Ingress with ALB

An Ingress with ingressClassName: alb tells the controller to provision an Application Load Balancer in front of your service. Annotations control the ALB's scheme (public or internal), TLS certificates, listener ports, and health-check path.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-2:123456789:certificate/abc-123
    alb.ingress.kubernetes.io/ssl-redirect: "443"
    alb.ingress.kubernetes.io/healthcheck-path: /health
spec:
  ingressClassName: alb
  rules:
    - host: app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-app
                port:
                  number: 80

Service with NLB

A Service of type LoadBalancer carrying the NLB annotations makes the controller provision a Network Load Balancer instead — the right choice for TCP/UDP traffic and for workloads that must preserve client IP addresses.

apiVersion: v1
kind: Service
metadata:
  name: my-tcp-app
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
    service.beta.kubernetes.io/aws-load-balancer-scheme: "internal"
spec:
  type: LoadBalancer
  selector:
    app: my-tcp-app
  ports:
    - port: 5432
      targetPort: 5432
      protocol: TCP

IAM Roles for Service Accounts (IRSA)

Assign IAM roles directly to pods without sharing node-level instance profiles:

# Create the service account with an IAM role
eksctl create iamserviceaccount \
  --name s3-reader \
  --namespace default \
  --cluster my-cluster \
  --attach-policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess \
  --approve

Once the ServiceAccount exists, assign it to a pod. The pod's AWS SDK automatically assumes the IAM role scoped to that ServiceAccount:

apiVersion: v1
kind: Pod
metadata:
  name: s3-reader-app
spec:
  serviceAccountName: s3-reader
  containers:
    - name: app
      image: amazon/aws-cli
      command: ["aws", "s3", "ls"]

See also

• Terraform EKS + Kubernetes — provisioning EKS with Terraform
• Helm Reference — package management for Kubernetes
• Node Setup — worker and control plane node configuration