Kubernetes and Docker

Docker and Kubernetes are two of the most consequential infrastructure technologies of the last decade. They changed how software is built, packaged, and deployed. They are also two technologies that most engineers use before they understand, which creates gaps in knowledge that show up at the worst times: a production outage, a security incident, a performance problem you cannot diagnose.

This guide builds understanding from the ground up. Every concept is introduced with the problem it solves. You will understand why containers exist before you understand what they are. You will understand why Kubernetes exists before you understand how it works. By the end, you will know not just how to run these technologies but how to reason about them.

Table of Contents

1. The Problem Containers Solve - Why Docker Exists
2. Docker Internals - What a Container Actually Is
3. Images - Building Portable Application Packages
4. Dockerfile - Writing Reproducible Builds
5. Docker Networking - Container Communication
6. Docker Volumes - Managing State
7. Docker Compose - Multi-Container Applications
8. The Problem Kubernetes Solves - Why Orchestration Exists
9. Kubernetes Architecture - The Control Plane and Data Plane
10. Core Kubernetes Objects - Pods, Deployments, Services, ConfigMaps, Secrets
11. Namespaces and RBAC - Multi-Tenancy and Access Control
12. Storage in Kubernetes - Persistent Volumes
13. Ingress - Routing External Traffic
14. Helm - Package Management for Kubernetes
15. Service Mesh - Istio and Advanced Traffic Management
16. AWS Container Services - ECS and EKS
17. Real Architecture Patterns

The Problem Containers Solve - Why Docker Exists

The Classic Failure Mode

A developer builds an application on their MacBook. It works. They hand it to the QA team. It does not work. They hand it to the operations team to deploy to production. It works differently than in QA.

"It works on my machine" is not a joke. It is a description of a real, chronic infrastructure problem. The application depends on:

• A specific version of Python, Node, Java, or Ruby
• Specific versions of libraries and dependencies
• Specific environment variables
• Specific file paths and configurations
• A specific operating system and version

Every environment - developer laptop, CI server, staging, production, has slightly different versions of these things. The application behaves differently in each one.

The traditional solution was configuration management, Ansible, Chef, Puppet, which describes how to configure a server. But configuration management operates at the server level. It is imperative. It has drift problems. It is slow. And it does not guarantee that the application has identical dependencies everywhere it runs.

The Container Solution

A container packages the application together with everything it needs to run: the runtime, the libraries, the dependencies, the configuration. The package is self-contained. It runs identically everywhere, developer laptop, CI server, staging, production, because the environment is baked into the package.

The developer's MacBook, the QA server, and the production cluster all run the exact same container image. The "it works on my machine" problem disappears because the machine is now inside the package, not beneath it.

Docker Internals - What a Container Actually Is

The Common Misconception

Many engineers believe containers are lightweight virtual machines. They are not. Understanding what they actually are explains their performance characteristics, their security model, and why they behave the way they do.

What a Container Actually Is

A container is a process (or group of processes) running on a Linux kernel, isolated from other processes using two kernel features: namespaces and cgroups.

Namespaces provide isolation. They make a process believe it is the only process on the system, even though it is sharing the kernel with many other processes. There are multiple namespace types:

cgroups (control groups) provide resource limits and accounting. They answer the question: how much of the host's resources can this container use? CPU, memory, disk I/O, network bandwidth, all controlled by cgroups. Without cgroups, one misbehaving container could consume all of the host's CPU and starve every other container.

The Container vs Virtual Machine Comparison

Containers start in milliseconds because there is no operating system to boot, the host kernel is already running. The container runtime (Docker Engine, containerd) sets up the namespaces and cgroups, then executes the process. That is all that happens.

The security boundary is weaker than a VM. A critical vulnerability in the Linux kernel could allow a container escape, a process inside a container gaining access to the host. This is rare but real. VMs provide stronger isolation because a hypervisor escape requires compromising a much smaller attack surface.

The Container Runtime Stack

When engineers say "Docker," they often mean the full stack:

Your Application
      ↓
Docker CLI / Docker Desktop
      ↓
Docker Daemon (dockerd)
      ↓
containerd (container runtime)
      ↓
runc (OCI runtime - sets up namespaces, cgroups, executes process)
      ↓
Linux Kernel (namespaces, cgroups, seccomp, AppArmor)

Docker is a user-friendly interface on top of a stack of open standards. Kubernetes does not use Docker directly, it uses containerd (or another OCI-compatible runtime) directly. Docker the CLI is for developers. containerd is for production orchestration systems.

Images - Building Portable Application Packages

What an Image Is

A container image is a read-only, layered filesystem that contains everything needed to run the application. It is the package. The container is the running instance of the image, the same relationship a class has to an object in object-oriented programming.

The Layer Model

Images are built in layers. Each layer represents a set of filesystem changes. Layers are immutable and cached. This is what makes Docker efficient.

Layer 4: COPY app.py /app/app.py              [your application code]
Layer 3: RUN pip install -r requirements.txt  [installed dependencies]
Layer 2: COPY requirements.txt /app/          [requirements file]
Layer 1: FROM python:3.11-slim                [base OS + Python runtime]

Each layer is a diff from the layer below it. Layer 1 is the base image, an official Python image that contains a minimal Linux filesystem with Python installed. Layer 2 adds your requirements file. Layer 3 adds the installed Python packages. Layer 4 adds your application code.

Why layers matter for performance:

When you rebuild the image after changing only your application code, Docker uses cached versions of layers 1, 2, and 3. Only layer 4 is rebuilt. A rebuild that would take minutes if done from scratch takes seconds because the cached layers are reused.

When you deploy the image to 50 servers, and all 50 already have layer 1, 2, and 3 from a previous deployment, only layer 4 is transferred. The network cost is proportional to what changed, not the full image size.

Image Registries

Images are stored in registries. A registry is a server that stores and serves images.

• Docker Hub - Public registry. Official images for most popular software: nginx, postgres, python, node, redis.
• Amazon ECR (Elastic Container Registry) - AWS's managed private registry. Images are stored in your account, scanned for vulnerabilities, and replicated across regions.
• GitHub Container Registry - Store images alongside your code in GitHub.
• Self-hosted - Harbor, Nexus, or other registries in your own infrastructure.

# Pull an image from Docker Hub
docker pull nginx:1.25-alpine

# Pull from ECR
aws ecr get-login-password --region us-east-1 | \
  docker login --username AWS --password-stdin \
  123456789012.dkr.ecr.us-east-1.amazonaws.com

docker pull 123456789012.dkr.ecr.us-east-1.amazonaws.com/my-app:v1.2.3

# Tag and push your image to ECR
docker tag my-app:latest \
  123456789012.dkr.ecr.us-east-1.amazonaws.com/my-app:v1.2.3

docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/my-app:v1.2.3

Dockerfile - Writing Reproducible Builds

What a Dockerfile Is

A Dockerfile is a text file containing the instructions to build an image. It is the recipe. Every time the Dockerfile is executed, it produces an identical image, the same layers, the same content, the same behaviour.

This is what makes builds reproducible. The Dockerfile is committed to source control. Anyone with the Dockerfile and access to the base images can produce an identical image.

A Production-Grade Dockerfile

# ─── Stage 1: Build ─────────────────────────────────────────────────
# Use a full Python image for the build stage
FROM python:3.11-slim AS builder

WORKDIR /app

# Copy only requirements first — enables layer caching
# If requirements.txt hasn't changed, pip install won't re-run
COPY requirements.txt .

# Install dependencies into a virtual environment
RUN python -m venv /opt/venv && \
    /opt/venv/bin/pip install --no-cache-dir --upgrade pip && \
    /opt/venv/bin/pip install --no-cache-dir -r requirements.txt

# ─── Stage 2: Production image ──────────────────────────────────────
# Start from a minimal base — no build tools, smaller attack surface
FROM python:3.11-slim AS production

# Security: run as non-root user
RUN groupadd --gid 1001 appgroup && \
    useradd --uid 1001 --gid appgroup --no-create-home appuser

WORKDIR /app

# Copy only the virtual environment from the build stage
# Build tools, pip cache, and intermediate files are NOT included
COPY --from=builder /opt/venv /opt/venv

# Copy application code
COPY --chown=appuser:appgroup . .

# Set the virtual environment path
ENV PATH="/opt/venv/bin:$PATH"
ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1

# Switch to non-root user
USER appuser

# Expose the port the application listens on
EXPOSE 8000

# Health check - allows the runtime to know if the app is healthy
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
    CMD curl -f http://localhost:8000/health || exit 1

# Use exec form - the process is PID 1, receives signals directly
CMD ["/opt/venv/bin/gunicorn", "--bind", "0.0.0.0:8000", "--workers", "4", "app:application"]

Key Dockerfile Best Practices

Multi-stage builds -Build in one stage, run in another. The production image does not contain build tools, compilers, or intermediate build artifacts. Smaller image. Smaller attack surface.

Layer ordering - Put instructions that change frequently near the bottom. Dependencies change less often than application code. Copy requirements.txt and install dependencies before copying the application code. This maximises cache reuse.

Non-root user - By default, processes inside containers run as root. A compromised process running as root inside a container has more capability than a compromised process running as a non-root user. Always run as a non-root user in production.

Specific base image tags - FROM python:3.11-slim is better than FROM python:latest. FROM python:3.11.4-slim is better still. Pinned versions produce reproducible builds. latest changes under you.

.dockerignore - Like .gitignore but for Docker builds. Exclude files that should not be in the image: .git, node_modules, test files, documentation.

# .dockerignore
.git
.github
*.md
tests/
node_modules/
__pycache__/
*.pyc
.env
.env.*

No secrets in images - Never put API keys, passwords, or private keys in a Dockerfile or image. They are stored in image layers permanently, even if you delete them in a later layer. Use runtime environment variables or secrets management systems.

Docker Networking - Container Communication

Default Networks

Docker creates three networks by default:

bridge - The default. Containers on the bridge network can communicate with each other by IP address. They cannot resolve each other by name unless using a user-defined bridge network.

host - The container shares the host's network namespace. No isolation. The container's port 8000 is the host's port 8000. Used for performance-critical applications where the networking overhead of bridge mode is a concern.

none - No networking. The container has only a loopback interface. Used for completely isolated containers.

User-Defined Bridge Networks

Always use user-defined bridge networks in production, never the default bridge. User-defined networks provide automatic DNS resolution, containers can reach each other by name, not just IP address.

# Create a user-defined network
docker network create \
  --driver bridge \
  --subnet 172.20.0.0/16 \
  --ip-range 172.20.10.0/24 \
  my-app-network

# Run containers on the network
docker run -d \
  --name postgres-db \
  --network my-app-network \
  --env POSTGRES_PASSWORD=secret \
  postgres:15-alpine

docker run -d \
  --name web-app \
  --network my-app-network \
  --env DATABASE_URL=postgresql://postgres:secret@postgres-db:5432/mydb \
  --publish 8000:8000 \
  my-app:latest

The web-app container can reach the database at postgres-db:5432, by name, not by IP. Docker's built-in DNS resolves postgres-db to the container's IP address on the network.

Port Mapping

Containers are isolated. Their ports are not accessible from outside by default. Port mapping (--publish or -p) creates a mapping from a host port to a container port:

# Map host port 8080 to container port 8000
docker run -p 8080:8000 my-app:latest

# Map all interfaces on host port 8080
docker run -p 0.0.0.0:8080:8000 my-app:latest

# Map only localhost on host
docker run -p 127.0.0.1:8080:8000 my-app:latest

# Let Docker choose a random host port
docker run -p 8000 my-app:latest

Docker Volumes - Managing State

The Ephemeral Container Problem

Containers are ephemeral by design. When a container stops and restarts, any data written inside the container filesystem is gone. This is the right default for stateless applications, each container starts fresh.

But some applications need to persist data: databases, file uploads, log archives. For these, you need storage that exists outside the container's lifecycle.

The Three Storage Options

Volumes - Managed by Docker. Stored on the host filesystem, but managed by Docker in /var/lib/docker/volumes/. The container does not know or care where the data lives on the host. Volumes can be shared between containers and backed up independently of containers.

Bind mounts - A specific directory on the host is mounted into the container. The container and the host share access to that directory. Changes on either side are immediately visible to the other. Used during development (mount source code into the container for live reloading) but not recommended for production on multi-node environments.

tmpfs mounts - Data is stored in the host's memory, not on disk. It disappears when the container stops. Used for sensitive data that should never be written to disk.

# Create a named volume
docker volume create postgres-data

# Run postgres with persistent data
docker run -d \
  --name postgres-db \
  --volume postgres-data:/var/lib/postgresql/data \
  --env POSTGRES_PASSWORD=secret \
  postgres:15-alpine

# Bind mount for development (source code hot-reload)
docker run -d \
  --name dev-app \
  --volume $(pwd)/src:/app/src \
  --publish 8000:8000 \
  my-app:dev

# Backup a volume
docker run --rm \
  --volume postgres-data:/source:ro \
  --volume $(pwd)/backups:/backup \
  alpine \
  tar czf /backup/postgres-data-$(date +%Y%m%d).tar.gz -C /source .

Docker Compose - Multi-Container Applications

The Problem

Running a web application with a database, a cache, a background worker, and a reverse proxy means running five docker run commands with the right flags, in the right order, on the right network. Doing this manually is error-prone. Automating it with shell scripts is fragile.

What Docker Compose Is

Docker Compose is a tool for defining and running multi-container applications. You describe the entire application, all services, networks, and volumes, in a single YAML file. One command starts everything.

# docker-compose.yml

services:
  web:
    build:
      context: .
      dockerfile: Dockerfile
      target: production
    image: my-app:latest
    ports:
      - "8000:8000"
    environment:
      - DATABASE_URL=postgresql://appuser:${DB_PASSWORD}@postgres:5432/mydb
      - REDIS_URL=redis://redis:6379/0
      - SECRET_KEY=${SECRET_KEY}
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_started
    networks:
      - app-network
    restart: unless-stopped
    deploy:
      resources:
        limits:
          cpus: '1.0'
          memory: 512M

  worker:
    build:
      context: .
      dockerfile: Dockerfile
      target: production
    command: ["/opt/venv/bin/celery", "-A", "app.celery", "worker", "--loglevel=info"]
    environment:
      - DATABASE_URL=postgresql://appuser:${DB_PASSWORD}@postgres:5432/mydb
      - REDIS_URL=redis://redis:6379/0
    depends_on:
      - web
    networks:
      - app-network
    restart: unless-stopped

  postgres:
    image: postgres:15-alpine
    environment:
      - POSTGRES_USER=appuser
      - POSTGRES_PASSWORD=${DB_PASSWORD}
      - POSTGRES_DB=mydb
    volumes:
      - postgres-data:/var/lib/postgresql/data
      - ./db/init.sql:/docker-entrypoint-initdb.d/init.sql
    networks:
      - app-network
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U appuser -d mydb"]
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped

  redis:
    image: redis:7-alpine
    command: redis-server --maxmemory 256mb --maxmemory-policy allkeys-lru
    volumes:
      - redis-data:/data
    networks:
      - app-network
    restart: unless-stopped

  nginx:
    image: nginx:1.25-alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      - ./nginx/ssl:/etc/nginx/ssl:ro
    depends_on:
      - web
    networks:
      - app-network
    restart: unless-stopped

volumes:
  postgres-data:
  redis-data:

networks:
  app-network:
    driver: bridge

# Start all services
docker compose up -d

# View logs from all services
docker compose logs -f

# View logs from a specific service
docker compose logs -f web

# Scale the worker service to 3 instances
docker compose up -d --scale worker=3

# Stop and remove all containers, networks (but not volumes)
docker compose down

# Stop and remove everything including volumes
docker compose down -v

# Rebuild and restart a specific service
docker compose up -d --build web

Docker Compose is for development and single-host deployments. For production deployments across multiple hosts, you need Kubernetes.

The Problem Kubernetes Solves - Why Orchestration Exists

What Docker Compose Cannot Do

Docker Compose runs containers on a single host. A single host has limits: limited CPU, limited memory, a single point of failure. When the host goes down, everything goes down.

For production applications with availability requirements, you need multiple hosts. But managing containers across multiple hosts manually creates a new class of problems:

Scheduling - Which host should run each container? Which host has the most available resources? If a container needs a GPU, which hosts have GPUs?

Failure handling - If a host fails, who detects the failure? Who restarts the containers that were running on it? On which other host?

Scaling - If traffic increases, who decides to run more containers? Which hosts do the new containers go on?

Service discovery - How does container A find container B when B could be running on any host in the cluster and its IP address changes every time it restarts?

Rolling updates - How do you update 100 instances of a service from version 1 to version 2 without downtime?

Health checking - How do you continuously verify that every container is healthy and route traffic away from unhealthy ones?

Resource management - How do you prevent one application from consuming all the resources on every host and starving other applications?

These are the problems Kubernetes solves. It is a container orchestration system - a system that manages containers at scale across a cluster of machines.

Kubernetes Architecture - The Control Plane and Data Plane

The Two Planes

A Kubernetes cluster has two distinct layers: the control plane and the data plane (worker nodes). Understanding what lives where explains how the cluster makes decisions and how failures propagate.

The Control Plane

The control plane is the brain of the cluster. It makes decisions about the cluster state - what should be running, where it should run, what is healthy. It does not run application workloads.

┌─────────────────────────────────────────────────────────────┐
│                        CONTROL PLANE                        │
│                                                             │
│  ┌──────────────┐  ┌────────────┐  ┌──────────────────────┐ │
│  │  API Server  │  │  etcd      │  │  Controller Manager  │ │
│  │  (kube-      │  │  (cluster  │  │  (reconciliation     │ │
│  │  apiserver)  │  │  state DB) │  │  loops)              │ │
│  └──────────────┘  └────────────┘  └──────────────────────┘ │
│                                                             │
│  ┌──────────────┐                                           │
│  │  Scheduler   │                                           │
│  │  (workload   │                                           │
│  │  placement)  │                                           │
│  └──────────────┘                                           │
└─────────────────────────────────────────────────────────────┘

kube-apiserver - The front door to Kubernetes. Every interaction with the cluster, from kubectl, from controllers, from nodes, goes through the API server. It validates requests, enforces admission control, and persists state to etcd.

etcd - The cluster's database. A distributed key-value store that holds the entire state of the cluster: every object, its current state, its desired state. If etcd is lost and not backed up, the cluster's state is gone. etcd must be backed up regularly in production.

kube-scheduler - Watches for unscheduled Pods and assigns them to nodes. It evaluates each node for resource availability, taints, affinities, and other scheduling constraints, then picks the best node for each Pod.

kube-controller-manager - Runs a set of controller loops. Each controller watches the state of some resource type and works to reconcile actual state with desired state. The Deployment controller ensures the right number of Pod replicas are running. The Node controller detects when nodes go offline. The ReplicaSet controller ensures replica counts are maintained.

The Data Plane (Worker Nodes)

Worker nodes run the actual application workloads. Each node runs three components:

kubelet - The agent that runs on every node. It receives Pod specifications from the API server and ensures the specified containers are running. It reports node and Pod health back to the control plane.

kube-proxy - Maintains network rules on each node. It implements the Service abstraction, when traffic arrives for a Service's ClusterIP, kube-proxy routes it to one of the backing Pods.

Container runtime - containerd (or another OCI-compatible runtime). Actually runs the containers.

The Reconciliation Loop - How Kubernetes Thinks

Kubernetes is a declarative system. You tell it what you want (desired state), and it continuously works to make reality match your declaration (actual state). This is the reconciliation loop.

You declare: "I want 3 replicas of my-app running."

Kubernetes observes: "I see 2 replicas of my-app running."

Kubernetes acts: "I will create 1 more replica."

Kubernetes observes: "I see 3 replicas of my-app running."

Kubernetes acts: "Nothing to do. Desired state matches actual state."

[A node fails. One replica dies.]

Kubernetes observes: "I see 2 replicas of my-app running."

Kubernetes acts: "I will create 1 more replica on a healthy node."

This loop never stops. It runs continuously. Any deviation from desired state triggers a corrective action. This is why Kubernetes is self-healing — it does not just detect failures and alert. It detects failures and fixes them automatically.

Core Kubernetes Objects

Pods

A Pod is the smallest deployable unit in Kubernetes. It is one or more containers that share:

• A network namespace (the same IP address and port space)
• A storage namespace (the same volumes)
• A lifecycle (they start and stop together)

Most Pods contain one container. Multiple containers in a single Pod is the sidecar pattern, used for logging agents, service mesh proxies, or other tightly coupled helpers.

apiVersion: v1
kind: Pod
metadata:
  name: my-app-pod
  labels:
    app: my-app
    version: v1.2.3
spec:
  containers:
    - name: web
      image: 123456789012.dkr.ecr.us-east-1.amazonaws.com/my-app:v1.2.3
      ports:
        - containerPort: 8000
      env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: app-secrets
              key: database-url
      resources:
        requests:
          cpu: 250m        # 250 millicores = 0.25 CPU
          memory: 256Mi    # 256 mebibytes
        limits:
          cpu: 500m        # Hard limit: 0.5 CPU (process is throttled if exceeded)
          memory: 512Mi    # Hard limit: 512Mi (process is OOM-killed if exceeded)
      livenessProbe:
        httpGet:
          path: /health
          port: 8000
        initialDelaySeconds: 30
        periodSeconds: 10
        failureThreshold: 3
      readinessProbe:
        httpGet:
          path: /ready
          port: 8000
        initialDelaySeconds: 10
        periodSeconds: 5
        failureThreshold: 3

Resource requests vs limits:

• requests - What the scheduler uses. The Pod is only placed on a node that has at least this much available.
• limits - What the runtime enforces. The container cannot use more than this. CPU is throttled. Memory is killed (OOM).

Liveness vs readiness probes:

• livenessProbe - Is the container alive? If this fails repeatedly, Kubernetes restarts the container.
• readinessProbe - Is the container ready to receive traffic? If this fails, the Pod is removed from Service endpoints. Traffic is not sent to it, but the container is not restarted.

You should almost never create Pods directly. Use Deployments.

Deployments

A Deployment manages a set of identical Pods. It is what you actually create for stateless applications. The Deployment controller ensures the specified number of replicas is always running, handles rolling updates, and manages rollbacks.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1    # At most 1 Pod can be unavailable during update
      maxSurge: 1          # At most 1 extra Pod can exist during update
  template:
    metadata:
      labels:
        app: my-app
        version: v1.2.3
    spec:
      containers:
        - name: web
          image: 123456789012.dkr.ecr.us-east-1.amazonaws.com/my-app:v1.2.3
          ports:
            - containerPort: 8000
          resources:
            requests:
              cpu: 250m
              memory: 256Mi
            limits:
              cpu: 500m
              memory: 512Mi
          livenessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 30
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /ready
              port: 8000
            initialDelaySeconds: 10
            periodSeconds: 5
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                labelSelector:
                  matchLabels:
                    app: my-app
                topologyKey: kubernetes.io/hostname
                # Prefer to spread Pods across different nodes

# Deploy
kubectl apply -f deployment.yaml

# Check status
kubectl rollout status deployment/my-app -n production

# Update image (triggers rolling update)
kubectl set image deployment/my-app \
  web=123456789012.dkr.ecr.us-east-1.amazonaws.com/my-app:v1.2.4 \
  -n production

# Rollback if the new version has problems
kubectl rollout undo deployment/my-app -n production

# Scale
kubectl scale deployment/my-app --replicas=5 -n production

Services

Pods are ephemeral. They restart, their IP addresses change. Other Pods that need to communicate with your application cannot rely on a fixed IP address.

A Service provides a stable network endpoint for a set of Pods. It has a fixed IP address (ClusterIP) and DNS name. Traffic sent to the Service is distributed across the healthy Pods that match the Service's selector.

apiVersion: v1
kind: Service
metadata:
  name: my-app-service
  namespace: production
spec:
  selector:
    app: my-app    # Routes traffic to Pods with this label
  ports:
    - name: http
      protocol: TCP
      port: 80         # Port clients connect to
      targetPort: 8000 # Port on the container
  type: ClusterIP    # Only accessible inside the cluster

Service types:

ConfigMaps and Secrets

ConfigMaps store non-sensitive configuration data, environment variables, configuration files, command-line arguments.

Secrets store sensitive data, passwords, API keys, TLS certificates. They are base64-encoded (not encrypted) by default in etcd. For real security, enable etcd encryption at rest, or use an external secrets manager (AWS Secrets Manager, HashiCorp Vault) with a Kubernetes secrets operator.

# ConfigMap
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: production
data:
  APP_ENV: "production"
  LOG_LEVEL: "info"
  MAX_WORKERS: "4"
  nginx.conf: |
    server {
        listen 80;
        location / {
            proxy_pass http://my-app-service:80;
        }
    }

---
# Secret
apiVersion: v1
kind: Secret
metadata:
  name: app-secrets
  namespace: production
type: Opaque
stringData:  # stringData handles base64 encoding automatically
  database-url: "postgresql://appuser:actualpassword@postgres:5432/mydb"
  api-key: "sk-actual-api-key-value"

# Using ConfigMap and Secret in a Deployment
spec:
  containers:
    - name: web
      envFrom:
        - configMapRef:
            name: app-config      # All keys become env vars
      env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: app-secrets
              key: database-url   # Specific key from secret
      volumeMounts:
        - name: nginx-config
          mountPath: /etc/nginx/nginx.conf
          subPath: nginx.conf     # Mount specific key as a file
  volumes:
    - name: nginx-config
      configMap:
        name: app-config

Namespaces and RBAC - Multi-Tenancy and Access Control

Namespaces

Namespaces provide logical isolation within a cluster. Think of them as virtual clusters. Resources in one namespace do not interfere with resources in another namespace (with some exceptions, Nodes and PersistentVolumes are cluster-scoped, not namespace-scoped).

# Common namespace structure
kubectl create namespace production
kubectl create namespace staging
kubectl create namespace development
kubectl create namespace monitoring

# Apply resources to a namespace
kubectl apply -f deployment.yaml -n production

# View resources in a namespace
kubectl get pods -n production
kubectl get all -n production

# View across all namespaces
kubectl get pods --all-namespaces
kubectl get pods -A

ResourceQuotas limit what a namespace can consume:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: production-quota
  namespace: production
spec:
  hard:
    requests.cpu: "10"          # Total CPU requests in namespace
    requests.memory: 20Gi       # Total memory requests
    limits.cpu: "20"            # Total CPU limits
    limits.memory: 40Gi         # Total memory limits
    pods: "50"                  # Maximum pods
    services: "20"              # Maximum services
    persistentvolumeclaims: "10"

RBAC - Role-Based Access Control

RBAC controls who can do what in the cluster. It has four objects:

Role - A set of permissions within a namespace.
ClusterRole - A set of permissions across the entire cluster.
RoleBinding - Assigns a Role to a user, group, or service account within a namespace.
ClusterRoleBinding - Assigns a ClusterRole cluster-wide.

# Role: read-only access to pods in the production namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: pod-reader
  namespace: production
rules:
  - apiGroups: [""]
    resources: ["pods", "pods/log"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["get", "list", "watch"]

---
# RoleBinding: assign pod-reader to the dev-team group
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: dev-team-pod-reader
  namespace: production
subjects:
  - kind: Group
    name: dev-team
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

Service Accounts - The identity for Pods within the cluster. When a Pod needs to call the Kubernetes API or an AWS service, it uses a service account.

# Service account for a pod that reads from S3
apiVersion: v1
kind: ServiceAccount
metadata:
  name: s3-reader
  namespace: production
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/S3ReaderRole
    # EKS IRSA: pods using this service account assume this IAM role

Storage in Kubernetes - Persistent Volumes

The Challenge

Pods are ephemeral. Their local filesystem disappears when they are deleted. For stateful applications, databases, file stores, message queues, you need storage that outlives individual Pods.

The Storage Abstractions

PersistentVolume (PV) - A piece of storage provisioned in the cluster. It is a cluster resource, like a node. It has a specific size, access mode, and reclaim policy. PVs can be provisioned manually by an administrator or automatically by a StorageClass.

PersistentVolumeClaim (PVC) - A request for storage by a Pod. A Pod claims a PVC, and Kubernetes binds the PVC to a suitable PV. The Pod does not need to know the details of the underlying storage.

StorageClass - Defines a class of storage. When a PVC requests a StorageClass, the cluster's storage provisioner dynamically creates a PV. On AWS with EKS, the EBS StorageClass provisions an EBS volume automatically.

# StorageClass for EBS gp3 (EKS)
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ebs-gp3
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
  encrypted: "true"
  kmsKeyId: "arn:aws:kms:us-east-1:123456789012:key/abc123"
volumeBindingMode: WaitForFirstConsumer  # Provision in the same AZ as the Pod
reclaimPolicy: Retain

---
# PersistentVolumeClaim
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-data
  namespace: production
spec:
  storageClassName: ebs-gp3
  accessModes:
    - ReadWriteOnce    # Only one node can mount this volume at a time (EBS limitation)
  resources:
    requests:
      storage: 100Gi

---
# StatefulSet for PostgreSQL (using the PVC)
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
  namespace: production
spec:
  serviceName: postgres
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - name: postgres
          image: postgres:15-alpine
          env:
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgres-secret
                  key: password
          volumeMounts:
            - name: postgres-data
              mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
    - metadata:
        name: postgres-data
      spec:
        storageClassName: ebs-gp3
        accessModes: [ReadWriteOnce]
        resources:
          requests:
            storage: 100Gi

StatefulSet vs Deployment for stateful apps:

StatefulSets give each Pod a stable, predictable identity: postgres-0, postgres-1, postgres-2. They are created and deleted in order. Each Pod gets its own PVC. This is required for databases where each replica has its own data and identity matters.

Deployments give Pods random names and no stable identity. Use Deployments for stateless applications, StatefulSets for databases and other stateful systems.

Ingress - Routing External Traffic

The Problem

You have 10 services in your cluster. Each one should be accessible at a different path or subdomain: api.myapp.com, app.myapp.com, admin.myapp.com. Creating a LoadBalancer Service for each one means 10 cloud load balancers. That is expensive and operationally complex.

What Ingress Is

An Ingress is a Kubernetes resource that defines routing rules for HTTP and HTTPS traffic. An Ingress controller (a Pod running in the cluster) watches for Ingress resources and configures the underlying load balancer accordingly. One load balancer can serve all your services.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app-ingress
  namespace: production
  annotations:
    kubernetes.io/ingress.class: "nginx"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    nginx.ingress.kubernetes.io/rate-limit: "100"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  tls:
    - hosts:
        - api.myapp.com
        - app.myapp.com
      secretName: myapp-tls
  rules:
    - host: api.myapp.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: api-service
                port:
                  number: 80

    - host: app.myapp.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: frontend-service
                port:
                  number: 80
          - path: /api
            pathType: Prefix
            backend:
              service:
                name: api-service
                port:
                  number: 80

On AWS with EKS, the AWS Load Balancer Controller provisions an Application Load Balancer (ALB) for each Ingress resource, or uses a shared ALB for multiple Ingress resources in the same namespace.

Helm - Package Management for Kubernetes

The Problem

Deploying a production-ready application to Kubernetes involves many YAML files: Deployment, Service, Ingress, ConfigMap, Secrets, HPA, NetworkPolicy, ServiceAccount, RBAC. Maintaining these files separately is manageable for one environment.

For multiple environments, development, staging, production, you need the same files but with different values: different image tags, different replica counts, different resource limits, different domain names. Duplicating the YAML files for each environment creates maintenance nightmares. One file is updated; the others are forgotten.

Helm is Kubernetes's package manager. It solves this.

What Helm Is

A Helm chart is a collection of templates with a values file. The templates are YAML files with variables. The values file provides the values for those variables. Different values files produce different configurations from the same templates.

my-app/
├── Chart.yaml          # Chart metadata
├── values.yaml         # Default values
├── values-staging.yaml # Staging overrides
├── values-prod.yaml    # Production overrides
└── templates/
    ├── deployment.yaml
    ├── service.yaml
    ├── ingress.yaml
    ├── configmap.yaml
    ├── secret.yaml
    └── hpa.yaml

# values.yaml (defaults)
replicaCount: 1
image:
  repository: 123456789012.dkr.ecr.us-east-1.amazonaws.com/my-app
  tag: latest
  pullPolicy: IfNotPresent

service:
  type: ClusterIP
  port: 80

ingress:
  enabled: false

resources:
  requests:
    cpu: 100m
    memory: 128Mi
  limits:
    cpu: 250m
    memory: 256Mi

autoscaling:
  enabled: false

---
# values-prod.yaml (production overrides)
replicaCount: 3
image:
  tag: v1.2.3

ingress:
  enabled: true
  host: app.myapp.com
  tls: true

resources:
  requests:
    cpu: 250m
    memory: 256Mi
  limits:
    cpu: 500m
    memory: 512Mi

autoscaling:
  enabled: true
  minReplicas: 3
  maxReplicas: 10
  targetCPUUtilizationPercentage: 70

# templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "my-app.fullname" . }}
  namespace: {{ .Release.Namespace }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      app: {{ include "my-app.name" . }}
  template:
    metadata:
      labels:
        app: {{ include "my-app.name" . }}
        version: {{ .Values.image.tag }}
    spec:
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          resources:
            {{- toYaml .Values.resources | nindent 12 }}

# Install (deploy) a chart to production
helm install my-app ./my-app \
  --namespace production \
  --create-namespace \
  --values values-prod.yaml \
  --set image.tag=v1.2.3

# Upgrade (update) an existing release
helm upgrade my-app ./my-app \
  --namespace production \
  --values values-prod.yaml \
  --set image.tag=v1.2.4

# Rollback to previous version
helm rollback my-app 1 --namespace production

# List releases
helm list --all-namespaces

# View release history
helm history my-app --namespace production

# Install a public chart (e.g. cert-manager from Artifact Hub)
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --set installCRDs=true

Service Mesh - Istio and Advanced Traffic Management

The Problem Kubernetes Services Do Not Solve

Kubernetes Services provide load balancing and service discovery. But they do not provide:

• Mutual TLS (mTLS) between services - encryption and authentication for service-to-service traffic
• Traffic policies - retry on failure, circuit breaking, timeout enforcement
• Canary deployments - route 10% of traffic to a new version, 90% to the old version
• Observability - distributed tracing, per-service traffic metrics, error rates
• Traffic mirroring - copy traffic to a shadow service for testing without affecting users

For a production microservices architecture, all of these matter. Implementing them in application code, every service handles its own retries, its own circuit breaker, its own mTLS, is expensive and inconsistent.

What a Service Mesh Is

A service mesh is an infrastructure layer that handles service-to-service communication. It is typically implemented as a sidecar proxy, a separate container injected into every Pod automatically. The application container connects to a local proxy. The proxy handles all network communication: load balancing, mTLS, retries, timeouts, circuit breaking, metrics collection.

The application knows nothing about this. It connects to localhost:port and the proxy handles everything else.

Istio is the most widely used service mesh. On AWS, AWS App Mesh is the managed alternative.

Istio Architecture

┌─────────────────────────────────────────────┐
│               CONTROL PLANE                 │
│                                             │
│  ┌──────────────────────────────────────┐   │
│  │           Istiod                     │   │
│  │  (Pilot: config distribution)        │   │
│  │  (Citadel: certificate authority)    │   │
│  │  (Galley: config validation)         │   │
│  └──────────────────────────────────────┘   │
└─────────────────────────────────────────────┘
                      ↕ config + certs
┌────────────────────────────────────────────────────────────────┐
│                        DATA PLANE                              │
│                                                                │
│  ┌─────────────────┐           ┌─────────────────┐             │
│  │  Pod A          │           │  Pod B          │             │
│  │ ┌───────────┐   │ - mTLS -> | ┌───────────┐   │             │
│  │ │ App       │   │           │ │ App       │   │             │
│  │ ├───────────┤   │           │ ├───────────┤   |             │
│  │ │ Envoy     │   │           │ │ Envoy     │   │             │
│  │ │ (sidecar) │   │           │ │ (sidecar) │   │             │
│  │ └───────────┘   │           │ └───────────┘   │             │
│  └─────────────────┘           └─────────────────┘             │
└────────────────────────────────────────────────────────────────┘

Traffic Management with Istio

# VirtualService: define routing rules
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: my-app
  namespace: production
spec:
  hosts:
    - my-app-service
  http:
    # Canary deployment: 10% to v2, 90% to v1
    - route:
        - destination:
            host: my-app-service
            subset: v1
          weight: 90
        - destination:
            host: my-app-service
            subset: v2
          weight: 10
      retries:
        attempts: 3
        perTryTimeout: 5s
        retryOn: "5xx,reset,connect-failure"
      timeout: 10s

---
# DestinationRule: define subsets and connection policies
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: my-app
  namespace: production
spec:
  host: my-app-service
  trafficPolicy:
    connectionPool:
      http:
        http1MaxPendingRequests: 100
        http2MaxRequests: 1000
    outlierDetection:
      # Circuit breaking: eject pods that return 5xx
      consecutive5xxErrors: 5
      interval: 30s
      baseEjectionTime: 30s
      maxEjectionPercent: 50
    tls:
      mode: ISTIO_MUTUAL    # mTLS between services
  subsets:
    - name: v1
      labels:
        version: v1
    - name: v2
      labels:
        version: v2

The canary configuration above routes 10% of traffic to the v2 Pods and 90% to v1, controlled by label selectors, not by replica count. You can gradually increase the v2 weight as confidence builds, without changing replica counts. If v2 shows problems, set the weight back to 0 immediately.

AWS Container Services - ECS and EKS

The Choice

AWS offers two managed container orchestration services. The choice between them is one of the most common architecture decisions in cloud infrastructure.

Amazon ECS - Elastic Container Service

ECS is AWS's own container orchestration service. It is simpler than Kubernetes. It is deeply integrated with the AWS ecosystem. It has no control plane to manage, AWS manages it entirely.

Core concepts:

Task Definition - The blueprint for a container (or group of containers). Equivalent to a Docker Compose service definition: image, CPU, memory, environment variables, ports, volumes, IAM role.

Task - A running instance of a Task Definition. Equivalent to a Docker Compose running service.

Service - Maintains a desired number of running Tasks. Replaces failed Tasks. Integrates with ALB for load balancing. Equivalent to a Kubernetes Deployment.

Cluster - The logical grouping of Tasks and Services.

Launch types:

• Fargate - Serverless. AWS provisions and manages the underlying compute. You pay per vCPU and memory per second that your Task runs. No nodes to manage, no capacity to plan.
• EC2 - You manage the EC2 instances that form the cluster. More control, more complexity, lower cost at high scale.

import boto3

ecs = boto3.client('ecs', region_name='us-east-1')

# Register a task definition
task_def = ecs.register_task_definition(
    family='my-app',
    networkMode='awsvpc',
    requiresCompatibilities=['FARGATE'],
    cpu='512',        # 0.5 vCPU
    memory='1024',    # 1 GB
    executionRoleArn='arn:aws:iam::123456789012:role/ECSTaskExecutionRole',
    taskRoleArn='arn:aws:iam::123456789012:role/ECSTaskRole',
    containerDefinitions=[
        {
            'name': 'web',
            'image': '123456789012.dkr.ecr.us-east-1.amazonaws.com/my-app:v1.2.3',
            'portMappings': [
                {'containerPort': 8000, 'protocol': 'tcp'}
            ],
            'environment': [
                {'name': 'APP_ENV', 'value': 'production'}
            ],
            'secrets': [
                {
                    'name': 'DATABASE_URL',
                    'valueFrom': 'arn:aws:secretsmanager:us-east-1:123456789012:secret:prod/database-url'
                }
            ],
            'logConfiguration': {
                'logDriver': 'awslogs',
                'options': {
                    'awslogs-group': '/ecs/my-app',
                    'awslogs-region': 'us-east-1',
                    'awslogs-stream-prefix': 'web'
                }
            },
            'healthCheck': {
                'command': ['CMD-SHELL', 'curl -f http://localhost:8000/health || exit 1'],
                'interval': 30,
                'timeout': 5,
                'retries': 3,
                'startPeriod': 60
            },
            'essential': True
        }
    ]
)

# Create a service
service = ecs.create_service(
    cluster='production-cluster',
    serviceName='my-app',
    taskDefinition='my-app',
    desiredCount=3,
    launchType='FARGATE',
    networkConfiguration={
        'awsvpcConfiguration': {
            'subnets': ['subnet-0abc1234', 'subnet-0def5678'],
            'securityGroups': ['sg-0abc1234'],
            'assignPublicIp': 'DISABLED'
        }
    },
    loadBalancers=[
        {
            'targetGroupArn': 'arn:aws:elasticloadbalancing:us-east-1:123456789012:targetgroup/my-app/abc123',
            'containerName': 'web',
            'containerPort': 8000
        }
    ],
    deploymentConfiguration={
        'maximumPercent': 200,
        'minimumHealthyPercent': 100,
        'deploymentCircuitBreaker': {
            'enable': True,
            'rollback': True    # Automatically rollback if deployment fails
        }
    }
)

Amazon EKS - Elastic Kubernetes Service

EKS is AWS's managed Kubernetes service. AWS manages the control plane, the API server, etcd, scheduler, and controller manager. You manage the worker nodes (or use Fargate for serverless nodes).

EKS is Kubernetes. Every tool, pattern, and concept in sections 9-15 of this guide applies directly to EKS.

# Create an EKS cluster
eksctl create cluster \
  --name production-cluster \
  --region us-east-1 \
  --version 1.28 \
  --nodegroup-name standard-nodes \
  --node-type m5.large \
  --nodes 3 \
  --nodes-min 2 \
  --nodes-max 10 \
  --managed

# Configure kubectl
aws eks update-kubeconfig \
  --region us-east-1 \
  --name production-cluster

# Verify connection
kubectl get nodes

# Install AWS Load Balancer Controller
helm repo add eks https://aws.github.io/eks-charts
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=production-cluster \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller

# Install EBS CSI Driver for persistent volumes
eksctl create addon \
  --name aws-ebs-csi-driver \
  --cluster production-cluster \
  --region us-east-1

ECS vs EKS — When to Use Which

The honest answer: If your team knows Kubernetes and you want to use standard tooling, use EKS. If your team is AWS-native and you want simplicity, use ECS with Fargate. Both are production-grade. Neither is wrong.

Real Architecture Patterns

Pattern 1 - ECS Fargate Production Stack

Internet
    ↓
Route 53 (DNS)
    ↓
Application Load Balancer (HTTPS)
    ↓
ECS Service (Fargate) - 3 Tasks across 3 AZs
    ↓              ↓
Aurora PostgreSQL  ElastiCache Redis
(Multi-AZ RDS)    (cluster mode)

All tasks run in private subnets. No public IPs. All traffic flows through the ALB. Secrets from AWS Secrets Manager injected at task startup. CloudWatch Container Insights for monitoring. ECS service auto-scaling based on ALB request count.

Pattern 2 - EKS Production Stack

Internet
    ↓
Route 53 (DNS)
    ↓
AWS ALB (via AWS Load Balancer Controller)
    ↓
Istio Ingress Gateway
    ↓
Kubernetes Services
    ↓
Pods (Deployment, 3 replicas, spread across 3 AZs)
    ↓              ↓              ↓
Aurora RDS     ElastiCache    S3 (via IRSA)

Istio handles mTLS between services, retry policies, and canary deployments. IRSA (IAM Roles for Service Accounts) gives Pods specific AWS permissions without static credentials. HPA (Horizontal Pod Autoscaler) scales based on CPU and custom metrics. Prometheus + Grafana for monitoring.

Pattern 3 - CI/CD to Kubernetes

Developer pushes code
    ↓
GitHub Actions triggers CI pipeline
    ↓
Run tests (unit, integration)
    ↓
Build Docker image
    ↓
Push to ECR (tagged with git commit SHA)
    ↓
ArgoCD detects new image in Helm values
    ↓
ArgoCD applies updated Deployment to EKS cluster
    ↓
Kubernetes rolling update (zero downtime)
    ↓
New pods pass readiness probe
    ↓
Old pods terminated

ArgoCD implements GitOps, the cluster state is declared in Git. Any change to the cluster goes through a pull request. ArgoCD continuously syncs the cluster to match the Git repository.

Written by Onyedikachi Obidiegwu | Cloud Security Engineer