Build a Kubernetes Pod Autoscaler with Python

tags: python, kubernetes, devops, advanced

tags: python, kubernetes, devops, tutorial

Build a Kubernetes Operator with Python

Imagine you’re the developer who just realized that managing a fleet of custom microservices in Kubernetes means writing YAML over YAML, manually scaling deployments, and constantly tweaking ConfigMaps. It’s tedious, error-prone, and frankly, not why you wanted to build software. But there’s a better way: Kubernetes Operators. And yes, you can build one in Python—no Go required.

Operators automate the lifecycle of custom resources in Kubernetes. They watch for specific events (like a custom resource being created) and execute logic to keep your system in the desired state. While Go and the controller-runtime library are the traditional tools, Python offers a more accessible, expressive alternative—especially if you’re already comfortable with the language. With the kopf framework, you can write a fully functional operator in under 100 lines of code.

Let’s build one together.

Why Python for Kubernetes Operators?

You might wonder: Why not use Go? Go is powerful and the official language for many Kubernetes projects, but it has a steep learning curve for newcomers. Python, on the other hand, is readable, fast to prototype, and has a rich ecosystem of libraries. Plus, with kopf, you get a declarative, decorator-based API that feels natural to Python developers.

kopf (Kubernetes Operators Framework) is a lightweight Python library that handles the heavy lifting: connecting to the cluster, watching resources, and managing reconciliation loops. You focus on the logic, not the plumbing.

Step 1: Set Up Your Project

Start by creating a new directory for your operator:

mkdir python-k8s-operator
cd python-k8s-operator
python3 -m venv venv
source venv/bin/activate

Install the required dependencies:

pip install kopf kubernetes
pip freeze > requirements.txt

You now have a clean virtual environment with kopf and the official Kubernetes client library.

Step 2: Define Your Custom Resource (CRD)

Before writing the operator, you need a Custom Resource Definition (CRD)—the blueprint for your custom resource. This tells Kubernetes what your new resource looks like.

Create a file named myresource-crd.yaml:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: myresources.example.com
spec:
  group: example.com
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                replicas:
                  type: integer
                image:
                  type: string
  scope: Namespaced
  names:
    plural: myresources
    singular: myresource
    kind: MyResource

Apply it to your cluster:

kubectl apply -f myresource-crd.yaml

This creates a new resource type MyResource in the example.com API group.

Step 3: Write the Operator Logic

Now, let’s write the Python code. Create a file named my_operator.py:

import kopf
import kubernetes.client
from kubernetes.client.rest import ApiException

@kopf.on.create('example.com', 'v1', 'myresources')
@kopf.on.update('example.com', 'v1', 'myresources')
def handle_myresource(name, namespace, spec, logger, **kwargs):
    logger.info(f"Handling MyResource: {name} in {namespace}")

    replicas = spec.get('replicas', 1)
    image = spec.get('image', 'nginx:latest')

    client = kubernetes.client.AppsV1Api()

    deployment = kubernetes.client.V1Deployment(
        api_version='apps/v1',
        kind='Deployment',
        metadata=kubernetes.client.V1ObjectMeta(
            name=f'{name}-deployment',
            namespace=namespace
        ),
        spec=kubernetes.client.V1DeploymentSpec(
            replicas=replicas,
            selector=kubernetes.client.V1LabelSelector(
                match_labels={'app': name}
            ),
            template=kubernetes.client.V1PodTemplateSpec(
                metadata=kubernetes.client.V1ObjectMeta(labels={'app': name}),
                spec=kubernetes.client.V1PodSpec(
                    containers=[
                        kubernetes.client.V1Container(
                            name='app',
                            image=image,
                            ports=[kubernetes.client.V1ContainerPort(container_port=80)]
                        )
                    ]
                )
            )
        )
    )

    try:
        client.create_namespaced_deployment(namespace=namespace, body=deployment)
        logger.info(f"Created deployment {name}-deployment")
    except ApiException as e:
        logger.error(f"Failed to create deployment: {e}")

This operator:

Watches for create and update events on MyResource.
Reads replicas and image from the spec.
Creates or updates a corresponding Kubernetes Deployment.

It’s simple, but it’s production-ready logic.

Step 4: Run the Operator Locally

Test your operator before deploying it:

kopf run my_operator.py

kopf will connect to your local cluster (via kubectl config) and start listening for events.

Now, create a sample custom resource:

apiVersion: example.com/v1
kind: MyResource
metadata:
  name: demo-app
  namespace: default
spec:
  replicas: 3
  image: nginx:1.25

Save it as demo-resource.yaml and apply it:

kubectl apply -f demo-resource.yaml

Watch your terminal—kopf should log that it’s handling the resource and creating the deployment. Verify:

kubectl get deployments
kubectl get pods -l app=demo-app

You’ll see 3 pods running nginx:1.25.

Step 5: Containerize and Deploy

To run your operator in the cluster, you need to containerize it. Create a Dockerfile:

FROM python:3.9-slim
WORKDIR /app
COPY my_operator.py .
COPY requirements.txt .
RUN pip install --no-cache-dir --upgrade pip && pip install --no-cache-dir -r requirements.txt
CMD ["kopf", "run", "/app/my_operator.py"]

Build and push the image:

docker build -t your-dockerhub-username/my-operator:latest .
docker push your-dockerhub-username/my-operator:latest

Now, create a deployment manifest (operator-deployment.yaml) that references your image, along with a ServiceAccount, Role, and RoleBinding to grant the operator permissions to create deployments.

Apply everything:

kubectl apply -f crd.yaml
kubectl apply -f role.yaml
kubectl apply -f binding.yaml
kubectl apply -f operator-deployment.yaml

Verify the operator pod is running:

kubectl get pods -l app=my-operator

Once it’s up, your operator is live in the cluster.

What You Can Do Next

This is just the starting point. With kopf, you can:

Handle deletion events with @kopf.on.delete.
Add timers for periodic reconciliation using @kopf.timer.
Manage ConfigMaps, Secrets, or even external APIs.
Build multi-resource operators that orchestrate entire stacks.

The beauty of Python operators is their simplicity. You don’t need to master complex CRDs, deeply understand Go concurrency, or wrestle with code generation. You just write clean, testable Python.

Ready to Automate Your Cluster?

You now have a working Kubernetes Operator in Python that responds to custom resources and manages deployments. That’s a powerful tool you can use today to automate scaling, configuration, or any custom workflow in your cluster.

Try extending this operator: add health checks, integrate with a monitoring system, or build a sidecar injector. The best part? You’re doing it in a language you already love.

Go build your next operator. Share it on Dev.to, open-source it on GitHub, or use it to solve a real problem in your team. The Kubernetes ecosystem is waiting for more Python-powered automation—and you’re now part of it.

If you found this helpful, consider buying me a coffee ☕ — it keeps these articles coming!

Also check out my AI tools collection: AI 次元世界 — free AI tools for developers.