Kubernetes Networking [Level-8: kube-proxy / iptables / IPVS / eBPF]

This is Level 8 of our Kubernetes networking series, and it's where we finally go inside the Service dataplane. Back in Level 3, we learned that a client Pod talks to a Service's ClusterIP, which somehow routes traffic to a backend Pod:

Client Pod → Service (10.96.120.50:80) → Backend Pod (10.244.1.10:80)

We deliberately left one question unanswered until now: how does traffic sent to that Service IP actually end up at a real Pod? That's exactly what this article unpacks.

Table of Contents

1. The Problem: A Service IP Isn't a Real Interface
2. The Basic Idea: A Service Dataplane
3. What Is kube-proxy?
4. kube-proxy's Job
5. kube-proxy Doesn't Create Pods
6. kube-proxy Runs on Every Node
7. The Classic Mental Model: iptables and DNAT
8. What Is DNAT?
9. Tracing the Packet Journey
10. The Full Picture: DNS to Pod
11. iptables in Kubernetes
12. Control Plane vs Dataplane
13. Why EndpointSlices Matter Here
14. Load Balancing Across Endpoints
15. Session Affinity
16. The Scaling Problem with iptables
17. IPVS: IP Virtual Server
18. iptables vs IPVS
19. An Important Modern Nuance
20. eBPF Enters the Picture
21. Why eBPF Is Interesting for Kubernetes
22. Cilium and eBPF
23. eBPF Can Replace Parts of kube-proxy
24. Don't Confuse kube-proxy Replacement with CNI
25. Three Generations to Remember
26. What kube-proxy Actually Watches
27. Why Services Survive Pod Churn
28. Where CNI Fits Into All of This
29. Service vs CNI vs kube-proxy: A Reference Table
30. Following One Complete Packet
31. What Is SNAT?
32. Why SNAT Is Needed
33. DNAT vs SNAT
34. Why Return Traffic Works
35. NodePort, Revisited
36. LoadBalancer, Revisited
37. A Layered Troubleshooting Model
38. A Production Troubleshooting Tree
39. What to Inspect on a Node
40. How Do I Know Which Dataplane I'm Using?
41. The Most Important Conceptual Separation
42. Interview Questions
43. Level 8 Mental Model
44. What's Next: Kubernetes Networking Internals

The Problem: A Service IP Isn't a Real Interface

Suppose we have a frontend Pod at 10.244.1.5, and a Service named backend with ClusterIP 10.96.120.50:80, backed by three Pods:

10.244.1.10:80
10.244.2.10:80
10.244.3.10:80

The frontend runs curl http://backend:80. DNS resolves backend → 10.96.120.50, so the outgoing packet looks like:

SOURCE:      10.244.1.5
DESTINATION: 10.96.120.50:80

Here's the strange part: 10.96.120.50 usually isn't a real, physical network interface attached to any Pod. So who transforms 10.96.120.50:80 into something like 10.244.2.10:80? That transformation is the job of the Service dataplane — and that's exactly what this article is about.

The Basic Idea: A Service Dataplane

Service (10.96.120.50:80)
       ↓
Service dataplane
       ↓
Backend Pod (10.244.2.10:80)

Historically, Kubernetes clusters implemented this using kube-proxy. Modern clusters can implement it very differently. The core concepts we need to understand across this article are: kube-proxy, iptables, IPVS, and eBPF.

What Is kube-proxy?

Despite its name, kube-proxy isn't simply a traditional HTTP proxy — that's a genuinely useful interview point to remember. kube-proxy is a component that helps implement Kubernetes Service networking on each Node:

Service
   ↓
kube-proxy / service dataplane
   ↓
backend Pods

It watches Kubernetes objects — specifically Services and EndpointSlices — and programs the Node's networking rules/dataplane to match what it observes.

kube-proxy's Job

Given a Service backend at 10.96.120.50:80 with EndpointSlice entries pointing to 10.244.1.10:80, 10.244.2.10:80, and 10.244.3.10:80, kube-proxy's conceptual job is simple:

"When traffic arrives for this Service IP and port, send it to one of these backend endpoints."

10.96.120.50:80
        ↓
 ┌──────┼──────┐
 ↓      ↓      ↓
Pod 1  Pod 2  Pod 3

kube-proxy Doesn't Create Pods

Another important distinction worth locking in:

Deployment        → creates/manages Pods
Service           → defines a stable endpoint
kube-proxy/dataplane → implements Service traffic forwarding

Three genuinely different responsibilities, each handled by a different component.

kube-proxy Runs on Every Node

kube-proxy typically runs as a DaemonSet, so every Node gets its own instance:

Node 1 → Pods + kube-proxy
Node 2 → Pods + kube-proxy
Node 3 → Pods + kube-proxy

Each Node needs the ability to handle Service traffic that arrives locally.

The Classic Mental Model: iptables and DNAT

Historically, kube-proxy commonly programmed iptables rules to do this work:

Pod
 ↓
Service IP
 ↓
iptables rules
 ↓
DNAT
 ↓
Pod IP

The key term here is DNAT.

What Is DNAT?

DNAT = Destination Network Address Translation. Suppose a packet starts out as:

SOURCE:      10.244.1.5
DESTINATION: 10.96.120.50:80

The Service dataplane can rewrite the destination:

SOURCE:      10.244.1.5
DESTINATION: 10.244.2.10:80

So 10.96.120.50 → 10.244.2.10 — that transformation is destination NAT in action.

Tracing the Packet Journey

Let's slow this down and walk through it explicitly.

Frontend: 10.244.1.5 — Service: 10.96.120.50:80 — Backend: 10.244.2.10:80

1. The frontend sends a packet: SRC = 10.244.1.5, DST = 10.96.120.50:80.
2. The Node's Service dataplane recognizes 10.96.120.50:80 and selects a backend, say 10.244.2.10:80.
3. The packet effectively becomes: SRC = 10.244.1.5, DST = 10.244.2.10:80.
4. From here, normal Pod networking (CNI, covered in Level 7) takes over and delivers it.

The Full Picture: DNS to Pod

Connecting this back to Level 4:

Pod
 ↓
DNS
 ↓
Service name
 ↓
ClusterIP
 ↓
Service dataplane
 ↓
Endpoint
 ↓
Pod IP
 ↓
CNI networking
 ↓
destination Pod

Or, with concrete values:

frontend (10.244.1.5)
   | "backend"
   ↓
DNS
   ↓
10.96.120.50
   ↓
Service dataplane
   ↓
10.244.2.10
   ↓
CNI
   ↓
backend Pod

iptables in Kubernetes

Linux has a built-in packet-filtering and NAT framework called iptables. Kubernetes can program iptables rules so that Service traffic gets redirected to the correct backend endpoint:

Packet
  ↓
iptables
  ↓
Service rule
  ↓
Backend selection
  ↓
DNAT
  ↓
Pod

Conceptually, given a Service at 10.96.120.50:80 with endpoints 10.244.1.10:80, 10.244.2.10:80, and 10.244.3.10:80, the Node might carry rules that resemble: "traffic to 10.96.120.50:80 → choose an endpoint → DNAT to one of the three addresses above." Real Kubernetes iptables rule sets are considerably more complex than this simplification — don't worry about memorizing actual chain names yet.

Control Plane vs Dataplane

It helps to think of Kubernetes networking in two layers:

Control plane decides what should happen: which Pods exist, which Services exist, which endpoints belong to a Service.

Dataplane actually handles packets.

Control plane → "Service backend Pods are A, B, C"
      ↓
Dataplane → "Send packets to A/B/C"

Why EndpointSlices Matter Here

Recall from Level 3: a Service like backend has an associated EndpointSlice listing its live backend IPs. The Service dataplane depends directly on this information:

Service + EndpointSlice → Service dataplane → forwarding rules

Without accurate, up-to-date EndpointSlice data, the dataplane has nothing correct to forward traffic to.

Load Balancing Across Endpoints

Given a Service with three backend Pods, a connection to the Service might land on any one of them:

Request 1 → Pod A
Request 2 → Pod C
Request 3 → Pod B
Request 4 → Pod A

The exact algorithm and distribution behavior depends on the dataplane implementation and configuration. The key idea to hold onto: the Service is a stable front door, while the dataplane is what actually selects a backend endpoint for each connection.

Session Affinity

Kubernetes Services can optionally enable session affinity, e.g.:

sessionAffinity: ClientIP

With this enabled, the same client tends to keep landing on the same backend Pod:

Client A → Pod A
Client A → Pod A
Client A → Pod A

Again, the precise implementation details depend on which dataplane your cluster is running.

The Scaling Problem with iptables

The classic kube-proxy → iptables → DNAT → Pod flow works well for many clusters. But there's a scaling consideration: compare 10 Services with 100 total endpoints to 10,000 Services with 100,000 total endpoints. The volume of networking rules involved can grow enormous. This is exactly the pressure that led to IPVS, and eventually to eBPF.

IPVS: IP Virtual Server

IPVS stands for IP Virtual Server — a Linux kernel technology purpose-built for high-performance load balancing:

Service IP (10.96.120.50:80)
       ↓
      IPVS
       ↓
 ┌───┼───┐
 ↓   ↓   ↓
Pod A Pod B Pod C

Instead of thinking in terms of a long chain of packet-filter rules, think of IPVS as a kernel-level virtual server / load-balancing mechanism purpose-built for exactly this job.

iptables vs IPVS

Simplified comparison:

iptables: packet → rules → rules → rules → backend

IPVS: packet → virtual service → backend selection

IPVS can offer more specialized load-balancing behavior, and historically was attractive for clusters with large numbers of Services thanks to its performance characteristics. That said, modern Kubernetes networking has moved beyond simply choosing between these two — eBPF-based dataplanes are increasingly significant.

An Important Modern Nuance

You may hear the claim: "IPVS is always faster than iptables." Don't treat that as an absolute rule. Real-world performance depends on cluster size, traffic patterns, kernel version, configuration, the specific dataplane implementation, and even the underlying hardware. The important conceptual difference to remember is simpler:

iptables → packet filtering/NAT rules
IPVS     → kernel virtual-server load balancing

eBPF Enters the Picture

Now we reach a genuinely important modern technology: eBPF. You don't need to master every detail yet — at a high level:

eBPF = programmable logic that can run inside the Linux kernel.

This lets networking systems perform operations directly in the kernel, rather than bouncing through separate userspace components or long rule chains.

Why eBPF Is Interesting for Kubernetes

The traditional path looks like:

Packet → iptables rules → NAT → routing

An eBPF-based networking implementation can inject custom packet-processing logic straight into the kernel:

Packet → eBPF program → routing / load balancing / policy → destination

This can avoid some of the overhead inherent in traditional networking paths and enables much more flexible dataplane behavior.

Cilium and eBPF

Cilium is one of the best-known Kubernetes networking implementations built on eBPF:

Kubernetes → Cilium → eBPF → Linux kernel → networking + load balancing + NetworkPolicy + observability

This is exactly why Cilium came up briefly back in Level 7 — and now the pieces should be clicking into place.

eBPF Can Replace Parts of kube-proxy

This is an important, genuinely modern Kubernetes concept. Some eBPF-based networking implementations can provide Service load balancing entirely on their own. Instead of:

Service → kube-proxy → iptables → Pod

you can have:

Service → eBPF dataplane → Pod

In such a setup, kube-proxy may not even be needed for Service handling at all. This is commonly referred to as "kube-proxy replacement," and Cilium is a well-known example of a system that offers it.

Don't Confuse kube-proxy Replacement with CNI

Keep these terms distinct in your head:

CNI                   = networking interface/ecosystem
Cilium                = a networking implementation
eBPF                  = a kernel technology
kube-proxy replacement = one specific capability/configuration

They're related, but not interchangeable. A cleaner way to see the relationship:

Cilium → uses eBPF → can implement Kubernetes networking → can provide Service load balancing → can replace kube-proxy

Three Generations to Remember

A simple learning model for Service traffic handling:

                    Service traffic

                       Service IP
                           |
            ┌──────────────┼──────────────┐
            ↓              ↓              ↓
         iptables         IPVS           eBPF
            ↓              ↓              ↓
        kernel rules   virtual server   kernel programs
            ↓              ↓              ↓
                         Pod

This isn't a strict historical replacement chain — real clusters can (and do) use any of these approaches — but it's a useful conceptual comparison to keep handy.

What kube-proxy Actually Watches

kube-proxy needs live information about Services and EndpointSlices. Whenever something changes — a Pod is added or removed, a Service is updated, an endpoint changes — the Service dataplane needs to be kept in sync:

EndpointSlice changes
        ↓
kube-proxy/dataplane notices
        ↓
forwarding state updated
        ↓
new traffic uses current endpoints

This is precisely how Kubernetes handles the constant churn of dynamic Pods without breaking client connectivity.

Why Services Survive Pod Churn

Suppose Service backend (10.96.120.50) initially has Pod A (10.244.1.10) and Pod B (10.244.2.10) as backends. Pod A dies, and a new Pod C (10.244.3.10) appears. The EndpointSlice updates:

OLD: A, B
NEW: B, C

But the Service IP itself never changes — it stays 10.96.120.50. The dataplane simply updates its internal backend information, and clients keep using backend without ever needing to know that a Pod was replaced underneath them. This is one of the single most powerful abstractions in all of Kubernetes.

Where CNI Fits Into All of This

Combining Level 7 and Level 8: CNI provides the underlying Pod network:

Pod → eth0 → veth → Node → Pod network

The Service dataplane provides Service-level forwarding:

Service IP → backend Pod

Together:

                    Kubernetes networking

Pod
 | CNI
 ↓
Pod network
 ↓
Service dataplane
 ↓
another Pod

The exact ordering within the Linux packet path can get more nuanced in practice, but this separation of responsibilities is the key mental model to carry forward.

Service vs CNI vs kube-proxy: A Reference Table

Following One Complete Packet

Let's trace one full request end-to-end. We have frontend (10.244.1.5), Service backend (10.96.120.50:80), and a backend Pod (10.244.2.10:80). The frontend runs curl http://backend.

Step 1 — DNS: backend → 10.96.120.50.

Step 2 — Application sends packet: SRC = 10.244.1.5, DST = 10.96.120.50:80.

Step 3 — Service dataplane: either kube-proxy → iptables/IPVS, or an eBPF dataplane, takes over.

Step 4 — Destination translation/selection: 10.96.120.50:80 → 10.244.2.10:80.

Step 5 — Pod networking: CNI provides the actual route to 10.244.2.10.

Step 6 — Backend receives traffic: Frontend → Service → Service dataplane → Backend Pod.

What Is SNAT?

We've covered DNAT — now meet its counterpart: SNAT = Source Network Address Translation.

DNAT → destination changes
SNAT → source changes

For example, SRC = Pod IP, DST = external destination might become SRC = Node IP, DST = external destination. This matters specifically when Pods communicate outside the cluster.

Why SNAT Is Needed

Suppose Pod 10.244.1.10 tries to reach the internet. The external destination generally has no idea how to route return traffic back to 10.244.1.10 — that's a private, cluster-internal address. So the networking system may perform SNAT:

10.244.1.10
      ↓ SNAT
192.168.1.10  (Node IP)

Conceptually: Pod → SNAT → Node IP → Internet. The exact behavior depends on your specific cluster's networking architecture.

DNAT vs SNAT

Worth memorizing permanently:

DNAT — D = Destination
SNAT — S = Source

Example (Service → Pod): 10.96.120.50 → DNAT → 10.244.2.10

Example (Pod → outside): 10.244.1.10 → SNAT → Node IP

Why Return Traffic Works

This is where NAT/connection tracking becomes essential. Given Client (10.244.1.5) → Service (10.96.120.50) → Backend (10.244.2.10), the forward path goes Client → Service IP → DNAT → Backend. For the response to correctly find its way back, the networking stack tracks the connection state so return traffic can be translated appropriately in reverse:

Forward: Service → Backend
Return:  Backend → Client

This bookkeeping is handled through connection/NAT state tracking within the Linux networking stack (often referred to as "conntrack").

NodePort, Revisited

With a NodePort Service, given a Node IP of 192.168.1.10 and NodePort 30080, external traffic to 192.168.1.10:30080 can be forwarded toward the Service's backend Pods:

External client
      ↓
NodeIP:30080
      ↓
Service dataplane
      ↓
Backend Pod

The exact mechanics again depend on which dataplane your cluster is running.

LoadBalancer, Revisited

With type: LoadBalancer, the flow typically looks like:

Internet
   ↓
Cloud Load Balancer
   ↓
Node / Service
   ↓
Backend Pods

Both the cloud load balancer and the Kubernetes Service dataplane participate in the overall traffic path — which is exactly why troubleshooting external traffic often requires checking multiple distinct layers.

A Layered Troubleshooting Model

Now that we understand the full stack, we can build a much better troubleshooting model. If curl http://backend fails, don't immediately blame DNS. Break it down layer by layer:

1. DNS
   ↓
2. Service
   ↓
3. EndpointSlice
   ↓
4. Service dataplane
   ↓
5. Pod networking
   ↓
6. Application

Step 1 — DNS:

nslookup backend

Does it resolve to the expected Service IP? If backend → 10.96.120.50, DNS is probably fine.

Step 2 — Service:

kubectl get svc backend

Check the ClusterIP, port, and targetPort.

Step 3 — EndpointSlice:

kubectl get endpointslices
kubectl describe svc backend

Ask directly: does the Service actually have backend endpoints? If you see Endpoints: <none>, don't waste time debugging iptables yet — the Service simply has no backends to send traffic to.

Step 4 — Pod:

kubectl get pods -o wide

Confirm the backend Pods are Running, Ready, and have assigned IPs.

Step 5 — Service dataplane: Ask whether the Service dataplane is correctly forwarding traffic — investigate kube-proxy, iptables, IPVS, or your eBPF/CNI dataplane as appropriate. This is exactly where Level 8 knowledge pays off.

Step 6 — CNI: If Service forwarding looks correct but the destination Pod still isn't reachable, investigate CNI, routing, the Pod's interface, Node connectivity, and NetworkPolicy.

A Production Troubleshooting Tree

                Request fails
                     |
                     ↓
                  DNS?
                 /     \
               NO       YES
               |         |
           Fix DNS     Service?
                         /   \
                       NO     YES
                       |       |
                  Fix Service  EndpointSlice?
                                /       \
                              NO         YES
                              |           |
                        Fix selectors   Dataplane?
                                         /     \
                                       NO       YES
                                       |         |
                                  kube-proxy/   CNI/
                                  eBPF/etc.    routing/
                                               policy

This structured approach beats randomly running commands and hoping something jumps out.

What to Inspect on a Node

For a traditional kube-proxy cluster using iptables, you can inspect:

iptables-save
# or
iptables -t nat -S

These outputs can be extremely large on real clusters — don't be surprised by the volume. For IPVS-based setups, if installed:

ipvsadm -L

For eBPF-based implementations, the relevant tooling depends heavily on the specific implementation — Cilium, for example, ships its own observability and status tooling. Don't run diagnostic commands blindly in production — first figure out which dataplane your cluster actually uses.

How Do I Know Which Dataplane I'm Using?

Start broad:

kubectl get pods -A

Look for names like kube-proxy, calico, cilium, flannel, or aws-node. Then:

kubectl get daemonsets -A

For example, seeing a kube-proxy DaemonSet suggests kube-proxy is deployed — but you should still inspect its configuration to determine whether it's actually running in iptables mode, IPVS mode, or something else entirely.

The Most Important Conceptual Separation

By now, you should be able to cleanly separate these layers in your head:

              Kubernetes networking

                 Service
                    |
                    ↓
             Service dataplane
          ┌─────────┼─────────┐
          ↓         ↓         ↓
      iptables     IPVS      eBPF
          \         |         /
           └────────┼────────┘
                    ↓
                 Pod IP
                    |
                    ↓
               CNI network
                    |
                    ↓
                  Pod

Interview Questions

Q1. What does kube-proxy do?
It traditionally implements Kubernetes Service networking on Nodes by programming a Service dataplane.

Q2. Is kube-proxy an HTTP proxy?
No — the name is genuinely misleading.

Q3. What is a ClusterIP?
A stable virtual IP associated with a Kubernetes Service.

Q4. How does a ClusterIP actually reach a Pod?
Through the Service dataplane — traditionally via iptables/IPVS, or through an alternative dataplane such as eBPF.

Q5. What is DNAT?
Destination Network Address Translation — conceptually, Service IP → Pod IP.

Q6. What is SNAT?
Source Network Address Translation — conceptually, Pod IP → Node IP in some outbound scenarios.

Q7. What is IPVS?
Linux IP Virtual Server — a kernel-level load-balancing mechanism.

Q8. What is eBPF?
A Linux kernel technology that allows programmable logic to run in kernel contexts, used by modern networking systems for routing, load balancing, security, and observability.

Q9. Can eBPF replace kube-proxy?
Yes — some implementations, such as Cilium, can provide full Kubernetes Service functionality without needing kube-proxy at all.

Q10. What does EndpointSlice provide?
Information about the current backend endpoints associated with a Service.

Q11. Is CNI the same as kube-proxy?
No. CNI/network implementation handles Pod networking; kube-proxy/service dataplane handles Service forwarding.

Q12. What happens if a Service has no EndpointSlices/endpoints?
The Service has no backend Pods to send traffic to — regardless of whether the Service dataplane itself is working correctly.

Level 8 Mental Model

If you take away only one diagram from this entire article, make it this one:

                  Service
              10.96.120.50:80
                      |
                      ↓
             Service dataplane
                      |
          ┌───────────┼───────────┐
          ↓           ↓           ↓
       iptables      IPVS        eBPF
          └───────────┼───────────┘
                      ↓
                Pod endpoint
                10.244.2.10
                      |
                      ↓
                     CNI
                      |
                      ↓
                    Pod

And the responsibility of each layer:

• CNI → "How do Pods communicate?"
• Service → "What stable endpoint do clients use?"
• EndpointSlice → "Which Pods are backends?"
• kube-proxy / dataplane → "How do Service packets reach those backends?"
• iptables / IPVS / eBPF → "How is that forwarding actually implemented?"

What's Next: Kubernetes Networking Internals

In Level 9, we'll connect everything into the actual Linux packet path — diving into network namespaces, veth pairs, bridges, routing tables, conntrack, NAT, the kube-proxy/CNI interaction, Node-to-Pod versus Pod-to-Pod traffic, and exactly what happens to a single packet as it moves through a Kubernetes Node.

Conclusion

The Service dataplane is where Kubernetes networking stops being abstract and becomes real packet manipulation. A ClusterIP is never a physical address a Pod actually owns — it's a rule, enforced by kube-proxy (or an eBPF-based alternative), that rewrites destinations via DNAT and tracks connections so return traffic finds its way home. Whether that enforcement happens through iptables rule chains, IPVS's kernel-level virtual server, or a fully custom eBPF dataplane, the underlying job is the same: turn a stable Service IP into real traffic to a real, currently-healthy Pod.

The biggest practical takeaway: when Service traffic breaks, work the layers in order — DNS, Service, EndpointSlice, dataplane, then CNI. An empty EndpointSlice will waste hours of your time if you jump straight to debugging iptables rules instead of checking it first.

Enjoyed this deep dive? Follow along for Level 9, where we trace a single packet's complete journey through the Linux networking stack inside a real Kubernetes Node. Drop your questions in the comments, and share this with a teammate who still thinks kube-proxy is a literal proxy server. Let's keep building this Kubernetes networking roadmap together. 🚀