How CDNs Decide Where to Serve Your Request From: The Routing and Caching Engine

Content Delivery Networks (CDNs) intercept user requests at the network perimeter, utilizing distributed edge infrastructure to execute routing decisions, terminate TLS sessions, mitigate distributed denial-of-service (DDoS) vectors, and evaluate local cache states before falling back to centralized origin infrastructure. When an end-user issues an HTTP request, the global routing and edge execution engine determines the optimal processing node by evaluating network topology, Anycast BGP announcements, device capabilities, and data residency constraints.

Featured Snippet Definition

A Content Delivery Network (CDN) decides where to serve a request from by combining Border Gateway Protocol (BGP) Anycast routing to direct user packets to the topologically closest edge data center, followed by a local data plane lookup of the request's cache key against local NVMe or RAM-backed storage layers. If a cache miss occurs, the edge node evaluates whether to fetch from an intermediate origin shield or query the primary application server.

Architectural Taxonomy & Core Trade-offs

Modern content delivery relies on separating the control plane—which handles configuration propagation, DNS zone updates, and SSL/TLS certificate distribution—from the data plane, which processes high-throughput TCP/UDP packet ingress, TLS termination, and request proxying at line rate.

[ User Device ] 
       │ (1) DNS Query / Anycast BGP Packet
       ▼
[ Anycast Edge Router ] 
       │ (2) Data Plane Packet Ingress
       ▼
[ Edge Proxy / TLS Termination ] 
       │ (3) Cache Key Generation & Lookup
       ├─────────────────────────────────┐
       │ (Cache Hit)                     │ (Cache Miss)
       ▼                                 ▼
[ Local NVMe / RAM Cache ]       [ Origin Shield / Intermediate Cache ]
       │                                 │
       │ (Return Response)               ▼ (If Shield Miss)
       └──────────────────────────> [ Central Origin Server ]

Primary Architectural Variants

The CDN Request Path: From User to Edge

The end-to-end traversal of a client request involves distinct networking and application phases. When analyzing system behavior, consider how distributed cloud storage architecture how object systems survive server failures informs how edge nodes pull immutable assets from backing object stores.

1. Network Routing & DNS Resolution

When a client resolves a domain name pointing to a CDN, the resolution path depends on whether the provider uses DNS-based routing or BGP Anycast.

• DNS-Based Routing: The CDN authoritative nameserver inspects the IP address of the client's recursive resolver (EDNS Client Subnet) and returns an A/AAAA record pointing to the geographically closest data center. This approach provides precise control over traffic distribution, but suffers from high latency when recursive resolvers are distant from the actual users (e.g., enterprise VPNs or public resolvers like 8.8.8.8 routing traffic through centralized exits).
• Anycast Routing: The CDN announces the exact same set of IP addresses from multiple Points of Presence (PoPs) worldwide via Border Gateway Protocol (BGP). Autonomous System (AS) path selection naturally steers the user's packets to the topologically closest router along the shortest path computed by Internet service providers.

2. Edge Ingress & Data Plane Processing

Once packets arrive at the edge PoP, the network interface card (NIC) handles hardware packet acceleration (such as DPDK or eBPF/XDP), passing raw TCP/UDP streams to the reverse proxy tier (typically customized NGINX, Envoy, or proprietary C/Rust runtimes).

• TLS Termination: The edge proxy performs cryptographic handshakes (TLS 1.3), utilizing session resumption tickets to minimize handshake round trips.
• WAF & Security Filtering: Inline Web Application Firewall (WAF) rules, bot filtering engines, and rate limiters evaluate the HTTP headers and payload before cache lookup occurs.

3. Cache Evaluation & Key Generation

The edge proxy constructs a cache key based on configurable parameters (URI, query strings, headers). It queries the local storage engine (operating across RAM, persistent NVMe arrays, and kernel page caches).

• If a valid, non-expired cache entry exists (Cache Hit), the response is serialized and returned immediately.
• If the asset is missing, expired, or explicitly marked no-store (Cache Miss), the request enters the origin fetch workflow.

Latency Is Not Just Geographic Distance

Engineers frequently conflate physical distance with network latency. A user in London connecting to an edge node 50 miles away can experience higher latency than a user 500 miles away due to underlying transport realities:

[ Physical Distance (km) ] ──> [ ISP Routing Policies & Peering ] ──> [ BGP Congestion & Packet Loss ] ──> [ Edge Processing Overhead ] ──> [ Final TTFB ]

• Physical Distance & Speed of Light: Light travels through fiber optic cable at roughly $200,000\text{ km/s}$, introducing an irreducible floor of ~0.01ms per kilometer of round-trip time (RTT).
• ISP Path & Peering Interconnects: Sub-optimal transit agreements between local ISPs and CDN edge networks force packets through third-party transit providers, adding unnecessary AS hops.
• Congestion and Packet Loss: TCP congestion control algorithms (such as BBR or CUBIC) react to dropped packets by throttling window sizes, degrading Time to First Byte (TTFB) independently of raw distance.
• Edge Processing Time: Complex WAF rule evaluations, heavy regex matching on query strings, or synchronous auth subrequests consume CPU cycles within the edge event loop, increasing queue latency.

Cache Mechanics, Keys, and Shielding

Cache-Key Design and Fragmentation

An unoptimized cache key design destroys cache efficiency. If a CDN constructs cache keys using every random query parameter (e.g., tracking tags like utm_source, session IDs, or timestamps), the effective cache hit ratio (CHR) drops toward zero, turning the CDN into an expensive pass-through proxy.

Anti-Pattern Cache Key:
GET /api/v1/products?id=42&utm_source=google&timestamp=1711900200&session_token=abc123xyz

Optimized Normalized Cache Key:
GET /api/v1/products?id=42 (with query parameters explicitly sorted, whitelisted, and stripped of tracking parameters).

Origin Shielding

To prevent cache stampedes and protect backend infrastructure from request amplification, modern architectures implement Origin Shielding. An origin shield is a designated intermediate CDN PoP placed in front of the origin infrastructure.

[ Edge PoP A ] ──┐
[ Edge PoP B ] ──┼──> [ Regional Origin Shield ] ──> [ Central Origin Server ]
[ Edge PoP C ] ──┘

When 50 geographically dispersed edge nodes experience a cache miss for the same asset simultaneously, they do not all hit the origin server. Instead, they query the Origin Shield. The Shield coalesces these simultaneous requests into a single upstream fetch, shielding the origin from load spikes and maximizing cache hit density at the regional tier.

Failure Modes and Resiliency

Production edge architectures must gracefully degrade when internal or external components fail:

• Edge-Node Outage: When a PoP fails due to power loss or fiber cuts, BGP Anycast automatically withdraws route announcements for that location. Upstream ISP routers dynamically recalculate routes, diverting user traffic to the next-closest healthy PoP within seconds.
• Cache Poisoning: Malicious actors manipulate unvalidated input parameters to force an error state or unauthorized response into the cache. Mitigate this by enforcing strict response-header validation (Vary headers, explicit cache-control directives) and canonicalizing input parameters.
• Origin Overload / Thundering Herd: When a popular asset expires across all edge nodes, concurrent downstream requests can overwhelm the origin. Mitigate this via request collapsing (lock-step queuing where only one worker fetches the origin while others await the result) and stale-while-revalidate policies.

Reconciled TCO Financial Model

When evaluating enterprise CDN architecture, organizations must account for multi-variable pricing structures across global data transfer, origin egress, request operations, and edge compute execution time.

Assumptions & Unit Conventions

• Data Volume Unit: Decimal gigabytes (1 GB = $1,000^3$ bytes).
• Pricing Tier: Illustrative enterprise volume rates for high-egress global distribution.
• Traffic Profile: 100 TB monthly egress, 10 billion HTTP requests, 200 million dynamic edge compute invocations.

Itemized Monthly Cost Breakdown

Illustrative Configuration

Below is an illustrative Terraform HCL snippet configuring enterprise CDN distribution routing rules, origin shielding, and cache-key normalization policies.

resource "aws_cloudfront_distribution" "enterprise_cdn" {
  enabled             = true
  is_ipv6_enabled     = true
  comment             = "Production Enterprise CDN Edge Routing Configuration"
  default_root_object = "index.html"

  origin {
    domain_name = "origin.internal.net"
    origin_id   = "PrimaryOrigin"

    custom_origin_config {
      http_port              = 80
      https_port             = 443
      origin_protocol_policy = "https-only"
      origin_ssl_protocols   = ["TLSv1.2", "TLSv1.3"]
    }
  }

  origin_shield {
    enabled              = true
    origin_shield_region = "us-east-1"
  }

  default_cache_behavior {
    allowed_methods  = ["GET", "HEAD", "OPTIONS"]
    cached_methods   = ["GET", "HEAD"]
    target_origin_id = "PrimaryOrigin"

    viewer_protocol_policy = "redirect-to-https"
    min_ttl                = 0
    default_ttl            = 86400
    max_ttl                = 31536000

    forwarded_values {
      query_string = true
      cookies {
        forward = "none"
      }
      query_string_cache_keys = ["id", "version"]
    }
  }

  restrictions {
    geo_restriction {
      restriction_type = "whitelist"
      locations        = ["US", "CA", "GB", "DE", "JP"]
    }
  }

  viewer_certificate {
    cloudfront_default_certificate = false
    acm_certificate_arn            = "arn:aws:acm:us-east-1:123456789012:certificate/abcdef-1234"
    minimum_protocol_version       = "TLSv1.2_2021"
  }
}

Production Decision CTA Rubric

Use this evaluation rubric when determining whether to adopt a multi-CDN strategy, implement edge compute, or deploy custom origin shielding.

[ Start Evaluation ]
       │
       ├──> Are monthly egress volumes > 50 TB AND global latency variance < 50ms required?
       │         ├── YES ──> Deploy Anycast CDN + Regional Origin Shielding
       │         └── NO  ──> Standard Single-Vendor DNS-Based CDN Sufficient
       │
       └──> Are dynamic API acceleration or custom auth validation rules required at the perimeter?
                 ├── YES ──> Implement Edge Compute (Wasm / V8 Workers) at PoP layer
                 └── NO  ──> Utilize Traditional Static Caching & Standard Edge Proxies

Originally published at WantsVibes.

Explore in-depth systems architecture breakdowns, distributed systems guides, and AI engineering benchmarks on WantsVibes.online.