System Design from Developer to Architect --- Part 1

In Part 0, we started with a simple rule: don't add architecture
because it looks scalable. Add it when a real problem makes it
necessary.

In this part, we'll start with one user, keep adding traffic, and
change the architecture only when something gives us a reason to.

Previous: Part 0 --- How to Think About System Design Without Just
Drawing
Boxes

A million users sounds like a completely different engineering problem
from one user.

And eventually, it is.

But the interesting part isn't the final architecture.

It's how we get there.

We won't begin with CDN, Redis, replicas, event brokers, and a cluster
of services.

We'll begin here:

User → Application → Database

Then we'll keep increasing traffic.

Every time something becomes a real constraint, we'll ask:

What broke, why did it break, and what is the smallest architectural
change that solves it?

That's the scaling journey.

Stage 1 --- 1 User: Keep It Boring

Imagine we've just launched a service-booking application.

One user opens the application, searches for a professional, checks a
slot, and creates a booking.

Our architecture can be extremely simple:

User
 │
 ▼
Application
 │
 ▼
Database

And that's enough.

No Redis. No Kafka. No Kubernetes. No read replicas.

There is nothing wrong with this architecture.

In fact, adding distributed infrastructure now would probably make the
system harder to build, deploy, debug, and operate without solving a
real problem.

The cheapest scaling problem is the one you don't have yet.

Stage 2 --- 100 Users: The Simple Architecture Still Works

Now we have 100 users.

Do we need microservices? Probably not.

Do we need Kafka? Probably not.

At this stage, the highest-value improvements are often much less
exciting:

• Add the right database indexes
• Fix inefficient queries
• Avoid N+1 queries
• Use database connection pooling
• Compress HTTP responses
• Handle static assets efficiently
• Add basic metrics and logs
• Measure latency before optimizing it

A surprisingly large amount of scale can come from simply making the
existing system efficient.

Sometimes:

Slow API
   ↓
Fix query / index
   ↓
Fast API

is a better scaling strategy than adding another server.

Before distributing the system, make the simple system efficient.

Stage 3 --- 10,000 Users: Scale Up Before Scaling Out

Traffic keeps growing.

Now we start seeing resource pressure:

CPU       78%
Memory    82%
p95       650ms
Traffic   ↑

The first response doesn't always need to be horizontal scaling.

We might simply give the machine more resources:

2 CPU   →   8 CPU
4 GB    →   32 GB RAM

That's vertical scaling.

It's simple because our architecture barely changes.

Vertical scaling can take us surprisingly far.

But machines don't grow forever, and a single server remains a single
failure domain.

Eventually, traffic may exceed what one instance can comfortably handle.

That's when the second server becomes interesting.

Stage 4 --- 50,000 Users: The Second Server Changes Everything

Eventually one application instance isn't enough.

So we add another.

Now we need something to decide where incoming requests should go.

Enter the load balancer.

                 Users
                   │
                   ▼
            Load Balancer
                   │
            ┌──────┴──────┐
            ▼             ▼
          App 1         App 2
            │             │
            └──────┬──────┘
                   ▼
                Database

Now we can scale horizontally by adding application instances.

But this works cleanly only when those instances are interchangeable.

And that creates our next problem.

The State Problem

Suppose the user logs in through App 1.

Their session is stored in that server's memory.

Their next request reaches App 2.

Login → App 1
Session stored locally

Next request → App 2
Session = ?

The load balancer did exactly what we asked.

Our application architecture didn't.

This is why horizontal scaling often pushes us toward stateless
application servers.

For shared session state, Redis is one possible solution:

              Load Balancer
                    │
           ┌────────┴────────┐
           ▼                 ▼
         App 1             App 2
           │                 │
           └────────┬────────┘
                    ▼
                  Redis
                    │
                    ▼
                 Database

Notice the sequence:

More traffic
    ↓
More instances
    ↓
Requests move between instances
    ↓
Local state becomes a problem
    ↓
Shared/stateless state becomes useful

Redis didn't appear because Redis is fashionable.

It appeared because the architecture developed a state problem.

Stage 5 --- 100,000 Users: Repeated Reads Start to Hurt

Some requests repeatedly fetch the same information:

GET /services
GET /categories
GET /professionals/123

If that data changes infrequently, sending every request to the database
is wasteful.

Now caching has a concrete job.

Request
   │
   ▼
Application
   │
   ▼
 Cache
  / \
HIT MISS
 │    │
 ▼    ▼
Return DB
       │
       ▼
     Cache
       │
       ▼
     Return

A cache hit avoids an unnecessary database read.

A cache miss falls back to the source of truth.

This reduces latency and database pressure.

But caching introduces new concerns:

• TTL
• Invalidation
• Eviction
• Stale data
• Cache stampedes
• Failure behavior

For a service description, some staleness may be acceptable.

For the last available booking slot, it may not be.

Caching isn't just a performance decision. It's also a correctness
decision.

Stage 6 --- 250,000 Users: The Bottleneck Moves

The API tier looks healthy.

Then monitoring shows:

API CPU             35%   ✓
API Memory          42%   ✓

DB CPU              92%   ⚠
DB Connections      95%   ⚠
p95 Query Latency   780ms ⚠

Should we add another API server?

No.

The API isn't the bottleneck anymore.

The bottleneck moved.

App 1 ──┐
App 2 ──┤
App 3 ──┼────> DATABASE 🔥
App 4 ──┤
App 5 ──┘

A struggling database does not immediately mean:

"We need sharding."

Start with evidence:

Database pressure
       │
       ├── Inspect slow queries
       ├── Verify indexes
       ├── Remove N+1 access
       ├── Reduce unnecessary reads
       ├── Cache appropriate data
       ├── Review connection usage
       └── Then consider infrastructure scaling

A bad query executed across ten replicas is still a bad query.

Scale the bottleneck only after understanding the bottleneck.

Stage 7 --- 500,000 Users: Separate Reads From Writes

Suppose our workload is heavily read-oriented.

Users browse much more often than they modify data.

We may introduce read replicas:

                    Application
                         │
              ┌──────────┴──────────┐
              ▼                     ▼
            Writes                 Reads
              │                     │
              ▼                     ▼
           Primary             Read Replicas
                                  │      │
                                  ▼      ▼
                              Replica  Replica

Writes continue to go to the primary.

Read-heavy traffic can be distributed across replicas.

But we've bought a new problem:

Replication Lag

Replication is not always instantaneous.

A user may create a booking on the primary and immediately read from a
replica that hasn't received the update yet.

The system scaled.

Consistency became more complicated.

Problem
  ↓
Solution
  ↓
New Trade-off

Stage 8 --- 750,000 Users: Stop Sending Everything Through the Backend

Images, JavaScript bundles, CSS, downloads, and other static assets
don't necessarily need to travel through application servers on every
request.

A CDN can move cacheable content closer to users.

                  Users
                    │
            ┌───────┴────────┐
            ▼                ▼
           CDN          Load Balancer
            │                │
      Static Content      API Servers

This reduces:

• Origin traffic
• Backend bandwidth
• Static-content latency
• Unnecessary application-server work

Again, the CDN has a reason to exist.

Stage 9 --- 1 Million Users: Stop Doing Everything Synchronously

Consider what happens when someone creates a booking.

Our API might perform:

Create Booking
      ↓
Send Email
      ↓
Send Push Notification
      ↓
Update Analytics
      ↓
Award Loyalty Points
      ↓
Notify Professional
      ↓
Return Response

The customer is waiting for work that doesn't necessarily need to finish
before the booking is acknowledged.

The core path may only require:

Validate
   ↓
Reserve
   ↓
Persist
   ↓
Confirm

Other work can happen asynchronously:

             Booking Service
                    │
                    ▼
              BookingCreated
                    │
                    ▼
               Event Broker
                    │
        ┌───────────┼───────────┐
        ▼           ▼           ▼
     Notify     Analytics     Loyalty

This improves responsiveness and decouples downstream work.

But now we need to think about:

• Duplicate events
• Consumer failures
• Ordering
• Retries
• Dead-letter queues
• Database/event consistency

For example:

Database COMMIT   ✓
Event publish     ✗

That's not just a messaging problem.

It's a consistency problem.

We'll explore that later in the series.

What Did We Actually Build?

We started here:

User → Application → Database

And ended somewhere closer to this:

                         Users
                           │
                           ▼
                          CDN
                           │
                           ▼
                     Load Balancer
                           │
                ┌──────────┼──────────┐
                ▼          ▼          ▼
              API 1      API 2      API 3
                │          │          │
                └──────────┼──────────┘
                           │
                   ┌───────┴────────┐
                   ▼                ▼
                 Cache          Database
                                   │
                          ┌────────┴────────┐
                          ▼                 ▼
                     Replica 1         Replica 2
                                   │
                                   ▼
                              Event Broker
                                   │
                         ┌─────────┼─────────┐
                         ▼         ▼         ▼
                       Worker   Notify   Analytics

That's a much more sophisticated architecture.

But here's the important part:

We didn't start by designing this architecture. We arrived at it.

Every component earned its place.

One Million Users Didn't Create One Scaling Problem

It created a sequence of different problems.

Growth Stage What Starts Hurting Architectural Response

1--100 Nothing significant Keep it simple

100--10K Inefficient Optimize and scale
code/queries, resource vertically
pressure

10K--50K Single-server capacity Horizontal scaling +
load balancing

50K--100K Instance-local state Stateless services /
shared state

100K--250K Repeated expensive Caching
reads

250K--500K Database pressure Query optimization +
read scaling

500K--750K Origin/static-content CDN / edge delivery
load

750K--1M Too much synchronous Async processing /
work event-driven workflows

These user counts are illustrative, not universal thresholds.

A read-heavy application may hit database limits much earlier.

A compute-heavy application may hit CPU limits first.

A media platform may need a CDN almost immediately.

A financial system may prioritize consistency and transaction boundaries
long before raw traffic becomes interesting.

Scale is workload-specific.

Don't Scale Users. Scale Bottlenecks.

You don't actually scale because you reached:

100,000 users

You scale because something measurable changed:

CPU              ↑
Memory           ↑
Connections      ↑
Queue depth      ↑
Database latency ↑
Error rate       ↑
p95 / p99        ↑

User count is context.

Resource pressure and system behavior tell you what needs to change.

Two applications with one million users can require completely different
architectures.

The Scaling Loop

The mental model from Part 0 still applies:

Traffic grows
     ↓
Observe
     ↓
Find bottleneck
     ↓
Understand cause
     ↓
Choose solution
     ↓
Measure again
     ↓
Repeat

Not:

Traffic grows
     ↓
Add every technology we know

That difference is where architecture starts becoming engineering.

The Real Goal

The goal isn't to build a one-million-user architecture on day one.

The goal is to build a system that:

• Works for today's requirements
• Is observable enough to tell you what is breaking
• Has clear boundaries where change is likely
• Can evolve when the next constraint appears

A three-box architecture that correctly serves your current workload is
better than a fifteen-service architecture whose complexity you don't
need.

Start simple.

Measure.

Find the bottleneck.

Solve that bottleneck.

Then repeat.

Don't design for one million users on day one. Design a system that
gives you a clear path to the next stage.

Up Next

We've spent this article scaling application infrastructure.

But eventually, almost every system-design discussion reaches another
question:

What database should we actually use?

SQL? NoSQL? Document? Key-value? Graph?

More importantly:

How do we choose without starting from technology hype?

Next → Part 2

How to Choose Between SQL, NoSQL, and Everything in Between

This is **Part 1* of System Design from Developer to Architect ---
a practical series about scalability, databases, APIs, distributed
systems, reliability, and the engineering decisions behind production
architecture.*

Previous: Part 0 --- How to Think About System Design Without Just
Drawing
Boxes