"Add rate limiting" sounds like one task. It's actually three separate layers, each defending against a different failure mode, with different trade-offs and different amounts of trust you can place in them. I only understood the distinction properly once I had to pick a strategy for two systems solving different problems: LogicVisor (a public AI tool anyone can hit anonymously) and Titan (a payments platform where the cost of a bad actor is different from the cost of someone burning through free AI credits).
1. Client-side: UX, not security
Debouncing a search input, greying out a submit button after the first click, backing off exponentially after a 429. All of this makes an app feel considerate. None of it stops anyone. A malicious actor skips your JavaScript entirely and hits the endpoint directly with curl. Client-side limiting is worth doing (it saves you real traffic and prevents accidental double-submits), but it is not a security control. If it's the only thing standing between your API and abuse, you don't have rate limiting, you have a polite suggestion.
2. Server-side: the layer that actually protects you
This is where LogicVisor and Titan diverge, because they're not defending against the same thing.
LogicVisor: several checks before a single AI token gets spent
LogicVisor is public. Anyone gets 3 free code reviews with no signup, which means the abuse surface is wide open by design. The submission route runs a stack of checks before it ever calls Gemini or Groq, because every AI call costs real money:
// 1. Check if this exact code has already been reviewed by this model
const cachedReview = await getCachedAIReview(preferred_model.id + "-" + canonicalHash);
if (cachedReview) {
return NextResponse.json({ success: true, data: cachedReview }, { status: 201 });
}
// 2. Enforce the actual rate limit (IP + session based for anon users)
const rateLimitResult = await enforceAIRateLimit(user.id, request);
// 3. Slow down premium users intelligently instead of hard-blocking them
const throttleDelay = await getThrottleDelay(user.id);
if (throttleDelay > 0) {
await new Promise((resolve) => setTimeout(resolve, throttleDelay));
}
// 4. Under heavy load, degrade gracefully instead of rejecting outright
const shouldDegrade = await shouldGracefullyDegrade(user.id, "ai_request");
if (shouldDegrade) {
// return a basic, non-AI response instead of a 429
}
A few things worth naming separately, because they get lumped together under "rate limiting" but aren't the same mechanism:
- Abuse prevention: anonymous sessions are capped at 3 reviews, tracked by a UUID session (7-day expiry) plus a browser fingerprint (User-Agent + Accept-Language + Accept-Encoding). This is the classic server-side layer, IP and fingerprint based, closest to the "token bucket per identity" pattern.
- Cost control via deduplication: before rate limiting even runs, the code is canonicalized (AST-based) and hashed. If someone (or the same person twice) submits identical logic, they get the cached review instead of triggering a new paid AI call. This isn't rate limiting in the strict sense, it's closer to database-level duplicate detection, but it does the same job of protecting a scarce resource.
- Graceful degradation over hard rejection: instead of a flat 429 when usage spikes, authenticated users can get a stripped-down, non-AI response. It costs nothing and still gives the user something, rather than a wall.
None of this is a single algorithm out of a textbook. It's several cheap, layered checks, ordered so the most expensive resource (the AI call) is the last thing hit, not the first.
Titan: NestJS Throttler backed by Redis
Titan's rate limiting is intentionally boring by comparison, and that's the correct choice for what it is. It's @nestjs/throttler wired up as a global guard, with Redis swapped in as the storage backend instead of the default in-memory store:
ThrottlerModule.forRootAsync({
imports: [ConfigModule],
inject: [ConfigService],
useFactory: (configService: ConfigService) => ({
throttlers: [
{
ttl: 60000,
limit: 10,
},
],
storage: new ThrottlerStorageRedisService(
configService.get<string>('REDIS_URL'),
),
}),
}),
// registered alongside the JWT guard as a global APP_GUARD
{
provide: APP_GUARD,
useClass: ThrottlerGuard,
},
10 requests per 60 seconds, globally, enforced before a request even reaches a controller. No per-route tuning yet, no per-user tiers, just a flat ceiling applied everywhere via APP_GUARD.
The reason Redis matters here isn't exotic: an in-memory counter only knows about requests hitting that one process. The moment you run more than one instance behind a load balancer, in-memory rate limiting stops being true rate limiting, each instance is independently under-counting. Swapping the storage to ThrottlerStorageRedisService gives every instance a shared, consistent view of who's made how many requests, which is the actual requirement once you're not running a single box.
LogicVisor's layered, identity-aware approach makes sense for a system where the main threat is "someone is farming free AI reviews." Titan's flat, Redis-backed throttle makes sense for a system where the main threat is "someone is hammering an endpoint," and the priority is consistency across instances over nuance per user.
3. Database-level: the layer I'm deliberately not using yet
Postgres and Supabase both support rate limiting closer to the data layer (unlogged tables with triggers, PL/pgSQL token bucket functions, TTL-based counters in Mongo). I don't use this in either project, and I don't think I should yet. It's the right call for protecting specific high-value operations (financial writes, heavy analytical queries) where you need the database itself to be the source of truth and can't tolerate a race between the app layer and the DB. LogicVisor's canonical-hash cache lookup is adjacent to this idea (checking the data layer before doing expensive work), but it's a caching pattern, not a rate limiter, and I'd rather keep that distinction honest than dress it up as something it isn't.
What actually mattered, in order
- Client-side limiting is UX. Treat it as UX. It's not a line item in your security review.
- Server-side is where the real decision lives, and the right shape of it depends on what you're protecting: identity-based abuse (LogicVisor) versus flat request volume across a distributed system (Titan). They are not the same problem and don't need the same solution.
- The most expensive resource should be the last thing your code touches. LogicVisor checks a cache, then a rate limit, then a throttle delay, before it ever calls an LLM. Ordering is a rate limiting decision too, even when it's not framed as one.
- Database-level limiting is a tool for a specific job, not a default. I'm not reaching for it until there's an operation that actually needs that guarantee.
Cover photo by HsinKai Tai on Unsplash
