Classic Machine Learning Through the Eyes of an SRE — Part 6
DBSCAN was the first algorithm in my study set that optimizes nothing.
No loss function. Nothing minimized. Nothing trained.
It's a DEFINITION that gets computed. Declare what a cluster is: a core point has at least minPts samples within radius eps, counting itself. A cluster is built by connecting density-reachable core points and their neighboring border points. Then traverse the data and collect whatever satisfies the definition.
Three roles fall out of that.
Core points have enough neighbors. Border points sit inside a core point's radius but don't have enough neighbors of their own. Noise is everything else.
Hold onto those border points. They come back later.
If you've done backend work, you already know this algorithm by another name: connected components. BFS flood-fill over neighborhoods.
I stared at DBSCAN for a week before realizing I'd implemented its skeleton years ago for an entirely different problem. Half of ML is old friends with new vocabulary.
The bet it makes
K-Means bets your groups are round blobs and that you know how many.
DBSCAN refuses both. Clusters can have arbitrary shapes, and the number of clusters falls out of the density structure rather than being supplied upfront.
Its actual bet is sneakier: ONE density bar works for your whole dataset.
You set eps and minPts globally. If dense regions and sparse regions both carry real structure, say dense ticket noise from big accounts and sparse but meaningful patterns from small ones, a single density threshold can't serve both. Tighten it and sparse structure dissolves into noise. Loosen it and dense regions can fuse together.
This is a known limitation of DBSCAN. It works best when meaningful clusters have reasonably similar densities.
That's where HDBSCAN becomes interesting. Instead of forcing one global eps, it considers clustering across varying density levels and selects persistent structure. If your domain genuinely contains clusters at different densities, that's a much more natural fit than endlessly searching for one eps that serves everyone.
The other thing that quietly breaks DBSCAN is dimensionality.
It's a distance-based algorithm. As dimensionality grows, distance measures can become less informative because points tend to look increasingly similar in distance. At some point, eps stops representing the neighborhood you thought it represented.
That's a problem the next article in this series exists to address.
The noise verdict
DBSCAN can say something no algorithm so far could:
"This point belongs to nothing."
That's a perfectly legal answer. K-Means force-assigns every point to a cluster, however poor the fit. DBSCAN has an explicit noise category instead.
And for ops data, that's often the more honest shape. Most days genuinely are unremarkable. Some events genuinely don't belong to any known pattern.
But the noise bucket gave me one of my favorite realizations from this algorithm: today's noise can become tomorrow's cluster.
A new failure pattern may start as a handful of isolated events before enough similar events appear to form a dense region. Treat noise as garbage and you may throw away early signals. Treat it as a watchlist and DBSCAN becomes much more interesting operationally.
When there's no optimizer, YOU are the optimizer
Here's the pattern that started crystallizing with this algorithm: the less the algorithm optimizes, the more the human setup becomes the intelligence.
There's no loss function standing between your parameter choices and the output.
eps. minPts. The feature space. The distance metric.
Those choices define what "dense" means.
My first-pass mistake was writing: "The structure is free. The density is up to the model to decide."
Exactly backwards. I set the density bar. The model enforces my definition.
That is the part worth remembering.
Deterministic failure
DBSCAN is deterministic for the same data, parameters, and input order.
But that does not mean the result is independent of input order.
Core-point membership is stable, but a non-core border point can be reachable from core points belonging to different clusters. In that case, its assigned cluster can depend on which cluster is encountered first.
So reorder your rows and a handful of border points can change cluster. Usually not the core structure. But potentially enough to matter if something downstream makes decisions from individual cluster membership.
And then there's the more dangerous lesson: reproducibility can masquerade as reliability.
With K-Means, instability across runs can at least tell you something is sensitive. DBSCAN can give you the exact same wrong answer every time. You can't monitor that with output variance because there may be none.
You need input-drift monitoring and periodic review of what the algorithm is actually putting into noise. Process guardrails, not statistical ones.
What I'd tell my ops team
Use DBSCAN when shape-freedom and a noise verdict match the domain: signature patterns in tickets, timesheet leakage, delivery-risk patterns that aren't blobs.
Budget your real effort for the parameters. Use a k-distance plot to help choose eps, and sanity-check minPts against what "pattern" actually means in business terms.
Then review the noise bucket regularly.
And write down WHY the parameters are what they are. When there's no optimizer, the parameter rationale is part of the model documentation.
Production takeaway
With no optimizer, your parameters ARE the model.
Document why eps and minPts are what they are. Use a k-distance plot to guide eps selection. Review the noise bucket regularly.
And monitor input drift rather than expecting output variance to warn you. This algorithm can repeat its wrong answer perfectly.
Common interview mistake
Saying "DBSCAN figures out the density for you." It doesn't. You define the density threshold through eps and minPts. DBSCAN discovers the clusters that satisfy that definition, and it can also identify noise.
Second: calling DBSCAN fully deterministic without qualification. For the same data in the same order, yes. But border-point assignments can depend on data ordering when a border point is reachable from multiple clusters.
Third, and the one that matters most in production: treating deterministic output as evidence of correctness.
Where I'd use this in a real production system
Incident and ticket pattern discovery where shapes are irregular, timesheet or usage leakage detection, and any case where "belongs to nothing" is a legitimate and useful answer.
Classic Machine Learning Through the Eyes of an SRE — Part 6. What ML algorithms bet about your world, through a production and ops lens. Previous: K-Means. Next: PCA, the algorithm that deletes your quietest signals first.
