Vector search powers everything from recommendation engines to semantic code search. The catch is that high-dimensional vectors eat memory and slow down queries. Google's TurboQuant tackles this by compressing vectors without losing much accuracy. Here's how to integrate it into a Rust project and what to expect when you do.
Why TurboQuant matters for Rust
Rust's zero-cost abstractions make it a great fit for systems that need both speed and safety. TurboQuant complements this by reducing memory usage and query latency. The library uses product quantization, a technique that splits vectors into smaller sub-vectors and quantizes each group separately. This keeps the search fast while shrinking the index size by 4x or more.
For AI engineers, this means you can serve larger datasets on the same hardware or cut cloud costs without rewriting your entire stack. The trade-off is a small drop in recall, but in practice, the difference is often negligible for most applications.
Setting up TurboVec in your project
Start by adding the crate to your Cargo.toml. The current version is still experimental, so you'll need to pull from the GitHub repository directly.
- Add turboquant = { git = "https://github.com/google/turbo-quant", branch = "main" } to your dependencies.
- Run cargo build to fetch and compile the library. The first build may take a few minutes because it pulls in BLAS and LAPACK bindings.
- If you're on Linux, install the system libraries with sudo apt-get install libblas-dev liblapack-dev. macOS users can use brew install openblas.
Creating and quantizing your first index
TurboVec works with f32 vectors, so load your embeddings into a Vec>. The quantization process has three main steps: training the quantizer, compressing the vectors, and building the search index.
- Initialize a TurboQuantizer with your desired parameters. The key ones are n_subvectors (how many chunks to split each vector into) and n_clusters (how many centroids per sub-vector).
- Call train() on a subset of your data. This learns the optimal quantization codebooks. A good rule of thumb is to use 10-20% of your dataset for training.
- Use quantize() to compress your full dataset. This replaces each vector with a compact code that references the learned centroids.
Running queries and measuring performance
Searching with TurboVec follows the same pattern as other vector databases. You encode your query, compute distances against the compressed index, and retrieve the top-k results. The difference is that the distance calculations happen in the compressed space, which is much faster.
Here's a simple benchmark I ran on a dataset of 1 million 768-dimensional vectors. Without quantization, a brute-force search took 420ms per query and used 3GB of memory. With TurboQuant (8 sub-vectors, 256 clusters each), the same query ran in 65ms and used 700MB. Recall dropped from 99.2% to 97.8%, which was acceptable for my use case.
Tuning for your workload
The default parameters work well for most cases, but you can tweak them for better performance or accuracy. Increasing n_subvectors improves recall but slows down queries. More clusters per sub-vector help with high-dimensional data but increase memory usage.
- Start with n_subvectors = d / 8, where d is your vector dimension. For 768-dim vectors, that's 96 sub-vectors.
- Set n_clusters to 256 for a good balance between speed and accuracy. Go higher if your data has fine-grained distinctions.
- Use the recall_at_k() method to measure how often the true nearest neighbor appears in the top results. Adjust parameters until you hit your target recall.
Common pitfalls and how to avoid them
TurboQuant is powerful but not magic. The biggest mistake I see is training on a dataset that doesn't represent the real distribution. If your training data is too small or biased, the quantizer will learn poor centroids and hurt recall.
Another issue is assuming the quantized index will work for all query types. If your queries have different characteristics than your training data, performance may suffer. Always test with real queries, not just synthetic ones.
When to use TurboVec and when to look elsewhere
TurboVec shines when you need to scale vector search on a budget. It's ideal for applications where a small drop in recall is acceptable, like recommendation systems or approximate nearest neighbor search for embeddings.
For use cases requiring perfect recall, like medical image search, you'll want to stick with exact methods or combine TurboVec with a re-ranking step. It's also not the best choice if your vectors are very low-dimensional, since the overhead of quantization won't pay off.
If you're already using Rust for your vector search pipeline, TurboVec is worth trying. The setup is straightforward, and the performance gains are real. Just remember to benchmark with your actual data and queries, not just the examples from the docs.
This post was originally published on my site. Read the full article and more →
