Advanced RAG Architecture for AI Agents: Four Stages to Own

Short answer: Advanced RAG architecture for AI agents is not a cleverer retriever. It is four
stages somebody owns end to end: an index you can rebuild from vectors plus payloads,
deduplication that runs before embedding and again at retrieval, context compression that reports
the ratio it actually achieved so you can budget it, and a search path with a filter, a score floor
and an optional reranker. An agent loop is what makes all four decisive - a single question
retrieves once, an agent retrieves on every step and pays for all of it - and the protocol layer the
agent speaks (MCP for tools, A2A between agents) decides how those stages reach a running loop.

Key takeaways

• One hub, one client, one explicit dimension. VectorStoreHub creates the collection on demand with dim=768 and cosine distance, and its write path returns the number of points it stored - the value you want in a log line when an ingest job half-fails.
• Deduplicate twice, for two different reasons. Exact duplicates are removed before the ANN index is fitted, because they cost index space and can never be returned; near-duplicates are removed at retrieval time with a similarity threshold, which is the reason a context window fills with the same paragraph three times.
• Compression is a budgeting primitive, not a trick. compress_prompt_llmlingua2 returns origin_tokens, compressed_tokens, ratio, rate and an estimated saving, so a compression decision can be measured instead of argued about.
• Chunk on tokens, at sentence boundaries. __chunk_context reserves two tokens for the classifier's [CLS]/[SEP], then searches backwards from the window end for a stop token, which is why a compressed prompt still reads as sentences.
• Retrieval needs a floor and an escape hatch. Memory.search refuses a filter with no entity scope, applies a score threshold, and reranks only when a reranker exists - falling back to the vector ordering if reranking throws.
• Do this next: pick one of the four stages, measure it on your own corpus this week, and write the number down. Stage-by-stage measurement beats a retrieval rewrite, and the number is what tells you which stage is actually costing you.

The short version for whoever signs off on the architecture

Retrieval-augmented generation has been the default way to ground a model in private content since
the original RAG paper in 2020 (Lewis et al.), and the
dense-retriever recipe - embed documents, embed the query, take the nearest neighbours
(Karpukhin et al.) - has not changed much since. What changed is
the workload. A single question used to retrieve once; an agent retrieves on every step,
accumulates context across steps, and pays for all of it. That shift is
why retrieval became a loop rather than a lookup, and it is the
reason a pipeline and a loop need different decisions even when
they share an index.

Four unglamorous stages become decisive at that point. An index that cannot be rebuilt is a
liability the first time you change embedding models. Duplicates that are never filtered turn a
10,000-token context into 3,000 useful tokens at full price. Compression without a reported ratio
cannot be budgeted, and a retriever without a score floor happily returns the least relevant
document in the store, because some document is always nearest.

The code below is the shipped implementation of those stages in SmartGate, an MCP-native algorithm
gateway for token control, traffic shaping, and agent audit. Each block is quoted file by file with
its source lines rather than summarised, because the details - the default dimension, the distance
metric, where the threshold applies, which failures fall back instead of raising - are the
architecture.

The four stages and the number each one reports

Every stage below has a number that proves whether it worked. If a stage cannot print one, it is a
setting rather than a stage.

The same pipeline is drawn box by box in the rag architecture diagram,
which is the version to hand somebody who has to see the seams before agreeing to the ownership
split.

The model context protocol connection: one POST route, mounted last

An agent reaches a retrieval stack through a transport, and the transport is a design decision
rather than a detail. The gateway exposes the protocol over Streamable HTTP on one path, and the
route is mounted after the compatibility patches are applied, not before.
If the words are the obstacle rather than the route, start with
MCP naming and abbreviations - the
acronym has two expansions in circulation and only one of them is the protocol.

# backend/smartgate/api/mcp.py — source lines 399–406 (mount_mcp_routes)
def mount_mcp_routes(app: FastAPI) -> None:
    """Expose POST /mcp (Streamable HTTP, stateless)."""
    apply_mcp_session_compat()

    streamable_app = mcp.streamable_http_app()
    streamable_app.router.lifespan_context = _noop_starlette_lifespan(streamable_app)
    app.mount("/mcp", streamable_app)
    logger.info("MCP Streamable HTTP at POST /mcp")

Three things to copy. POST /mcp with Streamable HTTP is stateless: the deployment does not need
session affinity, so a request can land on any worker. The compatibility layer runs inside
mount_mcp_routes - if the patch were applied lazily, the first request after a cold start would
take the unpatched path. And the mounted sub-application's lifespan is replaced with a no-op,
because the parent application already owns startup and shutdown; a nested lifespan that also
initialises clients is how you get two vector-store connections per worker. The log line is the
part that matters in production: when an agent says it cannot see the tools, "MCP Streamable HTTP
at POST /mcp" in the startup log tells you the route exists and the client is wrong.

Stateless MCP sessions: what the protocol compatibility layer patches

The protocol's session semantics assume a long-lived connection; a stateless HTTP deployment does
not have one. That gap is closed with an idempotent patch applied once per process.

# backend/smartgate/api/mcp_session_compat.py — source lines 89–103 (apply_mcp_session_compat)
def apply_mcp_session_compat() -> None:
    """Idempotent patches applied before mounting MCP SSE."""
    global _PATCHED
    if _PATCHED:
        return

    ServerSession._received_request = _compat_received_request  # type: ignore[method-assign]
    ServerSession._received_notification = _compat_received_notification  # type: ignore[method-assign]

    if not hasattr(_stateless_server_run, "_orig"):
        _stateless_server_run._orig = lowlevel_server.Server.run  # type: ignore[attr-defined]
        lowlevel_server.Server.run = _stateless_server_run  # type: ignore[method-assign]

    _PATCHED = True
    logger.info("MCP session compat enabled (stateless SSE + relaxed init gate)")

This is the most uncomfortable code in the page, and it is worth reading as a warning rather than a
pattern: it monkey-patches the vendor SDK's session object and the low-level server's run loop.
What makes it survivable is the guard on top - a module-level _PATCHED flag and a hasattr check
before wrapping Server.run, so importing the module twice or mounting it in two test fixtures does
not double-wrap anything. The _orig attribute it stores is the rollback: the original run stays
reachable, which is the difference between a patch and a fork. If you write something like this,
keep those two properties - idempotent and reversible - and keep it in one file, because the
upgrade that breaks it will break it silently.

The a2a protocol boundary: forwarding a peer agent's headers, not its identity

When one agent calls another, the receiving side needs a small, deliberate subset of the incoming
request - which means an allow-list, never a header spread.

# lib/connect/mcp-proxy.ts — source lines 14–37 (forwardMcpRequestHeaders)
function forwardMcpRequestHeaders(incoming: Headers): Headers {
  const out = new Headers();
  const auth = incoming.get("Authorization");
  if (auth) out.set("Authorization", auth);
  const contentType = incoming.get("Content-Type");
  if (contentType) out.set("Content-Type", contentType);
  out.set("Accept", "application/json");

  const platform =
    incoming.get("X-SmartGate-Agent-Platform") ??
    incoming.get("x-smartgate-agent-platform");
  if (platform) {
    out.set("X-SmartGate-Agent-Platform", platform);
  } else {
    out.set("X-SmartGate-Agent-Platform", "cursor");
  }

  const trace =
    incoming.get("X-SmartGate-Trace-Id") ??
    incoming.get("x-smartgate-trace-id");
  if (trace) out.set("X-SmartGate-Trace-Id", trace);

  return out;
}

The function builds a fresh Headers object and copies four things: the authorization header, the
content type, the agent platform, and the trace id. Everything else the client sent is dropped,
which is the point - hop-by-hop headers, cookies and client-specific headers have no meaning on the
upstream leg, and forwarding them is how a proxy leaks more than it intends. Two details are worth
copying verbatim. Accept is forced to application/json because MCP clients often advertise
text/event-stream even when they are about to read a single JSON reply, and an upstream server
that honours the advertisement will stream a response the caller cannot parse. And the platform
header falls back to a concrete default rather than null, because downstream routing rules keyed on
that header behave better when every request has one.

The MCP tools an agent actually calls: two context primitives, one contract

The four stages are useful to an agent only if it can call them, so they are registered as tools
with the same names the HTTP API uses. The two context primitives are the ones this page's
architecture depends on.

# backend/smartgate/api/mcp.py — source lines 150–196 (register_mcp_tools)
    @server.tool(
        name="smart_context_gate",
        description=TOOL_DESCRIPTIONS["smart_context_gate"],
        annotations=tool_annotations("smart_context_gate"),
    )
    async def smart_context_gate(
        text: str = Field(description="Long text to compress before the host LLM call."),
        ratio: float = Field(
            default=0.5,
            description="Target compression ratio (e.g. 0.3–0.7).",
        ),
        purpose: str | None = Field(
            default=None,
            description="Optional goal to pre-filter paragraphs (step intent, user query).",
        ),
    ) -> str:
        _, registry = _app_state()
        module = registry.get("context_gate")
        ctx = _tool_ctx()
        return await _run_with_audit(
            "compress",
            ctx,
            module.process(ctx, text=text, ratio=ratio, purpose=purpose),
            {"ratio": ratio, "purpose": purpose},
        )

    @server.tool(
        name="smart_dedup",
        description=TOOL_DESCRIPTIONS["smart_dedup"],
        annotations=tool_annotations("smart_dedup"),
    )
    async def smart_dedup(
        texts: list[str] = Field(description="List of text passages to deduplicate."),
        threshold: float = Field(
            default=0.9,
            description="Similarity threshold (0.0–1.0); higher keeps fewer duplicates.",
        ),
    ) -> str:
        _, registry = _app_state()
        module = registry.get("dedup")
        ctx = _tool_ctx()
        return await _run_with_audit(
            "dedup",
            ctx,
            module.process(ctx, texts=texts, threshold=threshold),
            {"threshold": threshold},
        )

Read the field definitions rather than the plumbing, because that is the contract a model sees. Both
tools take the number the gateway will be judged on: ratio defaults to 0.5 for compression,
threshold defaults to 0.9 for deduplication, and purpose on the compression tool is the optional
string that lets a caller pre-filter paragraphs against the step's intent before any tokens are
spent. Each tool resolves its module from the registry and returns through the same audit wrapper as
every other tool, which is why a compressed prompt and an uncompressed one are costed the same way
in the audit trail. The registry indirection matters too: the tool and the HTTP endpoint call the
same module object, so a threshold changed in one place is changed in both.

When the alternative on the table is a hosted search tool rather than an index you own, the
trade-off is not about quality but about who metered the call:
agentic search vs RAG is the comparison to read first.

Agentic RAG retrieval: top-k with filters, not a full scan

Retrieval is deliberately thin. A query vector, a limit, and a filter that goes to the store.

# backend/smartgate/core/resources.py — source lines 141–149 (search_vectors)
async def search_vectors(self, collection: str, query_vector, top_k: int = 10, filters=None):
        from qdrant_client.models import Filter as QFilter
        result = await self._client.search(
            collection_name=collection,
            query_vector=query_vector,
            limit=top_k,
            query_filter=filters,
        )
        return result

Two properties are worth keeping when you write your own. The limit is a parameter rather than a
constant, because the right top_k depends on what follows it: retrieve 50 and rerank to 8 if you
have a cross-encoder, retrieve 8 if you do not, and do not pay for 50 embeddings of latency you will
discard. And the filter is expressed in the store's own query-filter type, which keeps pre-filtering
inside the vector search instead of in a Python loop after it - the difference between searching
your tenant's vectors and searching everything and then deleting what you should never have read.
The function returns the store's own result objects rather than re-wrapping them, so nothing in the
call path can silently change the score semantics a threshold later compares against.

How much of this stage is settled practice and how much is still an open question in the literature
is what the agentic rag survey sorts out, paper by paper.

MCP vs API transports: the same result, two error shapes

The same tool result has to be presented two ways, and the difference is how failure is expressed.

# backend/smartgate/core/models.py — source lines 38–46 (smartgate_http_response_from_result)
def smartgate_http_response_from_result(result, *, status_code: int = 422):
    """Map ToolResult → API envelope; failed tools become HTTP errors (REST clients)."""
    from fastapi import HTTPException

    response = smartgate_response_from_result(result)
    if not response.success:
        detail = response.error or {"code": "ERROR", "message": "request failed"}
        raise HTTPException(status_code=status_code, detail=detail)
    return response

An MCP tool call reports a failed tool as content - the model is expected to read the error and
decide what to do next. A REST client has no such convention: it reads a status code. This function
is the seam, and its default is the interesting part: a tool-level failure becomes HTTP 422 rather
than 200-with-an-error-body or a bare 500, because the request was well-formed and the tool refused
it, which is exactly what 422 means. Keeping the mapping in one function means "what a failure looks
like" is defined once, and the detail payload carries the tool's own error object rather than a
stringified exception, so the client can branch on a code instead of parsing prose.

RAG architecture starts with one owner for the vector index

The first decision in a RAG architecture is who owns the vector-store client. If every module talks
to the driver directly, changing the store is a migration project rather than a deployment.

# backend/smartgate/core/resources.py — source lines 97–105 (VectorStoreHub)
class VectorStoreHub:
    """统一向量数据库接入 — Qdrant 默认。"""

    def __init__(self):
        self._client = None
        self._url: str = ""

    async def initialize(self, config) -> None:
        vs_cfg = config.get("vector_store", {}) if hasattr(config, 'get') else config

# backend/smartgate/core/resources.py — source lines 107–124 (VectorStoreHub)
        from qdrant_client import AsyncQdrantClient
        self._client = AsyncQdrantClient(url=self._url)

    @property
    def client(self):
        return self._client

    async def ensure_collection(self, name: str, dim: int = 768) -> None:
        from qdrant_client.models import Distance, VectorParams
        collections = await self._client.get_collections()
        existing = [c.name for c in collections.collections]
        if name not in existing:
            await self._client.create_collection(
                collection_name=name,
                vectors_config=VectorParams(size=dim, distance=Distance.COSINE),
            )
            logger.info(f"Created Qdrant collection: {name} (dim={dim})")

Three things to copy from this class. The client is created in initialize, not at import time, so
the URL comes from configuration and a process that never retrieves never opens a connection. The
collection is created on demand by ensure_collection, with an explicit dimension and
Distance.COSINE, so a dimension mismatch fails at write time instead of at query time - the
expensive failure mode, because it surfaces as empty results rather than as an error. And
store_vectors and search_vectors are methods on the same object as the client, which is what
makes "one hub" enforceable rather than aspirational. One line sits between the two quoted ranges on
purpose: the shipped default for the store URL names a local development endpoint, and a published
page should not carry one. The URL is configuration, which is the point of the line, and the exact
ranges are in the provenance table.

The decisions that surround this ownership question - where the retrieval boundary sits, what may
cross it, and who is allowed to call it - are laid out in
what is rag architecture.

Agent memory search: entity scope, floor, optional rerank

Long-running agents also retrieve over their own history, and that path has a requirement a document
retriever does not have: a tenant scope that cannot be omitted.

# backend/smartgate/modules/memory/algorithm.py — source lines 1193–1237 (Memory.search)
        if not any(key in effective_filters for key in ("user_id", "agent_id", "run_id")):
            raise ValueError(
                "filters must contain at least one of: user_id, agent_id, run_id. "
                "Example: filters={'user_id': 'u1'}"
            )

        limit = top_k

        # Apply enhanced metadata filtering if advanced operators are detected
        if self._has_advanced_operators(effective_filters):
            processed_filters = self._process_metadata_filters(effective_filters)
            # Remove logical/operator keys that have been reprocessed
            for logical_key in ("AND", "OR", "NOT"):
                effective_filters.pop(logical_key, None)
            for fk in list(effective_filters.keys()):
                if fk not in ("AND", "OR", "NOT", "user_id", "agent_id", "run_id") and isinstance(effective_filters.get(fk), dict):
                    effective_filters.pop(fk, None)
            effective_filters.update(processed_filters)

        keys, encoded_ids = process_telemetry_filters(effective_filters)
        capture_event(
            "mem0.search",
            self,
            {
                "limit": limit,
                "version": self.api_version,
                "keys": keys,
                "encoded_ids": encoded_ids,
                "sync_type": "sync",
                "threshold": threshold,
                "advanced_filters": bool(filters and self._has_advanced_operators(filters)),
            },
        )

        original_memories = self._search_vector_store(query, effective_filters, limit, threshold)

        # Apply reranking if enabled and reranker is available
        if rerank and self.reranker and original_memories:
            try:
                reranked_memories = self.reranker.rerank(query, original_memories, limit)
                original_memories = reranked_memories
            except Exception as e:
                logger.warning(f"Reranking failed, using original results: {e}")

        return {"results": original_memories}

Read the validation before the retrieval. A filter must carry at least one of user_id, agent_id
or run_id, and the function raises rather than searching unscoped - a memory search without an
entity scope is a cross-tenant leak waiting for its first bug report. Entity ids are trimmed and
validated on the way in; advanced operators are translated into the store's filter language and the
logical keys are removed from the effective filter afterwards, so the query the store receives has
one shape. Then the search runs with a deliberately low floor of 0.1, telemetry records the limit,
the version, the filter keys and the threshold, and reranking is applied only if a reranker exists -
wrapped so that a reranker outage degrades to the vector ordering instead of failing the request.
That fail-open choice is the right default for an agent loop, and it is also a risk worth naming:
a persistently broken reranker looks like a working search path.

Prompt compression with a reported ratio you can budget

Compression turns a context-window problem into a cost problem, and the value is in the reporting.

# backend/smartgate/modules/context_gate/algorithm.py — source lines 936–973 (PromptCompressor.compress_prompt_llmlingua2)
        if target_token > 0:
            rate = min(target_token / n_original_token, 1.0)

        if use_token_level_filter:
            compressed_context, word_list, word_label_list = self.__compress(
                context_chunked,
                reduce_rate=max(0, 1 - rate),
                token_to_word=token_to_word,
                force_tokens=force_tokens,
                token_map=token_map,
                force_reserve_digit=force_reserve_digit,
                drop_consecutive=drop_consecutive,
            )
        else:
            compressed_context, word_list, word_label_list = self.__compress(
                context_chunked,
                reduce_rate=0,
                token_to_word=token_to_word,
                force_tokens=force_tokens,
                token_map=token_map,
                force_reserve_digit=force_reserve_digit,
                drop_consecutive=drop_consecutive,
            )

        n_compressed_token = 0
        for c in compressed_context:
            n_compressed_token += self.get_token_length(c, use_oai_tokenizer=True)
        saving = (n_original_token - n_compressed_token) * 0.06 / 1000
        ratio = 1 if n_compressed_token == 0 else n_original_token / n_compressed_token
        res = {
            "compressed_prompt": "\n\n".join(compressed_context),
            "compressed_prompt_list": compressed_context,
            "origin_tokens": n_original_token,
            "compressed_tokens": n_compressed_token,
            "ratio": f"{ratio:.1f}x",
            "rate": f"{1 / ratio * 100:.1f}%",
            "saving": f", Saving ${saving:.1f} in GPT-4.",
        }

The window shown is the token-level path: a target_token converts into a rate, the compressor runs
with reduce_rate=max(0, 1 - rate), and the return value carries origin_tokens,
compressed_tokens, ratio as a multiplier, rate as a percentage and an estimated saving. That
last field is why this belongs in a gateway rather than in a helper library: the same call that
compresses the prompt states what the compression was worth. Two defaults are worth knowing before
you copy the signature - chunk_end_tokens defaults to a period and a newline, so chunks end on
sentence boundaries before any compression happens, and force_tokens exists so identifiers,
numbers or a schema fragment survive the token-level filter intact.

Context window management: cut on tokens, at a stop token

Compression over a window that was split mid-sentence returns fragments; this is the split.

# backend/smartgate/modules/context_gate/algorithm.py — source lines 2250–2277 (PromptCompressor.__chunk_context)
def __chunk_context(self, origin_text, chunk_end_tokens):
        # Leave 2 tokens for CLS and SEP in downstream TokenClfDataset.
        max_content = self.max_seq_len - 2
        origin_list = []

        with warnings.catch_warnings():
            warnings.simplefilter("ignore")
            origin_tokens = self.tokenizer.tokenize(origin_text)

        n = len(origin_tokens)
        st = 0
        while st < n:
            if st + max_content >= n:
                chunk = self.tokenizer.convert_tokens_to_string(origin_tokens[st:n])
                origin_list.append(chunk)
                break

            end = st + max_content
            for j in range(1, end - st + 1):
                if origin_tokens[end - j] in chunk_end_tokens:
                    end = end - j + 1
                    break
            end = min(end, st + max_content)
            chunk = self.tokenizer.convert_tokens_to_string(origin_tokens[st:end])
            origin_list.append(chunk)
            st = end

        return origin_list

Two details decide quality. The window keeps two tokens of headroom (max_seq_len - 2) for the
classifier's special tokens, so a chunk that fills the model exactly does not get truncated later by
the dataset that consumes it. And the cut point is searched backwards from the window end for a stop
token - a period or a newline - and then clamped with min(end, st + max_content), so a chunk can
never exceed the maximum even when no stop token is found. Chunking by characters is easier and
produces the artefacts you have seen in bad RAG output: half a sentence at the head of every chunk,
which is half a sentence of embedding signal thrown away, and a retrieval score computed on a
fragment that never occurred in the source.

Semantic dedup at retrieval: threshold, selected, filtered

Deduplication at retrieval time is where context windows are won, and it is a decision with two
sides.

# backend/smartgate/modules/dedup/algorithm.py — source lines 168–213 (deduplicate)
        dict_records = self._validate_if_strings(records)

        # Remove exact duplicates before embedding
        dict_records, exact_duplicates = remove_exact_duplicates(
            records=dict_records, columns=self.columns, reference_records=self.index.items
        )
        duplicate_records = []
        for record, duplicates in exact_duplicates:
            duplicated_with_score = add_scores_to_records(duplicates)
            duplicate_record = DuplicateRecord(record=record, duplicates=duplicated_with_score, exact=True)
            duplicate_records.append(duplicate_record)

        # If no records are left after removing exact duplicates, return early
        if not dict_records:
            return DeduplicationResult(
                selected=[], filtered=duplicate_records, threshold=threshold, columns=self.columns
            )

        # Compute embeddings for the new records
        embeddings = featurize(records=dict_records, columns=self.columns, model=self.model)
        # Query the fitted index
        results = self.index.query_threshold(embeddings, threshold=threshold)

        deduplicated_records = []
        for record, similar_items in zip(dict_records, results):
            if not similar_items:
                # No duplicates found, keep this record
                deduplicated_records.append(record)
            else:
                duplicate_records.append(
                    DuplicateRecord(
                        record=record,
                        duplicates=[(item, score) for item, score in similar_items],
                        exact=False,
                    )
                )

        result = DeduplicationResult(
            selected=deduplicated_records, filtered=duplicate_records, threshold=threshold, columns=self.columns
        )

        if self._was_string:
            # Convert records back to strings if the records were originally strings
            return map_deduplication_result_to_strings(result, columns=self.columns)

        return result

Four behaviours are worth naming. Exact duplicates are removed before embedding, so the expensive
step never sees them. Surviving records are embedded and queried against the fitted index with a
similarity threshold (0.9 by default), and anything above it is filtered out as a near-duplicate.
The result keeps both sides of the decision: selected for what survives and filtered with the
matching items and their scores for what did not - keeping the rejected set is what makes the
threshold auditable instead of a magic number. And when the input was a list of strings the result
is mapped back to strings, so a caller never has to know about the internal record shape. The order
matters more than the code: doing this after the index is fitted means paying index cost for
passages you will never return.

Agentic RAG architecture: the index is data plus a backend

The seam that makes the architecture portable is that an index is built from vectors you already
own, with an implementation chosen by name.

# backend/smartgate/modules/dedup/index.py — source lines 31–48 (from_vectors_and_items)
@classmethod
    def from_vectors_and_items(
        cls, vectors: np.ndarray, items: list[DictItem], backend_type: Backend | str, **kwargs: Any
    ) -> Index:
        """
        Load the index from vectors and items.

        :param vectors: The vectors of the items.
        :param items: The items in the index.
        :param backend_type: The type of backend to use.
        :param **kwargs: Additional arguments to pass to the backend.
        :return: The index.
        """
        backend_class = get_backend_class(backend_type)
        arguments = backend_class.argument_class(**kwargs)
        backend = backend_class.from_vectors(vectors, **arguments.dict())

        return cls(vectors, items, backend)

This is the difference between a vector database and an index. The store is durable and shared; the
index is a fitted view derived from stored vectors and payloads, so it can be rebuilt at any time
without re-embedding the corpus and without touching the source of truth. The backend is chosen by
name and constructed from its own argument class, which is what makes swapping an in-process ANN
library for another implementation a parameter rather than a rewrite - and what makes an embedding
model change a scheduled rebuild rather than a migration project. Keep the two deliberately
separate: the moment the store's client leaks into the dedup path, the index stops being
disposable, and disposability is the property you need most.

Whether a system is best described as RAG with a loop attached or as an agentic system that happens
to retrieve is more than a labelling question: what you instrument, what you can bound and what you
can bill follow from it, which is the argument in
rag vs agentic ai.

How SmartGate compares

The honest framing is that these primitives are not a retriever. You still choose your vector store,
your embedding model and your corpus. What the gateway adds is the stage list above, callable by an
agent through the same MCP tools reference a host already exposes;
the MCP specification walkthrough is the place to check
how a host surfaces them without extra wiring.

The differentiation is the billing shape rather than the feature list: pay for the platform, share
only when you save - and the compression ratio this article measures is the same number the pricing
model reads. Compare it against your own token bill on the
pricing page.

How to get started

1. Own the index. Wrap your store in one client, create the collection with an explicit dimension and cosine distance, and write with stable ids so re-ingesting a document is an upsert.
2. Deduplicate twice. Exact duplicates before the index is fitted against a reference set; near-duplicates with a threshold at retrieval, keeping the filtered set for audits.
3. Compress with a report. Use a compressor that returns original tokens, compressed tokens and a ratio, and keep identifiers on a force list.
4. Chunk on tokens at sentence boundaries. Leave headroom for the model's special tokens and cut backwards to a stop token.
5. Retrieve with a floor. Scope filters by entity, set a threshold low enough to be useful and high enough to exclude noise, and make reranking optional and fail-open.
6. Audit the threshold against the rejected pairs after every corpus change, before someone else notices a missing answer.

Start on the free tier - 2M tokens a month, all seven smart_* tools, no card:
start free. The compression and dedup
primitives are documented alongside the rest of the toolset in the
SmartGate docs, and the
contact form is the path for a scoped deployment with your
own stores.

FAQ

Do I need a reranker if I already have a good embedding model?
No, and you should not assume one helps. Reranking pays when you retrieve a wide set and cut to a
small one; at a top_k of 8 a cross-encoder is mostly added latency and cost. The design in
Memory.search - rerank optional, fail-open to vector order - is the right default because it lets
you measure the lift on your own queries before committing to it.

How do I choose the dimension for the collection?
Whatever the embedding model emits, and the value of making it explicit is that the mismatch fails
at write time. The default here is 768 with cosine distance, which is the right distance for
normalised text embeddings. If you switch to a model that does not normalise its vectors, change the
metric at collection creation time, because retrofitting it means re-ingesting the corpus.

What threshold should my deduplicator use?
Start at 0.9 and audit it against the pairs it rejected. The threshold is corpus dependent: a
boilerplate-heavy corpus tolerates tighter filtering than a corpus of short, distinct answers. What
matters is that you can name the content the threshold removes, which is why the deduplication
result keeps the filtered items and their scores.

Does compression hurt answer quality?
It can, which is why the compressor returns the ratio and the token counts rather than a single
string. Keep a force list for identifiers, digits and schema fragments, and evaluate compressed and
uncompressed answers side by side on a question set before you turn it on for real traffic.

Where does the agent memory search fit into this architecture?
It is the fourth stage applied to the agent's own history rather than to your documents. Scope every
search by user, agent or run, keep the score floor low, and record the limit and filter shape in
telemetry so a retrieval that quietly returns nothing is visible.

How does this reach an agent?
The same modules are exposed as MCP tools - context compression, deduplication, scoped memory
search, and the pipeline orchestrator - over Streamable HTTP, so the agent can budget its own
context instead of relying on the framework around it. The tool definitions are in the SmartGate
docs, and the protocol side is specified in the MCP specification.

Limitations and what this does not do

• Retrieval quality is corpus quality. Deduplication and compression make a good corpus cheaper; they make a bad corpus smaller. Nothing here fixes a document set that does not answer the question.
• Thresholds are corpus-specific defaults. The 0.9 deduplication threshold and the 0.1 memory floor are defaults from the shipped implementation, not recommendations for your data. Both need auditing on your own content.
• The compatibility shim patches a vendor SDK. apply_mcp_session_compat replaces session and server methods on the MCP library. It is idempotent and keeps the original methods reachable, but an upstream release that renames those internals can break it silently, so it is the first thing to re-check after an SDK upgrade.
• Fail-open reranking hides failures. A reranker that throws on every call leaves a search path that looks healthy and returns vector order. Measure ranking quality, not just latency.
• A quoted docstring is not in English. VectorStoreHub carries a Chinese docstring in the source. It is quoted as-is, because editing quoted code is not something this pipeline does, and rewriting it would mean the excerpt no longer matches the file it comes from.
• The snapshot is a point in time. Thresholds, model names and defaults come from the implementation quoted here as of September 2026.

Sources

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020): https://arxiv.org/abs/2005.11401
• Dense Passage Retrieval for Open-Domain Question Answering (Karpukhin et al., 2020): https://arxiv.org/abs/2004.04906
• Lost in the Middle: How Language Models Use Long Contexts (Liu et al., 2023): https://arxiv.org/abs/2307.03172
• LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression (Pan et al., 2024): https://arxiv.org/abs/2403.12968
• RAGAS: Automated Evaluation of Retrieval Augmented Generation (Es et al., 2023): https://arxiv.org/abs/2309.15217
• Model Context Protocol — server tools specification: https://modelcontextprotocol.io/specification/2026-07-28/server/tools
• Agent2Agent (A2A) protocol specification: https://a2a-protocol.org/latest/specification/
• Qdrant documentation — collections, distances and payload filtering: https://qdrant.tech/documentation/
• Sentence Transformers documentation — embedding models: https://www.sbert.net/
• model2vec — static embeddings for cheap similarity work: https://github.com/MinishLab/model2vec
• LLMLingua — prompt compression toolkit: https://github.com/microsoft/LLMLingua
• FAISS — in-process similarity search: https://github.com/facebookresearch/faiss
• mem0 documentation — memory layer behind the memory module: https://docs.mem0.ai/
• SmartGate — docs and pricing: https://smartgate.network/docs · https://smartgate.network/pricing

Method note

The code in this article is not transcribed. Each block was cut directly out of the slice body
returned by the SmartGate slice API and re-asserted byte-for-byte as a substring of that body before
publication, and the first line inside every fence records the file and the exact source lines.
Symbols were pinned by whole-name containment (rule A level 2) and confirmed by the service's
slot-proof endpoint before being written into the prose - 12 of the 12 planned sections pinned, no
abstentions. Three sections quote a window rather than a whole definition: the compression path
inside compress_prompt_llmlingua2, the retrieval body of Memory.search, and the two context tool
registrations inside register_mcp_tools. Two of them quote a range with a line missing from the
middle; in VectorStoreHub the missing line is the shipped default for the store URL, which names a
local development endpoint that a published page must not carry.

Slice provenance

Every fenced block above was cut from the slice body and re-asserted against it byte-for-byte before
publication. 12 of 12 sections pinned, 0 abstentions, 0 misses.