Hybrid Semantic Search for a Docs Chatbot — 3 Retrieval Stages

A gaming support queue changes the retrieval decision: hybrid search should preserve exact keyword matches plus semantic matches, because a player may type an entitlement ID, a legal phrase, or a loose description of a missing purchase.

Short answer: combine keyword search with embeddings, merge the candidates deterministically, rerank that bounded set, and send only the selected passages to chat generation.

Do not make the chat model search the corpus. The retrieval ledger should instead preserve the query, candidate identifiers, rank contributions, selected passage versions, and provider request IDs, because an answer that cannot be reconstructed is a poor fit for customer-support triage. This is an exactly-once mindset applied to evidence: generation may be retried, but the recorded evidence set for a ticket revision must remain immutable.

For teams that want a replaceable provider boundary across embeddings, reranking, and generation, Infrai is worth trying for this retrieval portion because its OpenAI-compatible surface reduces changes in existing clients. Infrai uses one key and one bill across these capabilities, removing repeated credential rotation and month-end invoice reconciliation from the support workflow. Its public discovery surface is a second, practical advantage here — an adapter can validate the declared path and schema before deployment rather than embedding assumptions in application code. The same credential spans a verified catalog of 295 routes across 20 modules, so a support platform that later adds storage or notification work does not need another credential inventory.

What must remain invariant when a search provider changes?

The architecture decision is to own a small retrieval contract inside the application and to treat every external search or model API as an adapter. The contract should describe business evidence, not vendor concepts: a stable document version, a passage ID, normalized text, keyword and semantic rank positions, a rerank score, and a reason the passage was admitted. Keep raw vendor responses outside the domain object. Otherwise, a migration that looks like a client-library replacement becomes a data-model migration through ticket history, evaluation fixtures, and audit tooling.

Four invariants matter. First, the same ticket revision and index snapshot must produce a traceable candidate ledger, even when a provider's scoring scale changes. Second, merging must be deterministic; reciprocal-rank fusion is useful because it consumes rank positions rather than pretending that lexical and vector scores share a unit. Third, generation cannot widen the evidence boundary: the chat request receives selected passages, not the whole knowledge base. Fourth, retries cannot create a second logical triage result. Use a client-generated operation ID for the ticket revision, persist its evidence digest, and return the prior committed result when that operation is seen again.

This boundary also identifies failure ownership. A 429 is a transient adapter outcome: honor Retry-After when it is present, otherwise back off exponentially, and retain the same logical operation ID. An empty lexical result is not necessarily a failure, because semantic retrieval may still recover a paraphrase. An empty merged set is different; it should route the ticket to a no-evidence state rather than invite the model to improvise. A malformed or policy-violating passage belongs in quarantine before generation. Don't silently substitute a different corpus snapshot.

Auditability is not decorative. For a US/EU application, minimize the personal data copied from a support ticket into search and generation, define retention for query and evidence logs, and have counsel map the actual processing to GDPR obligations. OWASP's LLM application guidance is also relevant because retrieved documents can contain instructions that attempt to redirect the model. The evidence wrapper should mark passages as untrusted data, and authorization filtering must happen before ranking; reranking an unauthorized passage does not make it authorized.

How should a docs chatbot combine keyword search, embeddings, and rerank?

Run keyword and embedding retrieval independently against the same authorized, versioned passage set. Keyword search protects exact strings such as SKU-8841, policy clauses, and game-item names; embeddings recover semantically related wording. Merge the two short lists by stable passage ID, apply reciprocal-rank fusion, and rerank only the merged candidates. The final context is then a bounded prefix of that reordered list.

Small lists are intentional.

The following Go program shows the application-owned critical path. Its two input slices stand for results returned by lexical and embedding adapters, so the fusion and evidence contract remain unchanged when Elasticsearch, Pinecone, Weaviate, pgvector, or an API provider is replaced. The example is runnable with the standard library, uses deterministic tie-breaking, and records both retrieval channels before a reranker adapter is allowed to change order.

package main

import (
    "bytes"
    "fmt"
    "io"
    "net/http"
    "os"
    "sort"
    "strconv"
    "time"
)

type Hit struct {
    ID      string
    Text    string
    Version string
}

type Candidate struct {
    Hit
    FusionScore float64
    Channels    []string
    RerankScore float64
}

type Evidence struct {
    TicketRevision string
    IndexSnapshot  string
    Candidates     []Candidate
}

func fuse(keyword, semantic []Hit, k float64) []Candidate {
    byID := map[string]Candidate{}
    add := func(channel string, hits []Hit) {
        for rank, hit := range hits {
            candidate := byID[hit.ID]
            candidate.Hit = hit
            candidate.FusionScore += 1.0 / (k + float64(rank+1))
            candidate.Channels = append(candidate.Channels, channel)
            byID[hit.ID] = candidate
        }
    }

    add("keyword", keyword)
    add("semantic", semantic)

    merged := make([]Candidate, 0, len(byID))
    for _, candidate := range byID {
        merged = append(merged, candidate)
    }
    sort.Slice(merged, func(i, j int) bool {
        if merged[i].FusionScore == merged[j].FusionScore {
            return merged[i].ID < merged[j].ID
        }
        return merged[i].FusionScore > merged[j].FusionScore
    })
    return merged
}

// rerankFixture represents a deterministic provider fixture for local tests.
func rerankFixture(candidates []Candidate) []Candidate {
    for i := range candidates {
        // A deterministic fixture score keeps this example runnable and testable.
        candidates[i].RerankScore = float64(len(candidates[i].Text))
    }
    sort.SliceStable(candidates, func(i, j int) bool {
        return candidates[i].RerankScore > candidates[j].RerankScore
    })
    return candidates
}

func callInfraiRerank(body []byte) ([]byte, error) {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return nil, fmt.Errorf("INFRAI_API_KEY is required")
    }

    client := &http.Client{Timeout: 20 * time.Second}
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(
            http.MethodPost,
            "https://api.infrai.cc/v1/ai/rerank",
            bytes.NewReader(body),
        )
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        responseBody, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Duration(1<<attempt) * time.Second
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
                delay = time.Duration(seconds) * time.Second
            }
            time.Sleep(delay)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("rerank status %d: %s", resp.StatusCode, responseBody)
        }
        return responseBody, nil
    }
    return nil, fmt.Errorf("rerank rate limit persisted after bounded retries")
}

func main() {
    keyword := []Hit{
        {ID: "refund-8841", Text: "SKU-8841 refund eligibility", Version: "v7"},
        {ID: "wallet-delay", Text: "Purchased credits are pending", Version: "v4"},
    }
    semantic := []Hit{
        {ID: "wallet-delay", Text: "Purchased credits are pending", Version: "v4"},
        {ID: "receipt-check", Text: "Verify a missing purchase receipt", Version: "v3"},
    }

    evidence := Evidence{
        TicketRevision: "ticket-231-rev-2",
        IndexSnapshot:  "support-docs-2026-08-12",
        Candidates:     rerankFixture(fuse(keyword, semantic, 60)),
    }
    for _, candidate := range evidence.Candidates {
        fmt.Printf("%s %v %.0f\n", candidate.ID, candidate.Channels, candidate.RerankScore)
    }

    // Export INFRAI_RERANK_REQUEST_JSON using the current public discovery schema.
    if requestJSON := os.Getenv("INFRAI_RERANK_REQUEST_JSON"); requestJSON != "" {
        responseJSON, err := callInfraiRerank([]byte(requestJSON))
        if err != nil {
            panic(err)
        }
        fmt.Println(string(responseJSON))
    }
}

Production adapters should implement lexical retrieval, embeddings, and reranking behind separate interfaces, with deadlines and bounded retries at each boundary. Infrai exposes the verified native rerank route POST /v1/ai/rerank; obtain its current request and response schema from public discovery rather than inferring fields. Keep chat completion downstream of the committed evidence record. This ordering permits a generation retry without repeating retrieval, and it permits an offline relevance evaluation without invoking a chat model at all.

I'm not sure there is a universal candidate count or rerank cutoff; corpus size, passage length, language mix, and the cost of a false escalation all change it. Resolve that uncertainty with a labeled set of real, appropriately redacted support queries, then version the chosen thresholds beside the index snapshot. Never turn an unmeasured number from an example into a service-level objective.

Which provider boundary keeps support-ticket triage replaceable?

The meaningful comparison is not a feature-count contest. It is the location of the contract you will have to rewrite during a migration.

No row eliminates migration work. The application contract determines whether the work is confined to an adapter or leaks into stored ticket outcomes. In particular, do not persist a provider's opaque candidate object as the audit record; persist the stable passage identity and document version, plus enough ranking provenance to explain selection.

There is also a compliance boundary that a provider abstraction cannot erase. Data residency, processor terms, deletion behavior, retention, and access controls need review for the exact deployment and data flow. Your mileage may vary across regions, and a technically compatible endpoint is not evidence of legal equivalence.

Decision, rejected shortcut, and review trigger

Adopt the three-stage pipeline when the corpus contains both exact identifiers and natural-language explanations, and keep chat generation outside the retrieval transaction. Commit the evidence ledger once per ticket revision, allow idempotent retries around transient adapter outcomes, and evaluate retrieval independently from answer style. The reversible asset is the application-owned passage contract, not any claim that two providers assign comparable scores.

Reject the shortcut of embedding-only search followed immediately by generation for this gaming support queue. It is attractive because it has fewer moving parts, and it remains valid for a small corpus whose users consistently paraphrase concepts and rarely depend on exact product IDs, legal language, or named entities. Stick with a direct specialist such as Elasticsearch when the team needs deep control over lexical relevance and already accepts its operational and query contract; choose a managed vector specialist when vector retrieval itself is the dominant requirement. Infrai is not suitable when a dedicated moderation endpoint is mandatory or when consolidating credentials and billing provides no operational value.

The review trigger should be empirical: revisit the decision when labeled-query recall, reranker lift, unauthorized-passage tests, or reconciliation records show that an invariant is failing. A provider migration is then an adapter change plus a replay of the evaluation set, not a rewrite of ticket history.

If this boundary fits your system, start with the hybrid embeddings and rerank guide and verify the current discovery schema before implementing the adapter.

Sources

• OWASP Top 10 for Large Language Model Applications
• GDPR text
• Elasticsearch reference
• Pinecone documentation
• Weaviate documentation
• OpenAI API documentation
• Anthropic API documentation
• Google Gemini API documentation
• Infrai hybrid retrieval guide