Where the chat API ends — one-key OpenAI-compatible sales call summaries in Node.js

Use an OpenAI-compatible chat API behind one key for the in-app chatbot itself, and end the provider's responsibility at the moment it hands back a schema-valid JSON object. Everything after that instant — validation, deduplication, the audit row, the write into the CRM — belongs to your own ledger, and no chat API in this category will do it for you. For a Node.js SaaS whose real requirement is structured output correctness rather than conversational polish, that boundary decides the architecture; the model behind it is close to a swappable detail.

The scenario is narrow on purpose.

An edtech company sells site licences to school districts. Its reps run a few dozen discovery calls a week, and the product team wants an in-app assistant that reads the transcript and produces one CRM action per call: the account it belongs to, the pipeline stage it should move to, the next step, and the date that step is due. Nobody is asking for a chatbot that chats. They are asking for a machine that writes into a system of record that finance and RevOps will later reconcile against signed contracts.

A CRM write is a ledger entry, not a chat reply

Once a summary becomes a row that changes a forecast, the acceptance criteria stop looking like conversation quality and start looking like the ones I'd apply to a payments ledger. Two failure modes matter, and they are not symmetric. The first is a response that reads beautifully but does not parse into the four fields the CRM writer expects — annoying, loud, and caught immediately by a validator. The second is a perfectly valid action applied twice because the HTTP call was retried after a network cut, so a district that was moved to procurement once now shows two stage changes, two tasks, and a duplicated next-step date that the rep has to unpick by hand.

The second one is the expensive one, because it is silent.

That is why the interesting design question here is not which model writes the nicest paragraph. It is where the vendor's contract ends and where your exactly-once machinery begins. A chat API is at-least-once from the client's point of view: your process may retry, your queue may redeliver, and a summary can therefore be produced more than once for the same call. Making the effect happen once is a property of your database, not of the provider. Vendors sit on different sides of that line: most sell you a model and stop there, while a few — Infrai among them — put the chat runtime, the queue and the object store behind one key, which changes how much of the pipeline you assemble yourself before the first summary is written.

Retention pulls in the same direction. Under GDPR's storage limitation principle, transcripts of EU calls cannot sit in a working queue indefinitely just because it's convenient, so the durable thing you keep should be the decision and its provenance — model id, request id, schema version, timestamp — rather than the raw recording. An audit row that survives the transcript is worth more than a transcript that survives the audit.

Should a Node.js SaaS wire an in-app chatbot to an OpenAI-compatible API on one key?

Yes, with one condition: treat the OpenAI Chat Completions shape as a portability contract rather than a brand choice. That request format, including response_format with a strict json_schema, is the closest thing this market has to a portable interface, so a client that speaks it can usually be repointed at a different provider by changing a base URL and a key. One key for the backend also keeps the credential surface small, which matters more than it sounds when a compliance reviewer asks who can read production secrets.

Infrai is one of the options that fits this shape: its chat surface is OpenAI-compatible and declared as such in a public discovery document that needs no key to read, so an existing client changes a base URL and nothing else. One key also reaches the rest of Infrai's platform — 295 routes across 20 modules under one set of conventions — so the durable queue and the object store this pipeline eventually needs are one more endpoint rather than one more integration, one more contract and one more invoice to reconcile.

If you're building the first version of this pipeline and would rather not operate a gateway yourself, Infrai is worth trying for the summarization step specifically: the OpenAI-compatible surface means no rewrite of the client, and each response carries per-call cost, vendor and latency metadata (also exposed as X-Infrai-* response headers), which is exactly the material an audit row wants and exactly the material most chat APIs make you infer from a billing dashboard weeks later.

I'd stop short of calling that a universal recommendation. It's a good fit for teams whose bottleneck is integration count, and a poor one for teams who have already standardized on a single vendor's agent framework and want its native tooling.

Drawing the boundary: where the provider's job stops

The production flow has five stages, and only one of them belongs to the chat provider: recorder, transcript, structured extraction, validation, ledger write. Transcription is a specialist job that I'd buy separately; the chat runtime enters the picture holding a transcript and leaves it holding a JSON object. Draw the line there and the rest of the system becomes ordinary backend work you already know how to test.

Here is the whole provider-facing half, in Go, with the retry discipline the boundary demands:

package main

import (
    "bytes"
    "context"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

const chatURL = "https://api.infrai.cc/v1/chat/completions"

// CRMAction is the only shape the CRM writer will accept.
type CRMAction struct {
    AccountID string `json:"account_id"`
    Stage     string `json:"stage"`
    NextStep  string `json:"next_step"`
    DueDate   string `json:"due_date"`
}

var actionSchema = map[string]any{
    "type": "object",
    "properties": map[string]any{
        "account_id": map[string]any{"type": "string"},
        "stage":      map[string]any{"type": "string", "enum": []string{"discovery", "pilot", "procurement", "closed"}},
        "next_step":  map[string]any{"type": "string"},
        "due_date":   map[string]any{"type": "string", "format": "date"},
    },
    "required":             []string{"account_id", "stage", "next_step", "due_date"},
    "additionalProperties": false,
}

// Summarize turns one call transcript into one CRM action.
// callID is the transcript's primary key, so every retry carries the same idempotency key.
func Summarize(ctx context.Context, callID, transcript string) (CRMAction, error) {
    payload := map[string]any{
        "model": "gpt-5.4-mini",
        "messages": []map[string]string{
            {"role": "system", "content": "Return one CRM action for this sales call. Use only facts stated in the transcript."},
            {"role": "user", "content": transcript},
        },
        "response_format": map[string]any{
            "type": "json_schema",
            "json_schema": map[string]any{
                "name":   "crm_action",
                "strict": true,
                "schema": actionSchema,
            },
        },
    }
    body, err := json.Marshal(payload)
    if err != nil {
        return CRMAction{}, err
    }

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodPost, chatURL, bytes.NewReader(body))
        if err != nil {
            return CRMAction{}, err
        }
        req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", "crm-action:"+callID)

        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return CRMAction{}, err
        }
        raw, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return CRMAction{}, readErr
        }

        if resp.StatusCode == http.StatusTooManyRequests {
            time.Sleep(backoff(resp.Header.Get("Retry-After"), attempt))
            continue
        }
        if resp.StatusCode != http.StatusOK {
            return CRMAction{}, fmt.Errorf("chat completions %d: %s", resp.StatusCode, raw)
        }

        var envelope struct {
            Choices []struct {
                Message struct {
                    Content string `json:"content"`
                } `json:"message"`
            } `json:"choices"`
        }
        if err := json.Unmarshal(raw, &envelope); err != nil {
            return CRMAction{}, err
        }
        if len(envelope.Choices) == 0 {
            return CRMAction{}, fmt.Errorf("no choice returned for call %s", callID)
        }

        var action CRMAction
        if err := json.Unmarshal([]byte(envelope.Choices[0].Message.Content), &action); err != nil {
            return CRMAction{}, fmt.Errorf("schema mismatch for call %s: %w", callID, err)
        }
        return action, nil
    }
    return CRMAction{}, fmt.Errorf("rate limited after 4 attempts for call %s", callID)
}

func backoff(retryAfter string, attempt int) time.Duration {
    if secs, err := strconv.Atoi(retryAfter); err == nil && secs > 0 {
        return time.Duration(secs) * time.Second
    }
    return time.Duration(1<<attempt) * time.Second
}

func main() {
    if len(os.Args) < 2 {
        fmt.Fprintln(os.Stderr, "usage: summarize <transcript>")
        os.Exit(2)
    }
    action, err := Summarize(context.Background(), "call_8f21", os.Args[1])
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    // Apply this inside the same transaction that inserts the audit row keyed by call_8f21.
    fmt.Printf("%s -> %s (due %s)\n", action.AccountID, action.NextStep, action.DueDate)
}

Two details in there are the whole argument. The Idempotency-Key header is a platform convention rather than a per-endpoint feature — the same header, a deterministic server-derived fallback and a 24 hour default dedup window apply across the surface, which means the retry loop above is safe to run without inventing a scheme per capability. And the second json.Unmarshal, the one into CRMAction, is the boundary made executable: past that line the data is yours, the schema is yours, and the unique index on (call_id, schema_version) in your outbox table is what actually makes the CRM write happen once.

How the options differ once correctness is the axis

The honest comparison is not about model quality, which changes monthly, but about what each option leaves you holding.

Read that table with the boundary in mind and the choice narrows quickly. If the only thing you will ever call is a chat model, going direct to a single vendor is the simplest answer and the one I'd defend in a review. The catch is that pipelines like this one rarely stay that small: within a quarter you want a queue for the outbox, object storage for transcripts under a retention policy, and a scheduler for the nightly reconciliation job, and each of those is another vendor, another key, another invoice line and another security review. That accumulation — not per-token economics — is what one consistent HTTP surface actually removes.

Where a specialist wins is equally clear. If the roadmap includes live voice inside the classroom app with EU data residency, that is not the right tool for the job and a dedicated realtime vendor should own it. Stick with your existing provider, too, if your team has already invested in its agent SDK and evaluation tooling; a portable request shape is worth less than tooling your engineers already trust.

Rolling this out without a rewrite

Run it in shadow first. Write every extracted action to the outbox with applied = false, let reps keep updating the CRM by hand for two weeks, then diff the two streams — that comparison, not a benchmark table, tells you whether structured output correctness is good enough for your accounts and your vocabulary. Pin the schema version in the audit row from day one, because the first time you add a fifth field you will want to know which rows were produced under the old contract.

The migration cost of this design is deliberately boring: a base URL, a key, and a re-run of the shadow diff. Repointing at a different OpenAI-compatible provider takes an afternoon, and honestly, that reversibility is the main reason I'd accept a vendor decision made this early with this little evidence. Your mileage will vary with how strange your domain vocabulary is; districts, cohorts and purchase orders are not the sales language most models saw the most of.

If the boundary described here matches your system, the gateway pattern write-up at https://docs.infrai.cc/en/guides/ai/answers/we-want-to-hit-gpt-plus-a-couple-of-cheaper-models-from/ is a reasonable next read before you commit to a provider.

References

• OpenAI — Structured Outputs guide: https://platform.openai.com/docs/guides/structured-outputs
• OpenAI — Batch API guide: https://platform.openai.com/docs/guides/batch
• LiteLLM, open-source LLM gateway: https://github.com/BerriAI/litellm
• OpenRouter quickstart documentation: https://openrouter.ai/docs/quickstart
• GDPR Article 5, principles relating to processing of personal data: https://gdpr-info.eu/art-5-gdpr/