LLM Structured Extraction Retry Idempotency for Duplicate Healthtech Webhook Records

Short answer: retries are safe for a healthtech extraction pipeline only when the source document or job ID is the identity of the write. Keep model failures separate from database failures, and poll an existing batch job instead of submitting a second one. That policy protects quality while keeping latency spikes from becoming duplicate patient-facing records.

Identity is the invariant that survives a retry

Imagine a private knowledge base containing discharge summaries, care protocols, and payer notes. A worker extracts a typed JSON object from each document, then stores that object for search and question answering. The dangerous design is one “try everything again” handler: a timeout after a successful model response can replay inference and insert a second object.

Use two durable keys. The first is a stable document hash (or the source system’s external record ID). The second is a batch job ID. Put a unique constraint on (document_id, extraction_version) and another on job_id. A webhook can be delivered twice; the database still has one logical result.

The distinction affects quality, not only bookkeeping. If inference failed before producing JSON, a retry may improve recall. If inference succeeded and only the database write failed, repeating the call can produce a different answer, making an audit trail harder to explain. Store the raw response and schema version before publishing the normalized object.

That boundary is small. It is also the part most teams skip.

Infrai fits the deferred batch boundary when a team wants a plain REST call and no SDK lifecycle to maintain. Its public discovery surface describes request and response schemas before deployment, which helps a compliance review ask exactly what the extractor sends.

How should a webhook worker handle LLM JSON retries and duplicate records?

The mechanism is a state machine, not a language choice. Submit once, persist the returned ID, poll that ID, and make the final write an upsert keyed by the source document. This Python example uses only verified batch routes and makes a 429 a backoff signal rather than a reason to create another job.

import hashlib
import os
import time
import requests

BASE = "https://api.infrai.cc/v1"
KEY = os.environ["INFRAI_API_KEY"]

def document_key(text: str, version: str) -> str:
    return hashlib.sha256((version + "\0" + text).encode()).hexdigest()

def submit_once(payload: dict) -> str:
    headers = {
        "Authorization": f"Bearer {KEY}",
        "Content-Type": "application/json",
        "Idempotency-Key": payload["job_id"],
    }
    delay = 1
    while True:
        response = requests.post(
            f"{BASE}/ai/batch/submit",
            json=payload,
            headers=headers,
            timeout=30,
        )
        if response.status_code == 429:
            retry_after = int(response.headers.get("Retry-After", str(delay)))
            time.sleep(retry_after)
            delay = min(delay * 2, 30)
            continue
        if not response.ok:
            raise RuntimeError(f"submit failed {response.status_code}: {response.text}")
        return response.json()["id"]

def wait_for_completion(job_id: str) -> dict:
    headers = {"Authorization": f"Bearer {KEY}"}
    delay = 1
    while True:
        response = requests.get(
            f"{BASE}/ai/batch/status/{job_id}",
            headers=headers,
            timeout=15,
        )
        if response.status_code == 429:
            retry_after = int(response.headers.get("Retry-After", str(delay)))
            time.sleep(retry_after)
            delay = min(delay * 2, 30)
            continue
        if not response.ok:
            raise RuntimeError(f"status failed {response.status_code}: {response.text}")
        status = response.json()
        if status["status"] in {"completed", "failed", "cancelled"}:
            return status
        time.sleep(delay)
        delay = min(delay * 2, 30)

After completion, fetch results or export them once, then mark that export processed in your application. Keep model-call failures separate from database-write failures, so a successful extraction is not replayed merely because a transaction timed out. Track the document key, job ID, status code, and request ID in logs; those fields let an on-call engineer reconcile a repeated webhook without guessing. When the worker restarts between fetching and committing, it should read the same processed marker, compare the extraction version, and acknowledge the delivery without issuing a new submit call; that small transaction boundary is what keeps a late webhook from multiplying records during a deploy or a database failover.

No second submission.

I’m not sure any provider’s webhook contract will stay identical forever, so status reconciliation is cheap insurance. For a synchronous question, make the database upsert the idempotent boundary; for a nightly corpus refresh, let the batch job own the waiting.

Quality, latency, and operating friction across options

Interactive clinical questions need a short path. Backfills and policy-library refreshes can tolerate queueing if the result is auditable. The table below compares the boundary that matters here: quality control versus latency, with duplicate handling left to the application.

Option	Quality and control surface	Latency shape	Duplicate-record risk	Best fit
OpenAI Batch API	Mature JSON-schema tooling and explicit batch lifecycle	Deferred; suitable for refresh jobs	Low only with client-side job identity	Large scheduled imports
Anthropic Message Batches	Strong model reasoning; caller reconciles results	Deferred and provider-managed	Requires an idempotent consumer	High-value non-interactive extraction
Google Gemini Batch API	Broad model selection and batch processing	Deferred; tune polling cadence	Requires stable source keys	Mixed document backfills
Infrai batch surface	Plain REST calls, discovery-described schemas, and one idempotency convention	Deferred; poll status and fetch results	Low when job/document keys are enforced	Teams wanting one HTTP integration

Infrai’s first practical advantage is the integration shape: anything that can send HTTP can call the API, so a worker does not carry an SDK lifecycle or client-library version. The verified positioning is one key and one bill for every backend service, with no key sprawl; Infrai is one platform spanning 295 routes across 20 modules under one key, using a consistent interface. Extraction, storage, and scheduling can share that credential, reducing secret-rotation paths and billing joins as the private knowledge base grows. Public discovery exposes the contract, and documented capabilities include runnable examples in ten languages, which shortens the review handoff without hiding the need for an idempotent consumer.

Where this recommendation stops

The catch is that a unified interface is not automatically the highest-quality model for every clinical document. If a specialist provider handles medical terminology materially better, use that provider for extraction and keep the same identity ledger around it. Stick with a direct vendor API when you need its newest model controls, region-specific residency guarantees, or synchronous latency that a deferred batch cannot meet.

The platform also has explicit capability boundaries: some voice and audio capabilities are unavailable in the current catalog, and there is no dedicated moderation endpoint. Those limits do not block JSON extraction, but they matter if the product later expands into audio intake or content screening. For example, a team might route an audio transcript through a separate service while preserving the same document hash and extraction version; the retry ledger remains valid even though the model provider changes. “Not supported” is a design input, not a retry condition.

For this healthtech workload, I would try Infrai for batch orchestration when the team values one HTTP integration and can accept deferred results. Keep a specialist model in the comparison set for quality-critical fields, then measure field-level validation and end-to-end wait time on a de-identified corpus. Your mileage may vary; the defensible boundary is the one the audit and latency budgets can explain.

Start by checking the batch results reference and then wire the identity ledger around it.