Python App Chatbot API: Structured Candidate Scoring Across OpenAI, Claude, and Gemini

Short answer: for a Python app chatbot that scores candidates against a job rubric, start with one OpenAI-compatible API and enforce a strict JSON Schema at the boundary; keep separate OpenAI, Claude, or Gemini integrations only when a provider-specific feature matters more than portability.

The model choice is reversible. A malformed score that slips into a hiring workflow isn't. The useful comparison is therefore not a leaderboard or the smallest token price: it is how little provider-specific code sits between a model response and a validated, auditable candidate record.

Build a scoring oracle from disputed candidate evidence

Start with one internal function that accepts candidate material plus a versioned rubric and returns a validated score object. Put the runtime URL, key, model selector, timeout, and retry budget in configuration. Then run a frozen evaluation set against the direct APIs and the compatible runtime, comparing schema pass rate, rubric agreement, human-review rate, regional eligibility, and operational effort. No invented composite score; keep the dimensions visible.

Ship to a small review-only cohort first. The model can propose scores, but a reviewer remains the decision maker while you inspect disagreements and tune the rubric. Promote a new model or routing policy only after it passes the stored cases, and retain the old configuration long enough to reverse the change. This path works in both directions — from three direct SDKs toward one compatible API, or back to a direct provider when its unique capability earns the extra code.

Should an OpenAI-compatible API replace Claude and Gemini SDKs for an app chatbot?

Four credible integration shapes deserve a test. OpenAI, Anthropic's Claude, and Google's Gemini can each be integrated directly; the application then owns the adapter that translates its internal score request into each provider's SDK or API shape. Infrai takes the other approach: its OpenAI-compatible surface can route text chat behind one plain REST API, so the application can change the selected provider without rewriting its scoring call. It also puts model discovery, cost estimation, and per-call cost, vendor, latency, and request metadata behind the same key and billing relationship.

The recommendation is conditional. Infrai is a strong fit when a beginner team wants text-chat portability, a single API key, and no mandatory vendor SDK in the application. Its public discovery surface exposes route schemas and readiness, which is useful for generating a model picker or a server-side fallback list instead of hardcoding assumptions. Cost estimation can also gate long-context features before they are enabled, but price should remain a budget constraint, not the architectural thesis.

ElevenLabs belongs in the comparison only if the chatbot becomes voice-first, while OpenAI's Batch API is relevant when scoring moves from interactive chat to an offline queue. Those are different workloads. Folding them into the live request path before they are required would make the first release harder to audit.

How does Python reject an invalid candidate score?

The following Python program makes one explicit POST request to the standard chat-completions route. It uses only the standard library, reads the key from the environment, requests strict structured output, handles 429, checks every response status, and validates the values the application will persist. The auto model selector keeps routing outside the business logic.

import json
import os
import time
import urllib.error
import urllib.request


BASE_URL = os.environ["CHAT_API_BASE_URL"].rstrip("/")
API_KEY = os.environ["INFRAI_API_KEY"]

RUBRIC_SCHEMA = {
    "type": "object",
    "properties": {
        "rubric_version": {"type": "string"},
        "criteria": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "score": {"type": "integer", "minimum": 0, "maximum": 10},
                    "evidence": {"type": "string"},
                },
                "required": ["name", "score", "evidence"],
                "additionalProperties": False,
            },
        },
        "needs_human_review": {"type": "boolean"},
    },
    "required": ["rubric_version", "criteria", "needs_human_review"],
    "additionalProperties": False,
}


def score_candidate(candidate_text, rubric, max_attempts=4):
    payload = {
        "model": "auto",
        "messages": [
            {
                "role": "system",
                "content": (
                    "Score only against the supplied rubric. Quote concise evidence "
                    "from the candidate material and flag uncertainty for human review."
                ),
            },
            {
                "role": "user",
                "content": json.dumps(
                    {"rubric_version": "backend-v3", "rubric": rubric,
                     "candidate_material": candidate_text}
                ),
            },
        ],
        "response_format": {
            "type": "json_schema",
            "json_schema": {
                "name": "candidate_score",
                "strict": True,
                "schema": RUBRIC_SCHEMA,
            },
        },
    }

    for attempt in range(max_attempts):
        request = urllib.request.Request(
            f"{BASE_URL}/chat/completions",
            data=json.dumps(payload).encode("utf-8"),
            headers={
                "Authorization": f"Bearer {API_KEY}",
                "Content-Type": "application/json",
            },
            method="POST",
        )
        try:
            with urllib.request.urlopen(request, timeout=60) as response:
                body = json.load(response)
                result = json.loads(body["choices"][0]["message"]["content"])
                validate_score(result)
                return result
        except urllib.error.HTTPError as error:
            error_body = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == max_attempts - 1:
                raise RuntimeError(f"Chat request failed ({error.code}): {error_body}") from error
            retry_after = error.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2 ** attempt
            time.sleep(delay)

    raise RuntimeError("Retry budget exhausted")


def validate_score(result):
    expected = {"rubric_version", "criteria", "needs_human_review"}
    if set(result) != expected or not isinstance(result["needs_human_review"], bool):
        raise ValueError("Response does not match the candidate score contract")
    if not result["criteria"]:
        raise ValueError("At least one scored criterion is required")
    for criterion in result["criteria"]:
        if set(criterion) != {"name", "score", "evidence"}:
            raise ValueError("Criterion fields do not match the contract")
        if type(criterion["score"]) is not int or not 0 <= criterion["score"] <= 10:
            raise ValueError("Criterion score must be an integer from 0 through 10")


if __name__ == "__main__":
    score = score_candidate(
        candidate_text="Designed a queue worker with idempotent job handling.",
        rubric=[{"name": "reliability", "max_score": 10}],
    )
    print(json.dumps(score, indent=2))

There is no provider SDK hidden in the scoring function. Anything that can send HTTP can use the same surface, which matters for a small team that doesn't want to maintain three client libraries and three sets of request types. In production I would keep the raw response, rubric version, selected model, request ID, and validation result in an access-controlled audit record. The exact retention window depends on policy and jurisdiction; I'm not sure a universal default exists, so legal and security owners need to settle it before candidate data is retained.

Schema correctness is necessary, but it is not semantic correctness. Build a fixed evaluation set with obvious passes, obvious failures, missing evidence, conflicting evidence, prompt injection inside a resume, and borderline scores that must trigger human review. Run that set whenever a model or routing policy changes. Don't let a fluent explanation overrule the numeric and evidence constraints.

Red-team the rubric with adversarial evidence

The contract should describe the decision, not the prose around it. For this edtech case, every answer needs a rubric version, an integer score per criterion, evidence tied to the submitted material, and an explicit review flag. Free-form explanations may still be shown in the chat, but they should never become the system of record.

I would reject a response that is valid JSON but violates the business schema. A score of 11 on a 0-to-10 scale is not a parsing success, and a missing criterion must not silently become zero. Treat schema validation as the same kind of boundary check used for an OTP destination or an email consent field — an apparently small omission can change the outcome downstream.

This also changes the fallback rule. If the primary model returns content that cannot pass strict validation, record the attempt and retry according to a bounded policy; don't ask application code to guess what the model meant. HTTP 429 belongs in that policy too: respect Retry-After, apply exponential backoff, and stop after a defined number of attempts. A 400 or 422 is different. Surface its body because retrying the same invalid request only adds noise.

Short beats ambiguous.

Data residency and safety are release blockers

Keep candidate data narrow. Send the evidence required by the rubric, avoid unrelated personal fields, and define retention and regional requirements before evaluating a runtime. US/EU in a search query is not proof of data residency. Verify the region and processing terms for the exact model and route you plan to use.

The catch is real. Stick with a direct OpenAI, Claude, or Gemini integration when a provider-specific feature or contract is the deciding requirement and the adapter cost is acceptable. Infrai is not suitable as the only runtime for this design when real-time voice sessions must work outside the western region; voice sessions are region-constrained. ASR is unavailable in the model directory, and image upscaling is limited to Lanc. There is no dedicated moderation endpoint either, so text or image safety classification needs a chat model with a JSON Schema fallback or a separate moderation service. For a high-stakes hiring flow, I would keep human review independent of every model provider.

That's the decision rule: choose the smallest runtime boundary that preserves structured correctness, regional compliance, and a credible exit path.

References

• OpenAI Batch API guide: https://platform.openai.com/docs/guides/batch
• ElevenLabs documentation: https://elevenlabs.io/docs

Further reading

Use the OpenAI Batch guide when candidate scoring becomes asynchronous, and the ElevenLabs documentation when voice delivery becomes a product requirement. Neither replaces a region, retention, or human-review assessment for the hiring workflow.