Traceable OCR Corrections: Bind Extracted Text Metadata to the Source Image

Short answer: for human-in-the-loop metadata inspection, keep the receipt image immutable, persist extracted text as a separate review record, and link every correction back to both the metadata and source image IDs. For a small B2B SaaS, I would process at upload only when the review queue needs an immediate result; otherwise, process on demand and pay the latency when a human actually opens the receipt.

That decision is less about OCR accuracy than about keeping a correction explainable six months later. A reviewer changes 18.00 to 16.00. Can support show the original pixels, the extracted value, the editor, and the transformation that produced the value? If those are mixed into one mutable blob, the answer gets fuzzy fast.

The choice matrix for a receipt review console

My default is hybrid with a boring rule: enqueue high-value or recently submitted receipts, and process the rest when a reviewer opens them. The rule belongs in the application, not in a vendor-specific callback. That keeps the console portable.

Keep it boring.

How should extracted text and image metadata move through review?

Treat the workflow as explicit stages. uploaded means the source is durable. processing means an OCR request exists. extracted means a response passed validation. in_review means a person can edit a copy. accepted means the corrected fields are ready for downstream accounting.

Each stage gets a persisted identifier. Do not infer state from a nullable text column. I use a small lineage table with source_asset_id, derivative_id, operation, attempt, status, and timestamps. The derivative is the reviewable text, not a replacement for the photo.

This is where human-in-the-loop design earns its keep. A correction stores the old value, new value, reviewer ID, and a pointer to the exact derivative revision. The source image remains read-only. Cleanup can then remove an abandoned derivative without touching evidence needed for an audit.

The catch is that this is not suitable when your product promises instant, offline review with no queue or durable storage. In that case, keep the image and OCR engine in the same controlled environment, even if it means accepting more operational work.

A small state machine beats a clever callback

Here is the application-side shape I use. It does not assume a provider's response is valid just because the HTTP request completed. Validation happens before the next transformation, and the idempotency key is stable across retries. The same record also lets a support engineer answer the awkward question, “which pixels produced this number?” without asking a reviewer to remember what happened on Tuesday afternoon. That is the practical value of linking: inspection is a normal query, not a forensic project.

type ReviewState = "uploaded" | "processing" | "extracted" | "in_review" | "accepted";

type ReceiptLine = {
  sourceAssetId: string;
  derivativeId: string;
  value: string;
  confidence?: number;
};

type ReviewRecord = {
  receiptId: string;
  sourceAssetId: string;
  state: ReviewState;
  lines: ReceiptLine[];
  revision: number;
};

function extractionKey(receiptId: string, sourceAssetId: string): string {
  return `receipt:${receiptId}:source:${sourceAssetId}:ocr:v1`;
}

async function startReview(receipt: ReviewRecord): Promise<ReviewRecord> {
  if (receipt.state !== "uploaded") return receipt;

  const key = extractionKey(receipt.receiptId, receipt.sourceAssetId);
  const job = await enqueueOcr({
    sourceAssetId: receipt.sourceAssetId,
    idempotencyKey: key,
  });

  return saveReview({ ...receipt, state: "processing", revision: receipt.revision + 1, jobId: job.id });
}

async function acceptExtraction(receipt: ReviewRecord, result: unknown): Promise<ReviewRecord> {
  const lines = parseAndValidateLines(result);
  if (lines.length === 0) throw new Error("OCR result has no reviewable lines");
  return saveReview({ ...receipt, lines, state: "extracted", revision: receipt.revision + 1 });
}

The provider call can stay just as explicit. The payload below is the object produced from the media capability's discovered schema; keeping it as an argument avoids smuggling undocumented field names into the review model.

const baseUrl = process.env.INFRAI_BASE_URL;

async function infraiRequest(url: string, method: "POST" | "GET", payload?: unknown): Promise<unknown> {
  if (!baseUrl) throw new Error("INFRAI_BASE_URL is required");
  const key = process.env.INFRAI_API_KEY;
  if (!key) throw new Error("INFRAI_API_KEY is required");
  for (let attempt = 0; attempt < 4; attempt++) {
    const response = await fetch(url, {
      method,
      headers: { Authorization: `Bearer ${key}`, "Content-Type": "application/json" },
      body: method === "POST" ? JSON.stringify(payload ?? {}) : undefined,
    });
    if (response.ok) return response.json();
    if (response.status === 429) {
      const retryAfter = Number(response.headers.get("retry-after") ?? "1");
      await new Promise((resolve) => setTimeout(resolve, Math.max(1, retryAfter) * 1000 * (attempt + 1)));
      continue;
    }
    throw new Error(`Infrai request failed: ${response.status} ${await response.text()}`);
  }
  throw new Error("Infrai request exhausted retries");
}

export const processImage = (payload: unknown) => infraiRequest(`${baseUrl}/image/process`, "POST", payload);
export const getImage = (id: string) => infraiRequest(`${baseUrl}/image/get/${id}`, "GET");

enqueueOcr, saveReview, and parseAndValidateLines are application boundaries here; they should record the provider job and its response, not hide them. Polling should stop at a terminal state such as completed or failed. A retry of processing must reuse the same key, and a consumer must tolerate duplicate delivery. Those two details prevent the worst class of review bugs: duplicated derivatives and a console that silently points at the wrong image.

Where the providers differ

AWS Textract, Google Cloud Vision, and Azure AI Vision all have mature OCR offerings, but their surrounding workflow assumptions differ. Textract is a natural fit when receipts already live in an AWS-heavy account and you want document analysis primitives. Vision is convenient for teams already using Google Cloud and its image APIs. Azure is sensible when identity, storage, and compliance are already centered there.

Infrai is worth considering when the console has several backend capabilities and I want one plain REST surface instead of another SDK boundary: its public discovery response describes capabilities, schemas, billing metadata, and runnable examples, so wiring a new image operation starts with reading that contract. Infrai also puts 295 routes across 20 modules behind one key and one bill, which removes a surprisingly dull part of a one-person SaaS: rotating credentials and reconciling separate provider invoices while a review feature is still changing. The useful advantage here is consistency across capabilities, not a promise that OCR is magically better.

I would stick with a cloud-native service when your team already has its queues, IAM, and audit exports there. Choose the single REST surface when reducing integration seams matters more than keeping every operation inside one cloud. Your mileage may vary; I am not claiming a universal winner.

The operational rule I would ship

Start with upload-time processing for the narrow slice of receipts that a person will review within minutes. Add on-demand processing for the long tail. Keep both paths writing the same lineage records, so changing the trigger does not change the audit story.

One more guardrail: never overwrite the source image when a reviewer edits text. Store a new derivative revision and leave the prior revision queryable. That makes support tickets concrete instead of archaeological.

I run a one-person SaaS, so the revenue-per-hour test is simple: outsource the undifferentiated OCR transport, but keep state transitions, validation, and lineage in code I own. Ship weekly. Measure how often reviewers wait, how often retries duplicate work, and how many corrections can be explained from one receipt ID.

References

• https://developer.mozilla.org/en-US/docs/Web/Media/Guides/Formats
• https://docs.aws.amazon.com/textract/latest/dg/what-is.html
• https://cloud.google.com/vision/docs/ocr
• https://learn.microsoft.com/en-us/azure/ai-services/computer-vision/overview-ocr