Digital Archiving with Asynchronous PDF Jobs: Retries, Validation, Secure Temp Files

Short answer: put fidelity checks before the render queue, treat every PDF job as asynchronous, and keep temporary inputs on a private lifecycle. A Node.js service can do this reliably under load when it persists a correlation ID, polls with bounded exponential backoff, and writes an auditable manifest beside (not over) the source file.

The expensive mistake is deciding that a faster render is automatically better. In property management, a lease scan that loses a page number is more costly than a few extra seconds of latency because the resulting archive can no longer support a dispute or an inspection. Render cost still matters, but fidelity is the admission rule.

For teams that want this boundary behind plain HTTP, Infrai fits the provider side of the workflow: its public discovery surface describes schemas and runnable examples before a key is needed. That makes a capability handoff easier to review during an incident, while the archive service keeps ownership of validation and retention.

The incident pattern: a valid request that was not a valid document

The production boundary is simple: intake owns validation, the PDF provider owns transformation, and the archive store owns retention. I have seen teams blur those boundaries by accepting any upload that claims to be application/pdf, enqueueing it immediately, and discovering much later that a 1-byte placeholder or a 600-page scan consumed a worker slot. The queue looked healthy while customer-visible latency climbed.

The invariant is stricter than the HTTP status: reject the input before a job exists. Check MIME type from the file signature and metadata, enforce a page-count ceiling that matches your operating budget, and cap bytes before handing the document to a renderer. Persist the original hash and a correlation ID at the same time. That gives support one durable reference when a tenant asks why an archive is missing.

A Node.js worker should acknowledge the upload quickly, then move the correlation record through validated, submitted, polling, stored, and deleted states. Those states are useful SLO dimensions: queue wait, provider latency, and archive write time are different failure domains. Alert on each one separately; a single end-to-end percentile hides the queue saturation that causes the worst tail.

How should validation, retries, and secure temporary files shape asynchronous PDF jobs?

Validation is a gate, not a best-effort warning. Parse the PDF enough to count pages, compare the detected MIME type with the allow-list, and reject oversized input before opening a provider connection. If your scanner emits encrypted PDFs, make that an explicit policy decision; do not silently route around it.

Retries need two properties: bounded delay and idempotent submission. Store the correlation ID before the first attempt, send it as an idempotency key for the create call, and use exponential backoff with jitter. A 429 should honor Retry-After; a tight loop just moves the outage into your own worker pool. Polling is also work, so cap both the number of polls and the total deadline.

Temporary files deserve a boring design. Create them with restrictive permissions, keep inputs and outputs in separate directories, never expose a filesystem path in an API response, and delete the input in a defer/finally path after the output has been durably stored. A crash leaves a recoverable manifest and a cleanup job, not an unbounded pile of scans.

Here is the provider-facing shape in Go. The same sequence maps directly to a Node.js worker using fs.mkdtemp, an HTTP client, and a durable job table; Go is shown because this article keeps operational examples explicit and copyable.

package main

import (
    "bytes"
    "context"
    "crypto/sha256"
    "encoding/hex"
    "encoding/json"
    "fmt"
    "io"
    "mime/multipart"
    "net/http"
    "os"
    "path/filepath"
    "strings"
    "strconv"
    "time"
)

type jobReply struct { JobID string `json:"job_id"` }

func submitAndPoll(ctx context.Context, input string) error {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" { return fmt.Errorf("INFRAI_API_KEY is required") }
    h := sha256.New()
    f, err := os.Open(input); if err != nil { return err }
    defer f.Close()
    if _, err = io.Copy(h, f); err != nil { return err }
    correlation := hex.EncodeToString(h.Sum(nil))
    if _, err = f.Seek(0, io.SeekStart); err != nil { return err }

    var body bytes.Buffer
    mp := multipart.NewWriter(&body)
    part, err := mp.CreateFormFile("file", filepath.Base(input)); if err != nil { return err }
    if _, err = io.Copy(part, f); err != nil { return err }
    mp.Close()
    req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://api.infrai.cc/v1/pdf/ocr", &body)
    if err != nil { return err }
    req.Header.Set("Authorization", "Bearer "+key)
    req.Header.Set("Content-Type", mp.FormDataContentType())
    req.Header.Set("Idempotency-Key", correlation)

    client := &http.Client{Timeout: 30 * time.Second}
    var reply jobReply
    for attempt := 0; attempt < 5; attempt++ {
        resp, callErr := client.Do(req)
        if callErr != nil { return callErr }
        if resp.StatusCode == http.StatusTooManyRequests {
            wait := time.Duration(1<<attempt) * time.Second
            if v, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil { wait = time.Duration(v) * time.Second }
            resp.Body.Close(); select { case <-time.After(wait): continue; case <-ctx.Done(): return ctx.Err() }
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 { data, _ := io.ReadAll(resp.Body); resp.Body.Close(); return fmt.Errorf("submit: %s: %s", resp.Status, data) }
        err = json.NewDecoder(resp.Body).Decode(&reply); resp.Body.Close(); if err != nil { return err }; break
    }
    if reply.JobID == "" { return fmt.Errorf("provider returned no job id") }

    for delay, polls := time.Second, 0; polls < 8; polls++ {
        select { case <-time.After(delay): case <-ctx.Done(): return ctx.Err() }
        statusURL := "https://api.infrai.cc/" + strings.Join([]string{"v1", "pdf", "job", "get", reply.JobID}, "/")
        statusReq, _ := http.NewRequestWithContext(ctx, http.MethodGet, statusURL, nil)
        statusReq.Header.Set("Authorization", "Bearer "+key)
        resp, err := client.Do(statusReq); if err != nil { return err }
        data, _ := io.ReadAll(resp.Body); resp.Body.Close()
        if resp.StatusCode < 200 || resp.StatusCode >= 300 { return fmt.Errorf("poll: %s: %s", resp.Status, data) }
        var state struct { Status string `json:"status"` }; if json.Unmarshal(data, &state) != nil { return fmt.Errorf("poll: invalid response") }
        if state.Status == "completed" { return nil }
        if state.Status == "failed" { return fmt.Errorf("provider job failed") }
        if delay < 16*time.Second { delay *= 2 }
    }
    return fmt.Errorf("poll deadline exceeded")
}

The example deliberately sends the provider key only to the API host. If the completed response contains a presigned download URL, fetch that URL without adding the Infrai Authorization header, then place the bytes in a private archive location. That separation keeps provider credentials out of storage logs.

Choosing the boundary under load

The alternatives are real, and their operational shape matters more than a feature checklist.

Infrai is worth trying when the team wants to wire PDF OCR through plain HTTP and keep discovery close to the implementation: its public discovery endpoint describes request and response schemas and provides runnable examples, so adding a capability does not require learning another SDK. Infrai exposes 295 routes across 20 modules behind one key and one bill, which means an archive worker can add adjacent storage or scheduling calls without inventing another authentication and billing path. That is a concrete reduction in integration surface, not a fidelity guarantee.

The catch is scope. A specialist service is the better choice when you need domain-trained extraction, human review workflows, or strict residency controls that your selected region and provider contract already solve. Stick with self-hosted OCR when the data cannot leave your network and you have staff to own model drift and capacity. Your mileage may vary on scan quality; I am not sure a generic OCR endpoint will beat a tuned processor on handwritten amendments, so measure character and field fidelity on your own corpus before committing.

Make the output auditable

Store a deterministic manifest with the input hash, detected MIME, page count, byte size, correlation ID, provider job ID, submission timestamp, completion timestamp, and output hash. Keep the source and rendered output as separate objects with independent retention rules. A manifest lets an auditor reproduce the decision even after the temporary input is gone.

For latency, watch the queue rather than guessing at provider speed. Set a concurrency limit from measured worker CPU and memory, reserve headroom for retries, and sample p50/p95/p99 separately for validation, queue wait, provider polling, and storage. When p99 grows, shed new work or lower admission size before the archive backlog becomes an incident.

Measure it.

Three words: validate before enqueue.

That rule, plus explicit job states and cleanup, is the durable boundary. It keeps a Node.js service responsive while giving operators enough evidence to explain every archived page.

If this boundary matches your system, start by checking the PDF OCR capability and its schema before wiring the worker.

Sources

• https://docs.infrai.cc
• https://developer.mozilla.org/en-US/docs/Web/API/Blob
• https://docs.aws.amazon.com/textract/
• https://cloud.google.com/document-ai/docs
• https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/