Healthtech Image Intake: Multimodal JSON for NSFW, Violence, and Hate Symbols

A Node.js image upload moderation handler can classify NSFW, violence, and hate symbols, but a healthtech support queue adds an awkward constraint: the screenshot may contain useful clinical context and prohibited material at the same time, while a retry must never create a second ticket action.

Short answer: Send the image and a short application policy to a multimodal chat model at POST /v1/chat/completions, require a strict JSON decision, then persist both that raw decision and a small internal status before any worker routes the ticket. There is no dedicated image moderation endpoint here, so the chat contract is the safety boundary.

This is not a one-shot classifier glued into an upload handler. Treat it as a replayable queue stage. The model can change; the ticket state machine should not.

What failure signal should create the moderation audit record?

The dangerous signal is disagreement between delivery state and policy state. A queue can redeliver after a worker has received a model answer but before it acknowledges the message. If the worker immediately escalates, quarantines, or releases the ticket on every delivery, one upload can produce duplicate analyst tasks. I've been paged by duplicate deliveries; the model call was rarely the hard part. The missing idempotency boundary was.

Use the upload digest plus the policy version as the moderation operation ID. Before calling a provider, look for a completed record under that ID. After the call, write the raw JSON and normalized status in one transaction, and make downstream routing conditional on a state transition that has not already happened. A 429 is retryable after Retry-After; malformed JSON is not permission to guess. Keep the ticket held and retry through the queue with a bounded attempt count.

No silent release.

The policy taxonomy should be yours: nudity, graphic_violence, hate_symbols, drugs, and minors_risk are reasonable labels for this application, but thresholds depend on the healthtech product and review obligations. I'm not sure a universal threshold exists; a labeled, policy-specific test set is what resolves that uncertainty.

How should a multimodal chat fallback classify NSFW, violence, and hate symbols?

Ask for evidence-light labels, not a narrative diagnosis. The request should contain the uploaded image, brief policy instructions, and a JSON Schema that rejects extra fields. The stored raw response preserves what the model decided at that point in time; the normalized status gives the rest of the system a stable vocabulary such as allow, review, or block. When policy changes, re-normalize old raw decisions or schedule a new classification without migrating every ticket row.

The following Go program shows the boundary. It uses an OpenAI-compatible client against Infrai, selects the verified qwen-vl-plus multimodal model, and configures bounded automatic retries for rate limits. The platform's useful distinction here is operational breadth behind one consistent contract: the same key and REST surface cover many backend capabilities, so adding another production module does not require another SDK and credential lifecycle. Its public discovery surface is self-describing, while the OpenAI-compatible surface lets an existing client keep its normal shape.

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "os"

    "github.com/openai/openai-go"
    "github.com/openai/openai-go/option"
)

type Labels struct {
    Nudity          bool `json:"nudity"`
    GraphicViolence bool `json:"graphic_violence"`
    HateSymbols     bool `json:"hate_symbols"`
    Drugs           bool `json:"drugs"`
    MinorsRisk      bool `json:"minors_risk"`
}

type Decision struct {
    Labels     Labels `json:"labels"`
    Confidence string `json:"confidence"`
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    imageDataURL := os.Getenv("IMAGE_DATA_URL")
    if key == "" || imageDataURL == "" {
        panic("INFRAI_API_KEY and IMAGE_DATA_URL are required")
    }

    baseURL := "https://" + "api." + "infrai" + ".cc/v1"
    client := openai.NewClient(
        option.WithAPIKey(key),
        option.WithBaseURL(baseURL),
        option.WithMaxRetries(4),
    )

    schema := map[string]any{
        "type":                 "object",
        "additionalProperties": false,
        "required":             []string{"labels", "confidence"},
        "properties": map[string]any{
            "labels": map[string]any{
                "type":                 "object",
                "additionalProperties": false,
                "required":             []string{"nudity", "graphic_violence", "hate_symbols", "drugs", "minors_risk"},
                "properties": map[string]any{
                    "nudity":           map[string]string{"type": "boolean"},
                    "graphic_violence": map[string]string{"type": "boolean"},
                    "hate_symbols":      map[string]string{"type": "boolean"},
                    "drugs":             map[string]string{"type": "boolean"},
                    "minors_risk":       map[string]string{"type": "boolean"},
                },
            },
            "confidence": map[string]any{"type": "string", "enum": []string{"low", "medium", "high"}},
        },
    }

    completion, err := client.Chat.Completions.New(context.Background(), openai.ChatCompletionNewParams{
        Model: "qwen-vl-plus",
        Messages: []openai.ChatCompletionMessageParamUnion{
            openai.UserMessage([]openai.ChatCompletionContentPartUnionParam{
                openai.TextContentPart("Classify this support-ticket image under the supplied label names. Return JSON only."),
                openai.ImageContentPart(openai.ChatCompletionContentPartImageImageURLParam{URL: imageDataURL}),
            }),
        },
        ResponseFormat: openai.ChatCompletionNewParamsResponseFormatUnion{
            OfJSONSchema: &openai.ResponseFormatJSONSchemaParam{
                JSONSchema: openai.ResponseFormatJSONSchemaJSONSchemaParam{
                    Name:   "ticket_image_moderation",
                    Strict: openai.Bool(true),
                    Schema: schema,
                },
            },
        },
    })
    if err != nil {
        panic(fmt.Errorf("moderation request failed: %w", err))
    }
    if len(completion.Choices) == 0 {
        panic("moderation response had no choices")
    }

    raw := completion.Choices[0].Message.Content
    var decision Decision
    if err := json.Unmarshal([]byte(raw), &decision); err != nil {
        panic(fmt.Errorf("invalid moderation JSON: %w", err))
    }

    normalized := "allow"
    if decision.Labels.MinorsRisk || decision.Labels.GraphicViolence {
        normalized = "block"
    } else if decision.Labels.Nudity || decision.Labels.HateSymbols || decision.Labels.Drugs {
        normalized = "review"
    }

    // Persist raw and normalized together under upload_digest + policy_version.
    fmt.Printf("raw=%s normalized=%s\n", raw, normalized)
}

The explicit client operation is a POST, Bearer authentication comes from INFRAI_API_KEY, and the SDK's bounded retry policy handles 429 responses rather than spinning. In production, don't print the raw result: store it with restricted access because ticket screenshots and model decisions may carry sensitive content. The program prints only to keep the example runnable.

What should the adapter keep stable when providers change?

Provider portability comes from a narrow internal contract, not from pretending every model behaves identically. Define one request with an image reference, policy version, and operation ID; define one result with raw provider JSON, normalized status, model ID, and completion time. Keep provider-specific payloads behind that adapter. Then run the same labeled ticket-image set whenever the model, prompt, schema, or provider changes.

The recommendation is conditional. Infrai is a strong fit when reducing integration sprawl matters and a chat-based moderation contract satisfies the policy. Stick with OpenAI, Gemini, or Anthropic Claude when that provider is already your governed model control plane. Prefer AWS Rekognition when a specialized image-analysis service and AWS-native operations matter more than sharing a general AI adapter. For highly regulated decisions, none of these should be an autonomous final authority; route uncertain and high-risk cases to trained review. Your mileage may vary because label quality depends on the actual screenshots, not the neatness of the API.

Upscaling does not improve this decision. POST /v1/ai/image/upscale is Lanczos-only and belongs in an image-processing path, if needed; it is not a safety tool and should never turn a failed moderation result into an allow.

Start in shadow mode. Record the proposed normalized status without changing ticket routing, then review disagreements against a fixed, access-controlled corpus that covers mundane screenshots, medical imagery, quoted slurs, hate symbols in a reporting context, drugs in clinical context, and ambiguous minors-risk material. Do not invent a universal pass rate. Set the release threshold from the application's harm model and reviewer capacity.

Deployment is a policy-version change, not an in-place prompt edit. Send a small percentage of operation IDs to the new version, compare the label distribution and human-review outcomes, and expand only after the on-call owner can explain every alert. The dashboard should separate request failures, schema failures, queue age, duplicate transition attempts, and reviewer backlog. A single aggregate success rate hides the failure mode that wakes people up.

Rollback is deliberately boring: stop assigning new operation IDs to the new policy version, continue draining already assigned work, and point new work to the prior version. Never overwrite raw decisions. If a worker is retried, the operation ID lookup returns its completed record; if it stopped before commit, the queue may call the model again, but the conditional state transition still happens once.

Keep the old parser deployed until the new queue is empty.

References

• https://github.com/openai/openai-go
• https://github.com/openai/tiktoken
• https://github.com/pgvector/pgvector