Short answer: for community image safety, validate the original upload before resize or OCR, then revalidate the derivative later in the lifecycle before it reaches the feed.
That ordering protects coverage. A thumbnail can hide a needle, a crop can remove the context that made a photo unsafe, and a format conversion can change what a detector sees. In a one-person SaaS, I care about revenue per hour, so I outsource the undifferentiated image plumbing while keeping the policy and the review queue in my code. Ship weekly, but make the gate boring.
What should a healthtech feed validate before transformation?
Start with bytes, not a browser filename. Check the declared media type, the magic bytes, dimensions, decoded pixel count, and an upper bound on compressed size. Reject files that cannot be decoded by a trusted library. Do not let EXIF orientation, embedded profiles, or animation frames become invisible policy inputs; normalize those details only after the first decision has been recorded.
The first pass should answer one narrow question: is this upload eligible to enter the transformation worker? A review result is a stop, not a soft warning. Store the object in a quarantine bucket with a short retention policy, attach the account and policy version, and give a human reviewer a stable reference rather than a public URL.
I once assumed a 12 MB phone photo was the hard case. It was not. The nasty case was a 2 MB file whose decoded canvas expanded past our memory ceiling. The upload looked harmless in the request log; the transformer was the component that paid for the mistake. A byte limit and a decoded-pixel limit belong at the same gate. That incident also exposed a quieter gap: our logs kept the object key but not the transform version, so we could not tell which derivative a reviewer had actually seen. We added both hashes, replayed the quarantined sample, and found that a center crop removed the warning label that had driven the original review decision. The fix was not a cleverer prompt. It was a second gate on the bytes we publish.
Small gate. Big payoff.
Keep the policy output small. allow, review, and block are enough for routing; a separate category and reason make appeals searchable. A model score is evidence, not the policy. The policy owner decides the threshold, and the version travels with every decision.
Can a second check after upload catch transformation drift?
Yes, and it catches a different class of failure. The derivative may be a JPEG instead of a PNG, a center crop instead of the full frame, or a frame extracted from an animated image. Run the same decision contract against the bytes that the CDN will serve, then compare the result with the original decision. If they disagree, route to review and keep both hashes.
The second pass should be asynchronous. It must not make a user wait for a thumbnail, but it must finish before the derivative receives a cacheable public URL. A queue record can carry original_sha256, derivative_sha256, policy_version, and transform_version; that is enough to reproduce which artifact was approved.
Here is the smallest TypeScript boundary I use. The endpoint is injected so the application can point at a managed API or an internal classifier without changing the lifecycle code.
type Decision = "allow" | "review" | "block";
type SafetyResult = {
decision: Decision;
category: string;
reason: string;
policyVersion: string;
};
type ImageRef = {
bytes: Uint8Array;
sha256: string;
kind: "original" | "derivative";
transformVersion?: string;
};
async function moderate(
endpoint: string,
image: ImageRef,
policyVersion: string,
): Promise<SafetyResult> {
const response = await fetch(endpoint, {
method: "POST",
headers: { "content-type": "application/octet-stream" },
body: image.bytes,
});
if (!response.ok) {
throw new Error(`moderation request failed: ${response.status}`);
}
const result = (await response.json()) as SafetyResult;
if (!(["allow", "review", "block"] as Decision[]).includes(result.decision)) {
throw new Error("moderation contract returned an unknown decision");
}
if (result.policyVersion !== policyVersion) {
throw new Error("moderation policy version mismatch");
}
return result;
}
export async function approveForFeed(
endpoint: string,
original: ImageRef,
derivative: ImageRef,
policyVersion: string,
) {
const before = await moderate(endpoint, original, policyVersion);
if (before.decision !== "allow") return { status: before.decision, before };
const after = await moderate(endpoint, derivative, policyVersion);
if (after.decision !== "allow") return { status: "review", before, after };
return {
status: "publishable",
before,
after,
hashes: { original: original.sha256, derivative: derivative.sha256 },
} as const;
}
The application should fail closed when the classifier or schema is unavailable. That does not mean deleting the upload: quarantine it, expose a retry state to the operator, and expire it under the retention rule. Never turn a timeout into allow just to keep the feed moving.
Build the queue around coverage, not latency alone
Coverage is the primary decision axis here. Measure false negatives on a policy-reviewed set that includes low light, screenshots, handwritten notes, medical instruments, nudity, and images with text overlaid. Include benign edge cases too; an overfull review queue is a safety problem because humans start rubber-stamping it.
Track original-pass rate, derivative-pass rate, disagreement rate, decode rejects, queue age, and reviewer overturns. Slice every metric by image kind and policy version. A single aggregate precision number hides the exact crop or format that is leaking through.
The experiment can be run in shadow mode for a week: decisions are logged, but an existing gate controls publication. Compare labels with two reviewers on disagreements, calculate confidence intervals, and promote a policy only after the disagreement queue is understood. I'm not sure any fixed threshold will transfer between a pediatric community and an adult wellness forum; your mileage may vary, which is why the corpus and policy owner matter more than a benchmark screenshot.
What changes when the feed gets larger?
At scale, separate the byte quarantine, transformation worker, and publication ledger. Give each job an idempotency key built from the original hash, transform version, and policy version. Cache a decision only for that tuple; otherwise a policy update can accidentally reuse an old approval.
Budget reviewer minutes, not just API calls. A ten percent disagreement rate can cost more than inference if every disputed image needs two humans. Add a dead-letter queue, alert on age rather than raw failure count, and retain the smallest evidence package that supports an appeal.
The catch is that two-pass moderation adds latency, storage, and a second place to tune policy. It is not suitable when the product must publish instantly and can tolerate only a coarse, pre-upload filter; in that case, keep the original private and choose a single synchronous gate with a human escalation path. A self-hosted detector may be the better choice when images cannot leave your jurisdiction, provided you can staff model updates and incident response. Managed classification is a poor fit if its taxonomy cannot express your safety policy.
I would change one thing first as traffic grows: move derivative moderation into a canary queue and sample every disagreement for review. That buys signal without pretending the model is the final authority. The weekly shipping habit stays; the evidence gets better.
