A bad cutout rarely announces itself as a bad cutout. The car lands on a new backdrop, the paint looks clean, then a thin piece is gone. An antenna. A tire lip. The dark seam under a rocker panel. The complaint that comes back is never technical. The vehicle looks wrong.
I wanted the last correction stage to fix fuzzy edges without handing it the whole car to rewrite. That sounds like a small distinction. It stops being small the first time a model improves one boundary and quietly damages another. So the rule is physical. Edit the uncertain strip. Leave the settled area alone.
This is Part 2. Part 1, "Negative Space Is a Label", was about supervision: what the pixels beside an object teach a model, and why a shadow touching a tire has to be labeled as evidence against foreground. This one moves from training to runtime. A mask already exists. Where is a learned stage allowed to act?
1. The contract lives in the band
CarSegNet is the research implementation here. Its pipeline module splits the route by media type, and the docstring says the design more clearly than any diagram I could draw after the fact.
Stills run SAM 3 text concept, then NSJ alpha, then composite. A detector box prompt and a depth prior are optional inputs. Video runs SAM 3.1 multiplex propagation, per-frame NSJ with temporal handling, a depth-parallax plate, composite, encode.
The list matters less than the handoff. SAM gives a semantic prior. NSJ receives a trimap band. The compositor receives a matte only after the prior and the refiner have each done bounded work.
flowchart TD
image[Vehicle Image]
segment[Concept Mask]
trimap[Trimap Band]
refiner[NSJ Alpha Refiner]
depth[Depth Prior]
composite[Showroom Composite]
frozen[Prior Frozen Outside Band]
image --> segment
segment --> trimap
trimap --> refiner
image --> depth
depth --> refiner
refiner --> composite
segment -.-> frozen
frozen --> composite
The diagram is a contract. It is not a model zoo. The refiner edits the uncertain strip. The semantic prior owns the rest of the frame.
Models build lazily. A pure recomposite run against cached mattes never pays to load a large segmentation checkpoint. The cost lands on whichever execution path needs that model first. I take that trade. Cached matte work should stay cheap and inspectable, and loading every model for every run hides an orchestration problem behind hardware capacity.
2. The band uses image disagreement
The alpha refiner lives in its own module, and the one-line file description is the entire design: trimap, band crop, NSJ, or deterministic fallback.
A plain band can be built from the prior alone. Take the hard mask, grow it, shrink it, call the ring unknown. That catches soft contour error. It fails on a prior that is confidently wrong.
The comment in the band builder names that case in capitals: WRONG AND CONFIDENT. Both halves of a morphological band are functions of the prior, so a prior with no doubt produces no band at all. Fill in a wheel opening and the hole becomes confident foreground. Drop a roof antenna and those pixels become confident background. Either way the missing area can sit nowhere near an iso-contour, and a morphology-only band never asks the refiner to look there.
So CarSegNet adds an image term. Where the photograph shows strong structure and the prior shows nothing happening, that disagreement opens the band. The photograph says edge. The mask says flat. That argument is worth examining.
It stays bounded. The search is restricted to the subject's own neighborhood, the edge criterion is relative to the image instead of a fixed number, and the band has a ceiling it cannot cross.
Those limits cost something. Widen the neighborhood and foliage, fence lines, or lot texture start lighting up the image term. Tighten it and the antenna case stays frozen. The ceiling is the one I would defend hardest. It stops a local repair path from turning into a full-frame request, which means a badly wrong prior has to be rejected upstream instead of handed to the refiner as though it were close.
That is the transferable part. A learned correction needs a declared edit domain. Here the domain is a trimap band carrying a disagreement term, so confident prior mistakes get a chance to be examined near the vehicle.
3. Full-frame alpha drifts quietly
The tempting design is simpler. Segmentation produces a rough mask, a neural refiner outputs a full alpha matte, compositing uses that alpha. Fewer moving parts. Far more hidden authority.
A full-frame output can fix a tire edge and move a roofline in the same pass. It can smooth a window halo and erase a mirror. The bad part is that such a trade can improve an averaged score, because most pixels in a vehicle photograph are easy background or easy paint. The listing still fails at the one boundary a buyer looks at.
NSJ is small. Size is not the safety property, and I want to be exact about that, because small models get described as safe all the time. A small unconstrained model can still damage broad vehicle topology. The constraint around the output does the work.
The completed checkpoint audit is what stopped me from reading boundary gain as deployment clearance. Boundary scores improved with the trained checkpoint. The same audit showed the model almost never recovered enclosed openings. The outline got better. The topology did not.
That changes what the stage is allowed to claim. NSJ is an edge repair component. It is not evidence that the system understands window holes, wheel openings, cabins, or glass ownership. Those errors are structural, so they need separate gates.
The fallback path has the same shape. With no trained checkpoint the refiner runs a deterministic guided-filter route instead of stopping the pipeline at missing weights. The system stays runnable on day one. The tradeoff is visible in what each path can actually do. Guided filtering cleans a local alpha transition. Learned interior reasoning waits for trained weights and a passing topology gate.
I cut a synthetic hard-case metric table out of this argument while drafting it. The numbers were useful during development. Without the full setup sitting next to them they were decoration, and they pulled attention away from the contract. The code path is the stronger evidence: band crop, NSJ or fallback, guided filter, prior preserved outside the declared edit area.
4. A guard catches the silent sliver
The same module that routes the models also rejects one specific mask failure before it reaches compositing. There is a dedicated error type for mattes that are fragments rather than vehicles.
The default came out of measurement rather than taste. Sliver mattes cluster far below anything a real vehicle produces under the serving prompt. Legitimate vehicles start well above that cluster and run up to most of the frame. Between the two populations sits a wide empty band, and the default lives inside it. That gap is the only reason I trust a single scalar here.
It is a different kind of trust region than the band. The trimap band limits where alpha refinement may edit. The subject-fraction check limits which priors are allowed into the rest of the route at all.
The asymmetry is what justifies it. A miss is visible and recoverable. A tiny foreground sliver composites silently and looks like a strange crop, so it gets its own exception type while still subclassing the pipeline error every caller already handles.
There is a cost. Small distant vehicles, odd crops, and dealer photos with unusual framing can trip it. The threshold moves through configuration and can be switched off entirely. I keep the default because the measured gap is wide for this serving path, and in a media workflow a silent sliver is worse than a loud miss.
5. Glass got a smaller claim
Glass is where boundary language gets overloaded. A side window is transparent, reflective, tinted, and part of the vehicle shell, depending on which question you are asking. One binary mask answering all of them is false authority.
The bootstrap model predicts a glass region of interest from a single RGB vehicle image. Its scope is deliberately narrow: find the reviewed glass region for later work. The module's constants name the capabilities it does not have. That is useful friction. Future code has to cross a named boundary before it can pretend a region mask solved reflection removal.
The follow-on model has a stricter data contract than ordinary open and closed window classification, and the pilot captures do not satisfy it yet. Screening material is not the same as clearing it to train. The tooling says so in its own docstring, and the pilot is still marked not training ready.
The part that belongs in a post about the trimap band is the decode rule. The learned model answers a narrow question inside reviewed bounds. Every pixel outside that region is copied from the input, exactly. Same design pressure, different subsystem.
6. The composite makes excess authority visible
The final judgement happens after the vehicle lands on a plate. The compositor harmonises color in LAB, matches defocus, adds a contact shadow, and applies light wrap. Those operations can make a correct cutout sit convincingly in a new scene. None of them restores a mirror that alpha refinement erased.
This is where vague model authority turns expensive. Old asphalt left under a tire means contact shadow competes with captured ground. A halo that survives around a window means light wrap makes the bad edge look intentional. An antenna dropped before composition means every downstream operation is polishing a false matte.
Video adds one more constraint. The pipeline uses a depth-parallax plate before compose and encode, so there is one still plate with local motion rather than a generated background per frame. Camera freedom drops. Repeatability goes up. Same matte, same plate, same depth map, same frames again.
The selftest keeps model availability separate from pipeline correctness. It runs on CPU with no network, no GPU, and no model downloads, exercising image and video runs against stubbed backends. Cached mattes, compositing, encoding, and quality reports can fail or pass without waiting on gated weights.
That split matters for what comes next. Part 3 goes inside the vehicle, where the opening stage stays disabled until calibration and a locked topology gate pass. The problem there is harder than fuzz at the edge. Subtract alpha inside a guarded interior region while preserving the filled outer silhouette.
A vision system gets safer when every learned edit carries a boundary condition.
🎧 Listen to the audiobook — Spotify · Google Play · All platforms
🎬 Watch the visual overviews on YouTube
📖 Read the full 13-part series
