Local generation on a Mac: where it is actually free, and where it costs two hours per second

Two hours and fourteen minutes of compute for one second of video. That is not a joke about running things on the CPU — it is a number from a log.

I spent a week putting the whole local generation stack through its paces on a single machine: Apple M5, 16 GB of unified memory, macOS 26.5, a locally built stable-diffusion.cpp. Stills, editing existing frames, video. The goal was to find out where "just run it locally, it's free" is honest advice and where it stops being an option at all.

Every number below comes from the log of a specific run.

First, kill the CPU myth

Because of the .cpp in the name, stable-diffusion.cpp gets filed under "the slow CPU version". The first line of the log says otherwise:

ggml_metal_device_init: GPU name: MTL0 (Apple M5)

That is the ggml Metal backend. The work runs on the GPU. The CPU gets one job here, and it is a humiliating one: finishing what is broken on Metal. That happens twice in this article.

The second thing that shapes everything: on Apple Silicon, memory is shared. A model has no private VRAM — it takes system memory, the same pool your browser and editor live in:

total params memory size = 9959.28MB (VRAM 9864.71MB, RAM 94.57MB)
  text_encoders     3395.09MB (VRAM)
  diffusion_model   6469.62MB (VRAM)
  vae                 94.57MB (RAM)

Ten gigabytes out of sixteen for one model. Everything else on the machine splits the rest. Most of what follows grows out of that single fact.

Stills: works, and genuinely free

Flux.1-schnell, 4-bit quant, 1024×512, four steps:

Ninety seconds per frame is a working pace. Not instant, but fast enough to iterate on composition without watching a billing counter. schnell is distilled — four steps is what it was trained for, cranking it to twenty buys nothing.

Now the trap that makes people conclude local generation is broken. One flag:

--vae-on-cpu

Without it, the Flux VAE decoder produces NaN on Metal and you get an empty white frame. The nasty part: there is no error in the log. It cheerfully prints save result image (success). You identify it by file size — a broken PNG is about 17 KB, a real one starts at 450 KB. The flag moves decoding to the CPU, and that is exactly the 24 seconds in the table above. A third of total generation time goes into working around a bug, and that is still the good news in this article.

The 29x memory cliff

Flux generates from scratch. Kontext, from the same family, edits: it takes an existing image and changes what you asked while keeping the rest. In practice it covers the one request a generator cannot serve — "keep everything, move this one thing." Ask a generator that and you get a new picture: different light, different furniture, different angle.

Then came the most instructive measurement of the week. Identical run — same model, same prompt, 768×432, eight steps, same flags — on the same machine:

Twenty-nine times, with nothing changed but the state of memory. Once the model runs out of physical RAM, every step starts going to disk and stops keeping up.

The practical rule is dull but saves hours: check sysctl -n vm.swapusage before starting a heavy model, and if swap is full, let memory settle instead of tuning steps and resolution. I spent two days convinced Kontext was hopelessly slow. It was queuing behind its own swap.

Video: three walls in a row

Then I tried to animate finished frames locally — even rough drafts would do. I took the smallest sane video model available, deliberately one that fits in memory. Three walls followed, each behind the previous one.

Wall one: 5 > 4. A video model convolves over time as well as width and height, so its first convolution kernel is a five-dimensional tensor. In ggml, the maximum tensor rank is four — a constant in the library core, not a build option:

gguf_init_from_file_ptr: tensor 'patch_embedding.weight'
  has invalid number of dimensions: 5 > 4

I assumed a bad conversion and downloaded a quant from a different repository by a different author. Identical failure — which is the signal that the file is not the problem. Official safetensors load fine through a different code path, at the cost of size: 4.1 GB instead of 1.7, which defeats the point of quantizing.

Useful corollary: updating the engine for this error is pointless. The limit is architectural.

Walls two and three: two unsupported ops. The model loaded, then the VAE encoder died with unsupported op — fixed by the same --vae-on-cpu flag as Flux, for an entirely different reason. Then the denoiser died for good:

ggml_metal_op_encode_impl: error: unsupported op 'IM2COL_3D'

Before blaming an old build, go read the source — which I did. IM2COL_3D does not exist in the Metal backend of either the fork stable-diffusion.cpp builds on or upstream ggml inside llama.cpp: zero mentions in ggml-metal-device.cpp and ggml-metal-ops.cpp in both repositories. The operation the model cannot run without was simply never written for Metal. Not "misconfigured", not "needs an update" — absent.

The CPU fallback, priced

That leaves --backend cpu: compute everything on the processor, around Metal and its holes. I ran it to close the question with a number instead of an opinion. Minimal task: one second of video, 512×288, 17 frames, 8 steps.

Note the spread: four to eighteen minutes per step, slower toward the end as the machine sinks deeper into swap.

The output was worse than the time. Composition drifted from the reference, objects disappeared and new ones appeared, there was almost no motion between first and last frame, and anatomy broke on limbs. Also worth knowing: in this mode the input image is a reference, not a locked first frame — the model treats it as a hint and owes you nothing. Locking first and last frames is a different mode that lives in 14B models, which is not a conversation you have with 16 GB of RAM.

Generation strangles your network

A side finding that first looked like a bad ISP. Downloading weights crawled at 2.5 KB/s, while the same link gave 1.6–3.3 MB/s on an idle machine. The culprit was local: a generation was running, swap was full, and the network died along with everything else.

Hence a rule that is not obvious until it bites: never run generation and downloads at the same time — chain them, weights first, compute second. And on downloading itself: hf download creates a new temp file on every broken attempt instead of resuming the old one, so on a flaky link it never finishes. After a series of attempts I had three stumps of one file: 0, 52 and 755 MB. What works is a resumable curl in a retry loop:

curl -sL -C - --speed-limit 51200 --speed-time 30 --max-time 600 -o "$F" "$U"

-C - resumes at the break, and --speed-limit with --speed-time kill a dead socket in 30 seconds instead of hanging in a half-hour timeout.

What "free" actually costs

No money, but it bills you in disk:

Nearly ten of those gigabytes are a paid lesson: weights for a model that will not render a single frame on this machine. Plus the time to download them, plus two hours of fan noise for one second of garbage. For reference, that second costs roughly $0.10 (480p) to $0.23 (720p) in the cloud at public Seedance 2.5 pricing — and comes back usable.

Where I landed

• Local: drafts and idea checks — composition, angle, "does the joke read". Ninety seconds a shot, no billing counter. The cover image of the original article was made exactly this way: local draft to approve the concept, paid generation for the final.
• Local: editing an existing frame with Kontext. Two minutes for something a generator will not do at all.
• Local: anything that should not leave the machine.
• Paid: final quality. A 4-bit quant on 16 GB loses detail, and a cover is what people see before the text.
• Paid: video, entirely. Not because local is more expensive, but because on this hardware local does not exist.

One direction I have not closed: MLX, Apple's tensor framework for unified memory. It runs outside ggml and its limits, the Flux port is mature, video ports exist but are raw. If I come back to local video on Apple Silicon, it will be through MLX, not through the path in this article.

If there is one line to take away: local generation is not a free replacement for the cloud, it is a different tool with its own domain. It covers drafts, iteration and privacy very well, and final quality and video not at all. Both slogans — "why pay for anything" and "it doesn't work on a laptop" — get in the way equally.

Originally published at klukyanov.ru.

Shorter weekly write-ups (in Russian) — on Telegram.