Running LLMs inside macOS virtual machines on Apple Silicon has been painfully slow — until now. A team at Cua has discovered that the performance bottleneck isn't the hardware or the hypervisor, but a conservative GPU capability report that forces llama.cpp into slow code paths. Their fix delivers 11-16x faster inference, bringing VM-based LLM performance to nearly bare-metal speeds.
The Problem
Apple's Virtualization.framework presents macOS guest VMs with a virtual GPU backed by the host's Apple Silicon GPU. This is paravirtualization — the host keeps control of the hardware and the guest submits Metal work through a virtual driver.
The problem: the virtual GPU reports a conservative Metal capability profile. In a stock macOS Tahoe VM, the device reports roughly an Apple 5-era GPU family, 32 KB maximum threadgroup memory, and no SIMD-group matrix support. Modern Metal applications use these capability answers to select kernels — so llama.cpp, seeing limited capabilities, selects slower code paths even though the underlying hardware can do much more.
This is different from traditional GPU passthrough on x86 Linux, where VFIO can assign a physical PCI device directly to a VM. Apple's Virtualization.framework doesn't offer true GPU passthrough — it always goes through the virtualization layer, which reports whatever capabilities Apple chose to expose.
The Fix
The Cua team built a process-scoped Metal capability shim — a compatibility layer that intercepts selected Metal capability queries and changes the answers returned to the guest process. The shim:
- Changes the reported Apple GPU family from ~5 through Apple family 9 (1009)
- Raises the reported maximum threadgroup memory from 32 KB to 64 KB
- Enables SIMD-group reduction, SIMD-group matrix, and bfloat16 paths in llama.cpp
This is not a system-wide modification. It's a process-scoped shim that only affects the specific process being optimized. Other guest processes see the stock capabilities.
The Results
The performance improvements are dramatic:
M1 Ultra with TinyLlama 1.1B:
- Prompt processing: 11.08x faster (reaching 98% of bare-metal speed)
- Token generation: 16.36x faster
M1 Ultra with Gemma 4 12B QAT Q4_0:
- Prompt processing: 7.20x faster (reaching 99.59% of bare-metal speed)
- Token generation: 14.54x faster (reaching 94.82% of bare-metal speed)
M1 Ultra with Muse Glimmer 30B:
- Prompt processing: 7.55x faster
- Token generation: 8.87x faster
In every case, the unlocked VM reached 94-99% of bare-metal performance. The capability shim essentially eliminates the virtualization penalty for LLM inference on Apple Silicon.
Why This Matters
This has significant implications for several use cases:
CI/CD for ML: Running LLM inference tests inside VMs is now practical. Previously, the 11-16x performance penalty made VM-based testing painfully slow for any non-trivial model.
Local AI agents in VMs: Developers who want to run AI agents inside isolated VMs (for security, reproducibility, or multi-tenant environments) can now do so without massive performance penalties.
Development environments: Tools like Cua and Lume that provide macOS VMs for development can now offer near-native LLM performance, making VM-based development workflows viable for AI workloads.
Security isolation: Running untrusted AI agents inside a VM with near-native LLM performance means you get security isolation without the performance tax that previously made this approach impractical.
The Broader Lesson
The most interesting aspect of this work is what it reveals about virtualization performance. The bottleneck wasn't CPU overhead, memory translation, or GPU scheduling — it was a few bytes of capability information that caused applications to select suboptimal code paths.
This suggests that other virtualization performance problems might have similar root causes: not fundamental architectural limitations, but conservative capability reporting that prevents software from using the full power of the underlying hardware.
For anyone running LLMs on Apple Silicon — whether bare metal or in VMs — this work is worth following. The capability shim is released under a permissive license, and the team is actively mapping which Apple Silicon chips, macOS releases, and Metal workloads benefit from the unlocked capabilities.
The era of "VMs are too slow for AI" may be coming to an end.
