A 27B Model Fit on an 8 GB GPU. It Was Slow.

I wanted one fact from the 8 GB card I use as the sizing desk truth node. Would Qwen3.8-27B load at all? It did. Then it wrote 64 tokens at 2.07 tokens per second. That is a fit. It is not a model I would sit in front of.

Short answer: Fit and usable are different measurements. On 2026-08-15, Unsloth Qwen3.8-27B-GGUF:Q3_K_S used 7,435 of 8,192 MiB on an RTX 3070 and finished a 64-token generate in 47.042 seconds. Decode ran 2.07 tok/s. I would not put an interactive coding agent on that row.

Canonical URL: https://bmdpat.com/blog/qwen38-27b-q3ks-8gb-fit-2026

Did a 27B model actually fit on 8 GB?

Yes, at one quant, one context, and one output cap. The host was FLUARMN. The GPU was an NVIDIA GeForce RTX 3070 with 8,192 MiB. The runtime was Ollama 0.32.3. The artifact was hf.co/unsloth/Qwen3.8-27B-GGUF:Q3_K_S, 27.3B parameters, 13,331,322,630 bytes on disk.

I set num_ctx=2048 and num_predict=64. Temperature was 0. The prompt asked when a llama.cpp user should choose GPU-layer offload versus tensor splitting. The model consumed 86 prompt tokens and emitted 64 output tokens, which is the cap, not a finished answer.

VRAM during the run was 7,435 MiB used of 8,192 MiB. Ollama reported size_vram of 5,470,064,474 bytes. After an explicit stop, the card returned to 1,147 MiB used. That last number is why I still unload local models after a test. A fit that never leaves the card poisons the next measurement.

What speed did I get?

The Ollama generate fields, not a review site.

2.07 tokens per second is about 0.48 seconds per token. That is not a typing-adjacent rate. Sixty-four tokens took 30.913 seconds of decode. The same wall-clock lesson showed up when I compared tokens per second to the real run time. A rate column can look tidy while the human still waits.

This run also hit the output cap. The 64 tokens are a ceiling, not proof the model finished the job. I treat a num_predict hit as truncated unless the row is a fixed-length rate test.

Why is fit not enough?

I already had a VRAM-before-download rule. Check VRAM fit before you pull the weights. This row is the next check. A model can land inside 8 GB and still be the wrong tool.

Q3_K_S is an aggressive quant. Context was 2,048, not a long agent window. Output was capped at 64 tokens. The 5090 leg never ran, because that host refused SSH during the pass. I am not inferring a 32 GB number from an 8 GB number.

If I published "27B runs on 8 GB" and stopped there, a reader would expect a usable desk. The receipt says otherwise. Fit answers "did it load." Usable answers "would I wait for this on a real task." Those are two rows.

What do I record from a fit test now?

Every first-party fit row carries the same fields I wrote down on 2026-08-15.

1. Exact model tag and digest. This one was 1555ef967520ea544f80aef940e52bc430878d2ee164d4282d71cddc1e0de2d2.
2. Host, GPU, and total VRAM. Not "an 8 GB card." The 3070, 8,192 MiB.
3. Runtime and version. Ollama 0.32.3 here.
4. num_ctx, num_predict, temperature, and the prompt itself.
5. Prompt tokens, output tokens, and whether output hit the cap.
6. Load, prompt-eval, decode, and wall times, plus the two rates Ollama computed.
7. VRAM during the run and VRAM after an explicit unload.

Skip field 7 and the next model inherits a dirty card. Skip field 5 and a truncated row looks complete. Skip the digest and you cannot prove which file you timed.

When would I use this 27B on 8 GB?

For a one-shot fit check. For a batch job I can leave overnight. Not for an interactive coding agent. Not for a writer I have to watch. Not as a 5090 substitute. The 5090 measurement is still missing, and this row does not stand in for it.

The useful product of the run is the bound. On this host, this quant, this context, and this cap, Qwen3.8-27B fits and crawls. That is enough to keep it out of the live desk until a faster card, a smaller quant path, or a shorter model proves a better rate.

Accompanying prompt

What the prompt does: This prompt scores a local LLM fit test so a loaded model is not treated as a usable model.

Copy/paste this prompt:

Role: Local LLM fit-test reviewer

Context:
I will give you one first-party run with model tag, digest, host, GPU,
total VRAM, runtime version, num_ctx, num_predict, prompt tokens,
output tokens, decode tok/s, wall time, VRAM during the run, and
VRAM after unload.

Task:
1. Say whether the model fit in VRAM.
2. Say whether the output hit the cap.
3. Mark the row fit-only, usable, or unknown.
4. List the missing fields before this row can change a model route.

Output:
- One verdict line: fit-only, usable, or unknown.
- The evidence you used.
- The exact rerun settings if the row is truncated or incomplete.

Constraints:
- Do not read a successful load as a usable rate.
- Do not infer a second GPU from this row.
- Say unknown when decode tok/s or VRAM-after-unload is missing.

Copy the block above.

Get the next measured local AI run in the 5090 Reports email list.

Get the local AI lab notes (benchmark rows, VRAM fit, quant choices, what runs on consumer GPUs), M-F only when there is something worth sending: https://bmdpat.com/newsletter?utm_source=blog_md&utm_medium=aeo&utm_campaign=qwen38-27b-q3ks-8gb-fit-2026

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.