The 26B Model Hit the Cap. The 8B Finished.

I ran the same code task through two local models on the RTX 5090. One finished the answer. One got cut off at the output cap. The speed column did not say which was which.

Short answer: Record done_reason with every local LLM benchmark row. It tells you whether the model stopped because it was done or because it hit your output cap. A row that ended at the cap is a truncated answer, not a completed task.

Canonical URL: https://bmdpat.com/blog/local-llm-benchmark-stop-reason-2026

What is done_reason in Ollama?

done_reason is a field Ollama returns with a generation response. It says why the model stopped. Two values matter for a benchmark. stop means the model ended the answer on its own. length means the runtime cut the answer off at num_predict.

The Ollama generate API documents the field. Most benchmark tables drop it. Mine did too, until I read a row that looked good and was not.

What did the two models actually do?

Both runs used Ollama 0.31.1 on the RTX 5090, Q4_K_M weights, num_ctx=4096, num_predict=512, and temperature=0. Same workload, same cap.

Read the speed column alone and you get a 10% gap between two working models. Read the stop reason and you get a different result. The 8B model wrote 459 tokens and ended the answer itself. The 26B model was still writing at 512 tokens and the cap ended it.

Why does a truncated row break a benchmark?

A benchmark row exists to answer a question. On an agent code task the question is whether the model produced usable output. A cut-off answer cannot answer it. The tokens were fast and the task still failed.

It also breaks the comparison. The 26B model produced 512 tokens because that was the ceiling, not because that was the answer. I do not know how long its answer would have run. I cannot compare 459 finished tokens against 512 unfinished ones and call the difference a result.

Is a length stop always a failure?

No, and this is where I had it wrong first. Five of my six rows from that run ended with length. Four of them were supposed to.

The short-generation and long-context rows fix the output length on purpose. I want a generation rate at a known token count, so I set the cap and the cap fires every time. A length stop there is the design, not a fault.

The rule is about intent. If the row measures a rate at a fixed length, length is expected. If the row measures whether a model completes a task, length means it did not.

What do I record now?

Every measured row carries the cap and the stop reason together. Without the cap, length has no meaning. Without the stop reason, the cap has no result.

1. The output cap I set (num_predict).
2. The tokens the model actually produced.
3. The done_reason the runtime returned.
4. Whether the row is a fixed-length rate test or a task-completion test.

Field four is the one people skip. It is what turns length from noise into a verdict. I keep it beside the speed and power fields in the local LLM benchmark decision record.

How does this change model selection?

It stops one class of bad route. A model that hits the cap on your real task is not ready for that task, however fast it generates tokens.

I saw the same shape when I compared generation rate to wall time. A high rate can sit inside a slow or incomplete run. That post is the faster local model run that took 83x longer. Different field, same lesson. One number never carries the whole result.

Raise the cap and rerun before you judge the bigger model. Then compare finished answers against finished answers.

Accompanying prompt

What the prompt does: This prompt checks a local LLM benchmark row for a truncated answer before the row is used for a model decision.

Copy/paste this prompt:

Role: Local LLM benchmark row reviewer

Context:
I will give you one benchmark row with a model tag, runtime version,
output cap, generated token count, tokens per second, and a stop reason.

Task:
1. State whether the row is a fixed-length rate test or a task-completion test.
2. Say whether the model stopped on its own or hit the output cap.
3. Mark the row valid or truncated for its stated purpose.
4. List the missing fields before this row supports a model decision.

Output:
- One verdict line: valid, truncated, or unknown.
- The evidence you used.
- The exact rerun settings if the row is truncated.

Constraints:
- Do not read tokens per second as task success.
- Do not treat a cap hit as failure on a fixed-length rate test.
- Say unknown when the stop reason is missing.

Copy the block above.

Get the next measured local AI run in the 5090 Reports email list.

Get the local AI lab notes (benchmark rows, VRAM fit, quant choices, what runs on consumer GPUs), M-F only when there is something worth sending: https://bmdpat.com/newsletter?utm_source=blog_md&utm_medium=aeo&utm_campaign=local-llm-benchmark-stop-reason-2026

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.