Last Tuesday, my pipeline called a free model three times on the same commit.
The output looked confident every time.
It was wrong every time.
The model did not fail. My prompt did.
I almost blamed the model. Then I looked at the job log. The same ambiguous instruction went out again and again. Every retry was a fresh token budget hit on the same mistake.
That made me ask a simple question: why was I paying attention to model output before I had checked the prompt?
The invisible cost is not the model. It is the bad call.
A free model still has a cost.
- Time waiting for a response.
- CI minutes spent on a job that should not exist.
- A false verdict that someone has to review manually.
- Confusion about whether the model or the input caused the failure.
MonkeyCode includes free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Free access can hide waste. You stop paying per token, so you stop noticing how many calls you repeat.
The fix is not a better model. It is a smaller gate before the model.
Treat prompts like source, not like glue
Most teams version their code.
Most teams do not version their prompts with the same care.
A prompt is a contract between you and the model. If the contract is vague, the output is vague. If the contract changes silently, the CI result becomes unpredictable.
So I started linting prompt files before any model call.
The rule is simple:
- The prompt must have a role.
- The prompt must state constraints.
- The prompt must define an output format.
- The prompt must be long enough to be specific.
- The prompt must not rely on phrases like do your best.
That is not a semantic review. It is a static contract check. It catches the cheap mistakes first.
A prompt contract linter you can run in GitLab CI
Here is the small linter I keep in the repository.
#!/usr/bin/env python3
import sys
from pathlib import Path
REQUIRED = ['role', 'constraints', 'output_format']
FORBIDDEN = ['do your best', 'be concise', 'be smart']
MIN_LEN = 200
MAX_LEN = 4000
def lint_prompt(path: Path):
text = path.read_text()
lower = text.lower()
problems = []
for section in REQUIRED:
if section not in lower:
problems.append(f'missing section: {section}')
for phrase in FORBIDDEN:
if phrase in lower:
problems.append(f'ambiguous instruction: {phrase}')
if len(text) < MIN_LEN:
problems.append('prompt is too short to specify a contract')
if len(text) > MAX_LEN:
problems.append('prompt is too long; split into sub-prompts')
return problems
def main(paths):
failed = False
for raw in paths:
path = Path(raw)
problems = lint_prompt(path)
print(f'{path.name}: {len(problems)} problems')
for problem in problems:
print(f' - {problem}')
failed = True
sys.exit(1 if failed else 0)
if __name__ == '__main__':
main(sys.argv[1:])
This runs fast. It does not call a model. It tells you whether the prompt deserves a model call.
A prompt file might look like this:
# prompts/diff_review.md
role: Review small diffs for obvious regressions.
constraints:
- Do not invent code.
- Cite the exact file and line.
- Say unsure when the change is too large.
output_format:
verdict: ok | needs-work | unsure
reason: one sentence
The linter checks structure, not correctness. That is exactly the point.
Put the linter before the model call
The GitLab CI job is small.
prompt-contract:
stage: test
image: python:3.12-slim
script:
- python scripts/prompt_contract.py $(find prompts -name '*.md')
rules:
- changes:
- prompts/**/*
If the prompt file changes, the contract job runs. If it fails, the pipeline stops. No model call happens.
That order matters.
- The lint job is deterministic.
- The lint job uses no model quota.
- The failure message points at the prompt, not the model.
- The model job only sees prompts that passed a basic contract.
Where a free server option fits
You do not need to install Python inside every runner.
If you have a free server option, deploy the linter as a tiny HTTP endpoint. Then the CI job becomes one request.
prompt-contract-remote:
stage: test
script:
- curl --fail -X POST http://contract-check.internal/check --data-binary @prompts/diff_review.md
rules:
- changes:
- prompts/**/*
The contract file still lives in the repo. The checker runs somewhere stable. The CI job stays thin.
I use the local version in small projects. I reach for the server version when the runner image gets messy or when multiple pipelines need the same check.
Neither version makes the model smarter. Both versions stop the model from being asked a bad question.
Limits of this approach
Static prompt linting is not magic.
- It catches missing sections, not missing judgment.
- A prompt can pass and still produce the wrong answer for the domain.
- A hash gate notices prompt changes, not model behavior changes.
- It does not replace human review for code that can affect production.
I do not rely on this linter for correctness. I rely on it to stop the obvious waste.
Who should skip this
Skip this if you have one prompt and call it rarely.
Skip this if you need a model result in every pipeline no matter what.
Skip this if your prompts are generated at runtime and never stored in the repo.
Skip this if your team already reviews prompts in a separate tool.
The overhead is small, but it is not zero. Another file in the repo is still another thing to maintain.
What I changed this week
I stopped treating model calls as the unit of work.
Now the unit is the prompt contract.
First I lint the contract.
Then I decide whether to call the model.
Then I interpret the output against the contract.
That sequence removed most of the repeated failed calls from my CI runs. The remaining model failures were real problems, not vague instructions.
A free model will not fix a bad prompt. But a cheap lint gate can keep you from feeding that bad prompt to the model five times before lunch.
What would you check first: the prompt or the output?
