If you read one engineering post from us this year, please make it this one. Most LLM features we’ve been asked to rescue did not fail because the model was wrong. They failed because nobody was measuring the right thing.
The pattern we see.
An LLM feature works beautifully in the demo. It works in the staging environment. It works for the four prompts in the QA spec. Then it hits production, where users do things QA never imagined, and quality quietly slides. Six weeks later, somebody on the leadership team types something embarrassing into the box, takes a screenshot, and the feature goes back into the backlog.
There is a known answer to this and it is dull. You build evaluations the way you build tests. You have a golden set. You have a regression suite. You run it on every change to the prompt, the retrieval pipeline, the model, the data. You publish the deltas. That’s it.
A starter budget.
- 200–500 curated golden examples per task, hand-labelled by someone who actually does the work — not a prompt engineer guessing.
- A regression suite that runs in CI within three minutes for the cheap tier, fifteen for the expensive.
- Per-task quality bars expressed as actionable numbers — ‘94% of customer billing questions answered correctly without escalation’, not ‘high-quality answers’.
- Human review on regressions, with clear rubric. We use a three-tier rubric and pay reviewers per item, not per hour.
What this buys you.
Confidence to ship — and the kind of confidence you can defend to a regulator. When we moved a 20-million-customer lender onto an LLM-assisted underwriting assistant, the deciding artefact in the regulator meeting was the eval harness output, not the architecture diagram.
“The eval harness was the artefact that closed the regulatory file. Not the model card.”
What it costs.
One engineer-week of setup, half an engineer-week per month of upkeep, plus the labelling cost (typically a few hundred dollars at our volumes). For features that touch real customers, that is the cheapest insurance you will buy this year.