Measuring LLM Quality: A Practitioner's Guide to Evaluation Metrics That Actually Matter

When teams first deploy a large language model, they almost always reach for the same evaluation metrics: BLEU score, perplexity, ROUGE. These metrics come from the academic NLP tradition and they measure something real — but for most production use cases, they measure the wrong thing entirely.

Why Academic Metrics Fail in Production

BLEU score measures n-gram overlap between generated text and reference text. It was designed for machine translation, where there's a relatively correct answer. For open-ended generation, summarization, or question-answering, a response can be completely correct while scoring poorly on BLEU — and completely wrong while scoring well. Perplexity measures how surprised the model is by a sequence of tokens. A lower perplexity means the model finds the text more probable — but probable text is not the same as correct text.

The Metrics That Actually Matter

• Factual accuracy rate: On a domain-specific benchmark of verifiable claims, what percentage does the model get right? Build this benchmark from your own knowledge domain.
• Hallucination rate: How often does the model assert something confidently and incorrectly? Measure separately for questions within the model's knowledge domain vs. outside it.
• Instruction following score: Given a set of constrained instructions (format, length, tone, content restrictions), what percentage of outputs satisfy all constraints?
• Consistency score: Ask semantically equivalent questions phrased differently. How often does the model give consistent answers? Inconsistency is a reliability red flag.
• Refusal precision and recall: Does the model correctly refuse harmful requests (recall) without over-refusing legitimate ones (precision)? Both matter equally.

Building Your Evaluation Dataset

The most important investment you can make in LLM quality is a curated evaluation dataset specific to your use case. Generic benchmarks like MMLU or TruthfulQA are useful for baseline comparisons — they tell you how a model performs on average across domains. But they don't tell you how it performs on your documents, your user queries, your edge cases. Build 200–500 hand-labeled examples from real usage data. Review and update them every quarter. This dataset becomes your most valuable QA asset.

Human Evaluation at Scale

Automated metrics are fast and cheap. Human evaluation is slow and expensive. You need both. Use automated metrics for regression testing — catching regressions between model versions quickly. Use human evaluation for calibration — periodically verifying that your automated metrics are still tracking what you actually care about. A common failure mode: teams optimize their automated score, only to discover later it had diverged from real-world quality.