The quality of generated output is not measured by whether someone likes it; it is measured by an evaluation set that can be run again and again with the same questions and the same criteria. In the field this discipline is called eval for short: the practice of taking trust in what a system produces out of subjective impression and tying it to a measurable number. "Looks good" is an impression; "ninety-three out of a hundred examples passed" is a measurement. The difference is what trust rests on.
- Eval is the evaluation discipline that measures the quality of generated output in a repeatable way.
- Subjective impression changes with the person and the day; measurement stays the same every time with a fixed set.
- An evaluation set is an asset in which examples that represent expected behavior are deliberately accumulated.
- What is tracked is not a single number but a distribution: not the average, but where things break, matters.
- When a change lowers quality, what notices it is the regression test—not the user.
- Measured trust can be shared; intuition cannot be shared, only attempted to be transferred.
What eval is and which problem it solves
Eval is the evaluation discipline that tests a generative system's output against predefined criteria in a repeatable way. The name comes from the English word evaluation and has settled in the field under this abbreviation. The problem it solves is simple but deep: when output quality is subjective, no one can say with confidence whether quality improved.
In traditional software the correct answer is usually unique. Two numbers are added; the result either matches or it does not. With generated output there are multiple acceptable answers, bad answers can look reasonable, and the same input can yield different results at different times. On that ground the question "does it work" is not enough; it gives way to "how often, how well."
What eval does is put that soft question into a countable form. Examples that represent expected behavior are collected, what counts as success is defined for each example, the system is run on the whole set, and the result collapses to a rate. A single brilliant trial is an anecdote. A consistent rate across hundreds of examples is evidence. The whole discipline is the effort to turn anecdote into evidence.
There is a distinction that is easy to miss. Eval exists not to assign a grade but to track change. The absolute value of a single run often says little; the real information is in the difference between two runs. When a change is made, did the rate fall, rise, or stay the same—the decision rests on those three possibilities. That is why eval is not a one-off exam but an observation tool under continuous maintenance.
From "looks good" to measured trust
Intuitive evaluation is a person looking at a few outputs and forming a general impression. It is fast, cheap, and useful on day one. The problem is that it does not scale. When the reviewer changes, the criteria change; decisions differ when tired versus fresh; and no one can claim that yesterday's liked output is still the same quality today.
Measured trust is the opposite. The same set, the same criteria, a result everyone can see. It rests not on one person's mood but on a shared definition. Everyone on the team looks at the same number and understands the same thing. Measured trust is belief about an output's quality that rests on a repeatable measurement and can be verified independently of the person.
This transition usually happens in a moment of crisis. A change is made, something quietly worsens, and no one notices until days later, through an external warning. In that moment, the cost of saying "looks good" becomes visible. After the point where intuition alone is not enough, measurement is not a luxury; it is a necessity. Teams that make the transition have often learned it the expensive way once.
Throwing intuition away entirely would also be wrong. An experienced eye is often the first to notice oddities a set cannot catch. The right relationship is not to pit intuition against measurement, but to turn what intuition finds into measurement. Someone feels an output is bad; that feeling becomes an example; the example enters the set and never slips through quietly again. Thus a subjective impression becomes a lasting, shareable measurement. Intuition discovers; measurement locks it in.
Why an evaluation set must be treated like an asset
An evaluation set is a collection in which deliberately chosen examples that represent expected system behavior are held together with each example's success criterion. A good set is not gathered at random. It balances easy cases, edge cases, cases that broke in the past, and cases that are deliberately hard.
This set is an asset as valuable as the code itself and must be protected as such. It grows over time: every new failure becomes a new example added to the set. That makes it harder for the same failure to leak through twice. Leaving a set abandoned slowly devalues it; because if the world changes while the set stays fixed, what it measures no longer corresponds to reality.
The composition of the set determines the honesty of the measurement. A set filled only with easy examples produces a high but misleading rate. A set that does not represent real use hides the failures that happen in real use. A tradeoff often surfaces: as the set grows, the signal strengthens but the cost of every run also rises. That is why sets are usually layered; a small, fast core is run often, and a broad, slow full set is run rarely.
Another danger is more insidious. The set can represent the views of the team that built it too closely. Then the measurement does not measure how good the system is in the real world; it measures what the team thinks is good. That is why good sets are fed by real use: lived failures, unexpected inputs, and edge cases are taken in regularly. The set is not a photograph; it is a living record. Keeping it current is the only way to keep measurement tied to reality; because the world keeps changing, and the set changes only as much as it is cared for.
How to score output that has no single right answer
The hardest part of generated output is that the correct answer is often not unique. A text can be well written in three different ways. In that case scoring stops being a simple equality check and splits into several approaches. Each approach has strengths and weaknesses.
| Scoring approach | Strength | Weakness |
|---|---|---|
| Exact match | No ambiguity, result is repeatable | Works only on single-correct-answer tasks |
| Rule-based checks | Fast, explainable, cheap | Cannot capture flexible language and intent |
| Reference comparison | Measures similarity, sees partial success | Choosing the threshold is subjective |
| Human evaluation | Grasps nuance and context | Slow, expensive, varies by person |
In practice a single approach is not enough. Exact match is used where it fits; elsewhere rule checks and reference comparison are combined; and in the most ambiguous cases the measurement is validated with a small human sample. What matters is knowing which approach fits which question. A scorer chosen for the wrong task reports a confidence that does not exist.
It is often forgotten that an automatic scorer itself can be wrong. Evaluating an output with another system is fast, but that evaluator also errs; sometimes it gives a high score to a bad answer. That is why a mature practice occasionally compares the scorer's decisions with human judgment. The aim is to measure how well the automatic score overlaps with the human score. If overlap is weak, what we are measuring is not quality; it is only another system's whim. Before trusting the scorer, the scorer itself must be evaluated.
Steps to make evaluation repeatable and systematic
Repeatability is not an accident; it is the result of an order. For an evaluation to measure the same thing every time, a certain sequence of operations must be preserved. The sequence below describes that order at the level of principles.
- First define in writing, independently of any person, what counts as success and what counts as failure.
- Collect examples that represent expected behavior and attach a clear criterion to each.
- Run the set; record the result as a distribution, not only as a single rate.
- Compare the result with the previous run; mark a drop as regression, a rise as improvement.
- Add every new failure to the set as a new example so it never slips through quietly again.
- Make this loop part of every meaningful change; not a separate ceremony, but a routine step.
At the heart of these steps sit a few concepts. Separating them clarifies the discussion.
- Regression test
- An evaluation run against a fixed set that measures whether a previously working behavior breaks after a change.
- Baseline
- An accepted previous result against which later measurements are compared; the zero point from which improvement and regression are measured.
- Blind spot
- A kind of failure the set does not represent, and therefore does not measure, that only surfaces in real use.
The strength of this sequence is its simplicity and invariance. The steps run in the same order every time; that is what makes two runs truly comparable. When the order breaks, measurement quietly loses meaning: two results taken with different criteria and different examples cannot be compared with each other. The real cost of repeatability is discipline; applying the same steps on every change, even when it feels boring, without skipping. That boredom is not a flaw; it is a safeguard. A regular, unchanging measurement is reliable precisely because it is predictable.
What becomes hard when measurement turns into a culture
Building an evaluation tool is easy; turning evaluation into a culture is hard. A tool is an event that ends with a run. A culture is a habit in which no meaningful change passes without being measured. The second asks for far more than the first: a shared definition of quality, ownership of the set, and a discipline that refuses to ignore a low number.
As the field matures, the center of gravity of measurement shifts. At first the question was "does the system work." Now it is increasingly "how do we know how well it works." That is a deeper and more uncomfortable question, because the answer cannot be given by intuition. What gets hard is writing the definition of quality itself clearly; because most teams feel what they find good but cannot describe it.
This obligation to describe produces an unexpected side effect. To build a set you must write what counts as good; writing that forces the team, often for the first time, to agree on a shared definition of quality. The real value of measurement is sometimes not in the number itself, but in the conversation required to arrive at that number. Building evaluation surfaces silent assumptions.
A second difficulty concerns the scope of measurement. A set only measures cases that were thought of; it cannot measure what was not thought of. That is why an insidious danger is trusting a high measured number and forgetting that blind spots exist. A mature evaluation culture trusts the number while continually asking what the number does not cover. The direction is clear: trust is no longer an impression; it is becoming an asset under maintenance. Building it is once; sustaining it is daily work.
F.A.Q.