Evals are becoming the FDE's definition of done, and a contract term too

FDE Times covers an argument from FDE Hub that evaluation pass rates are becoming the definition of done for AI deployment work — not only an engineering metric, but a term both vendor and customer commit to. That replaces the informal “it is running and the customer is using it” bar many teams still use.

PostHog’s FDE handbook lands in the same place: every engagement needs a measurable outcome, a rough timeline and a shared definition of done before delivery starts. Scope that doubles becomes a new quote, not an informal extension. Anthropic’s framing of an eval — give an input, grade the output — and its emphasis on regression suites sit alongside the warning that a high pass rate on a stale suite can mislead both sides.

Hamel Husain’s guidance, cited in the piece, is to start from real logs rather than inventing a rubric first: define failure from what the system actually gets wrong, then turn those failures into cases. FDE Academy, the article notes, already grades FDE performance on a documented harness rather than a demo that runs.

For candidates, that means describing projects with suite size, log sources, pass-rate movement and who signed off on “done” — and asking interviewers who decides when an engagement is finished.