Evals are repeatable tests that check how well an AI application does its job.
Example
Before every update, you test your support bot with the same 100 questions and compare the hit rate.
For pros
Unlike public benchmarks, evals test your own task with your own data. Scoring is automatic, by people or with LLM-as-a-judge. Good evals include hard edge cases and run on every change to prompt, model or data.
