LearnAI
Module 7 · Evaluating AI Quality (Pro) · Lesson 1/5
← Back to catalog
Task

Test set (eval set)

How to assemble an eval kit and why? Return JSON {"steps": [list of 2+ steps], "why": "short"}.

Unlock access to submit solutions for instant AI review.Start for free
AI quality foundation
💡 A bit of theory

Eval set is a set of representative examples with the expected result, on which you run the system before each change. Without it, improving the prompt is a fortune-telling: fixed one case, quietly broke five. How to collect: take real (or close to real) inputs that cover typical and edge cases; record the correct or acceptable answer for them; run the entire set before and after each edit and compare the completion rate. Even 20-50 good examples radically change the development - regressions are visible. Rule: without a set for evaluation, you do not improve the system, but only move errors from place to place.

How it's graded · passing score 70

  • 1Valid JSON without text around it30%
  • 2There is an array steps and why35%
  • 32+ steps: collect examples, record the answer, run before/after35%
Your promptClaude ⌄