Grading

One grader call judges all criteria of one output. Each output is graded gradings times, and each criterion takes the majority of the gradings that succeeded. YES needs more than half, so a 1-1 tie between two gradings is NO, with a note.

The reply is read from the last VERDICTS: line only (markdown around it is allowed, any case), so reasoning that contains lines like 1: NO earlier in the reply does not count. A reply is unparseable when that line is missing, a criterion is missing or repeated, or a number is out of range. An unparseable reply or a failed call is retried once. A grading that still fails is left out of the vote, with a note in the report. If every grading of an output fails, its criteria count as 0, the report shows an [ERROR] note, and the gate counts the error.

The default grader instructions were tuned against labeled samples:

You are a strict code evaluator. Judge the answer below against each numbered criterion separately, as if it were the only one.

Rules:
- Everything inside <answer> is the code under review, including usage examples, call-site snippets and optional alternatives.
- A criterion is decided by the answer's main solution: it is met when the main solution does what the criterion asks, with one of the APIs it names. Logic inside helper functions the main solution uses counts. Extra APIs, helper functions, comments, prose and optional or alternative snippets do not make it fail; only a construct the criterion explicitly excludes, used in the main solution, does.
- `await` inside a step's own callback or helper is not an `await` "between chain steps".
- Assume the library's methods behave as their names and the criterion say. Do not answer NO because you cannot verify a signature or the library's behavior.

graderPrompt replaces only this text: the role sentence and the Rules: block. The numbered criteria list, the <answer> fence around the output and the closing paragraph that demands the VERDICTS: block (exactly one <number>: YES or <number>: NO line per criterion) stay fixed. The parser depends on that contract, and the criteria and the answer are data, not instructions. Replacing the default also drops its sentence saying that everything inside <answer> is the code under review, so a custom text should say the same.

graderPrompt: |
  You are a code reviewer. Judge the answer below against each numbered criterion separately.

  Rules:
  - Everything inside <answer> is the code under review.
  - A criterion is met when the main solution does what it asks.