Triage

skill-be triage [skill...] looks closer at the failures of the latest run: the newest directory under results with a summary.json. It never blocks (exit code 0), but it needs a run to read, so run the benchmark first.

A failing criterion is one that caused a block in that run: a zero block (triaged on every model under test) or a regression block (on its model). A code test that caused one is triaged the same way. With --below <pct> it also takes criteria that scored under that percentage with the skill. The [ERROR] block names no criterion and is listed as not triaged.

For each failure it generates fresh with-skill samples (--samples, default 2), grades them asking the grader for its reasons, and asks a model to classify the failure. triage.model picks that model; by default it is the failing output’s own model under test, not the grader, since grader errors are one of the categories.

triage:
  model:
    adapter: claude-cli
    id: claude-sonnet-5-5

Each failure gets one proposed classification for a human to confirm:

Category	Meaning
skill gap	The skill does not teach it clearly enough
eval too strict	The answer is correct and idiomatic, but the criterion demands one form, or something the prompt did not ask for
grader error	The answer satisfies the criterion and the grader still says NO
model choice	The skill is clear and the model picks another reasonable approach

For a failing code test, test error replaces grader error: the test, not the answer, is wrong (Triage of code tests).

A failure whose samples could not be generated, or whose classification could not be read, is unclassified. The code, the grader’s reasons and the proposals are saved side by side in triage-<timestamp>.md inside that run’s directory, and the terminal lists the proposals:

Triage: 1 failure(s)
  money [split-the-bill] claude-sonnet-5-5 "The shares it returns always add up exactly to the total": grader error — the samples split with allocate, whose shares always add up to the total
Report: .skill-be/results/2026-10-31T16-20-05-112Z/triage-2026-10-31T16-23-40-518Z.md