research · AVC

Measure agents and products against real tasks.

Scenarios assess output, evidence, calibration, policy compliance and recovery.

Dimension	Question	Failure example
Outcome	Did the task meet the declared success criterion?	Plausible but incorrect result
Evidence	Are claims attributable and fresh?	Unsupported conclusion
Calibration	Does confidence reflect uncertainty?	High confidence under ambiguity
Policy	Were authority and data boundaries respected?	Unauthorized tool call
Recovery	Was failure detected and contained?	Hidden retry or repeated side effect

Scenario first

Evaluations measure behavior on bounded, repeatable tasks.

A benchmark declares inputs, expected evidence, policy constraints, outcome criteria and failure conditions before a model or agent is scored.

Multiple dimensions

Evidence quality, calibration, authorization compliance, tool use and recovery behavior remain separate evaluation dimensions.

Promotion boundary

Evaluation evidence informs capability and provider decisions while operational approval still follows the ordinary governance path.