IECQ003: unscored_reason

Score.unscored() passes a reason= and the legacy unscored_reason metadata key is gone.

Category: code_quality · Applies to: eval, helper

What it does

Flags Score.unscored(...) calls whose reason= is missing, None or empty, and any occurrence of the string "unscored_reason" outside a docstring. Only attribute calls count, so a locally defined metric named unscored() is not mistaken for the constructor. One diagnostic per site.

Why is this bad?

Score.reason (inspect_ai 0.3.261) is the first-class record of why a sample was left unscored; metrics and log tooling read it there. The interim metadata["unscored_reason"] convention is superseded, and an unscored sample with no reason cannot be told apart from one the scorer forgot.

Example

return Score.unscored(explanation="grader returned nothing")

Use instead:

return Score.unscored(reason="grader_failed", explanation="grader returned nothing")

Suppress on a line with # inspect-evals-lint: ignore[IECQ003] or ignore[unscored_reason]; select or ignore it in configuration by either, or by the prefix IECQ.