IETS005: custom_solver_tests

Every @solver or @agent function name appears somewhere in the tests.

Category: tests · Applies to: eval, helper

What it does

Finds functions decorated with @solver or @agent in the package and checks each name appears in a test file. An agent is the solver of a sandboxed evaluation, so it is held to the same standard. For an evaluation the search covers tests/<name>/; for a helper package, the whole tests root, because shared components are usually tested next to the evaluation that motivated them. One diagnostic per untested function. This is a presence check, not a quality check.

Why is this bad?

A custom solver or agent is the evaluation's own logic, the part no upstream test covers. One test that at least constructs it catches import errors and signature changes.

See also

• Solvers: Custom Solvers
• Custom Agents

Suppress on a line with # inspect-evals-lint: ignore[IETS005] -- <reason> or ignore[custom_solver_tests] -- <reason>; select or ignore it in configuration by either, or by the prefix IETS.