IEBP006: gpu_sandbox_check

An evaluation requiring a GPU ships a maintenance sandbox check task.

Category: best_practices · Applies to: eval

What it does

When eval.yaml declares metadata.requires.gpu, tasks must include a task whose name ends _sandbox_check and which is declared with kind: maintenance.

Why is this bad?

GPU sandbox images cannot be exercised in ordinary CI, so a broken image (a missing package, the wrong Python, a CUDA toolchain that does not work) would only show up as errored samples in a real run. The check task certifies the image on GPU hardware through the evaluation's own scorer, and kind: maintenance keeps its accuracy out of listings that present model results.

Example

tasks:
  - name: kernelbench
  - name: kernelbench_sandbox_check
    kind: maintenance
metadata:
  requires:
    gpu: true

Suppress on a line with # inspect-evals-lint: ignore[IEBP006] or ignore[gpu_sandbox_check]; select or ignore it in configuration by either, or by the prefix IEBP.