Skip to content

IEBP007: dockerfile_locking

Dockerfile builds consume locked inputs: committed locks, digest-pinned images, fixed sources.

Category: best_practices ยท Applies to: eval

What it does

Reads every Dockerfile* under the evaluation statically (nothing is executed) and reports one warning per instruction with a build input that is not locked, pointing at the instruction's first line:

  • Dependency installs must consume a committed lock or hashed snapshot: uv sync --locked with uv.lock copied or bind-mounted in beforehand, or pip install --require-hashes -r <snapshot>. uv sync alone (may update the lock), uv sync --frozen (skips freshness validation), uv lock during the build, pip install <packages>, requirements files without --require-hashes, and installing the project with dependency resolution all warn; pip install . --no-deps after locked dependencies is accepted. Manifests are not inspected: ranges there are fine when the consumed lock resolves them.
  • Image and source inputs must be immutable: FROM and COPY --from references need an @sha256 digest (scratch and earlier build stages are exempt), Git dependencies on a pip command line need a full commit, and an installer script piped from curl or wget into a shell warns because nothing verifies it. COPY --from a digest-pinned tool image is the accepted way to bring in uv.
  • Dynamic references (${VAR} images, remote or interpolated COPY sources, templated package names) and unsupported package managers (npm, cargo, conda, ...) are reported as unverified, never as passing.
  • COPY sources resolve against the build context: the # BUILD_CONTEXT=<path relative to the repository root> directive inspect_evals uses, else the build.context of a compose service that builds the Dockerfile (compose resolves dockerfile relative to it), else the Dockerfile's directory. Missing sources and sources outside the repository warn. BuildKit heredocs are skipped and .dockerignore is not consulted.

Limits, kept visible: OS package installs (apt-get install and the like) cannot be locked by this rule, so a Dockerfile whose other inputs are locked still passes, with each such step named in the pass message. Build-isolation dependencies of source builds and the interpreter version against the sandbox project's requires-python are not checked. Passing means the supported installs consume locked inputs, not that a rebuild is byte-identical. Every finding is a warning in this release. Dockerfiles matched by exclude are not read; an evaluation without a Dockerfile skips.

Why is this bad?

A digest-pinned published image fixes what runs today, but not what a rebuild produces: a Dockerfile that installs whatever the index or registry serves on the day gives a different sandbox each time the image is rebuilt, so results stop being comparable without anything in the repository changing, and a broken upstream release can turn into errored samples. Consuming a committed lock makes the rebuild a function of the repository alone.

Example

FROM python:3.12-slim
COPY pyproject.toml ./
RUN curl -LsSf https://astral.sh/uv/install.sh | sh && uv sync

Use instead:

# BUILD_CONTEXT=.
FROM python:3.12-slim@sha256:<digest>
COPY --from=ghcr.io/astral-sh/uv:0.12.5@sha256:<digest> /uv /usr/local/bin/uv
COPY src/inspect_evals/my_eval/sandbox/pyproject.toml src/inspect_evals/my_eval/sandbox/uv.lock ./
RUN uv sync --locked --no-dev --no-install-project

Options

  • dockerfile_locking.host-lock-coupling: "warn" reports a build that copies the repository root's pyproject.toml or uv.lock, because unrelated host dependency updates would then change the sandbox image; "allow" accepts it, for a standalone repository whose root project is the sandbox. warn in the monorepo preset, allow elsewhere.

Suppress on a line with # inspect-evals-lint: ignore[IEBP007] or ignore[dockerfile_locking]; select or ignore it in configuration by either, or by the prefix IEBP.