container image scanning in the build pipeline

container image scanning in the build pipeline

TL;DR

Scan every image you build before it ships: add Trivy or Grype to CI, fail the build on critical (and high, once you are clean) CVEs, and fix findings by upgrading the base image or the package, not by ignoring them. Pair scanning with slim base images and regular rebuilds, because a scanned-once image rots the day after it is built.

container image scanning in the build pipeline

Use this when

• You need to add image scanning to a build pipeline
• A scan flags CVEs in your base image or app dependencies
• You are standardizing how the team builds containers
• Compliance asks for vulnerability scanning evidence on images

Not for this skill when

• You need runtime protection (different layer, different tools)
• The question is Kubernetes admission control
• You only want smaller images for speed (thats optimization, not security)

Steps

1. Scan your current images and see the baseline

Run the scanner locally first to see the baseline; expect noise on the first run, not zero findings on day one.

trivy image [registry]/[image]:[tag]

Expected: a table of CVEs by severity per layer and package. Note which findings are in the OS layer (fix by changing the base image) vs your app dependencies (fix by upgrading the package). Grype works the same way: grype [registry]/[image]:[tag].

2. Add the scan as a CI step that can fail the build

Put the scan after the image build and before the push to the registry. Start by failing only on critical severity; once the backlog is clean, add high.

trivy image --exit-code 1 --severity CRITICAL [registry]/[image]:[tag]

Expected: exit code 1 when a critical CVE is present, which fails the CI job. Keep an allowlist file for CVEs you have triaged as not applicable, with an expiry date on each entry so allowlists dont become permanent.

3. Fix findings at the right layer

OS-package CVEs get fixed by bumping or slimming the base image; app-dependency CVEs by upgrading the package and rebuilding.

grep -i "^FROM" Dockerfile

Expected: your base image lines. Prefer minimal bases (distroless, alpine-based official images, or your distro's minimal variant) and pin the digest, not just the tag, so rebuilds are reproducible.

4. Rebuild regularly, not just on code changes

Base images get security updates constantly; an image built six months ago carries six months of CVEs no matter how clean it was at build time. Schedule weekly rebuilds of your base images even when app code hasnt changed.

printf 'base-image rebuild: weekly, [day]\nowner: [team or name]\n' | tee -a image-maintenance.txt

Expected: a documented rebuild cadence with an owner. CI rebuilds the image from the pinned Dockerfile, rescans, and pushes only if the scan passes.

5. Scan before push and keep the evidence

Make the registry the enforcement point too: only scanned-clean images get pushed to the production repository. Save scan reports as CI artifacts so audits and incident response can see exactly what shipped.

trivy image --format json --output scan-report.json [registry]/[image]:[tag] && ls -la scan-report.json

Expected: a JSON report artifact attached to the CI run. When a CVE drops next month, you grep the saved reports to answer "which of our images are affected" in minutes.

Variant: trivy vs grype which to use

Both are good and free. Trivy covers more (misconfigurations, secrets, licenses) in one binary; Grype pairs naturally with Syft SBOMs. Pick one, standardize, and move on; the scanner you actually run beats the theoretically better one.

Variant: too many vulnerabilities in base image

Switch to a slimmer base first (most findings live in packages your app never uses), then pin the digest, then set up the weekly rebuild. If the official image is still noisy, check whether your distro publishes a minimal or hardened variant.

Variant: scan docker image before docker hub push

The pre-push hook version: run the scan locally with --exit-code 1 before docker push, or better, push only from CI where the scan is mandatory. Humans forget; pipelines dont.

Why this happens

Images bundle an entire OS userland plus your app, and every package in there is a CVE waiting to be disclosed. Nobody reads the full package list of a base image, so vulnerabilities accumulate invisibly until someone scans. Scanning in the pipeline turns "we think our images are fine" into a checked fact on every build.

Edge cases and pitfalls

• Scanner database lag: fresh CVEs take hours to appear in scanner DBs; a clean scan is "clean as of the last DB update," not a guarantee.
• Unfixable OS CVEs: some findings have no fixed version yet; allowlist with expiry and revisit, dont just suppress forever.
• Distroless debugging: minimal images have no shell, which complicates debugging; keep a debug-tag variant with busybox for incidents.
• Multi-arch images: scan every architecture you ship; CVEs can differ between amd64 and arm64 builds.
• Secrets in layers: scanners flag leaked secrets too; a secret in an old layer is still in the image even if a later layer deletes the file.
• Scan time on huge images: multi-GB images make CI slow; slimming the image fixes both the CVE count and the pipeline speed.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_5lJe12Furvj5C85Drtvdlg