Image Scanner

This page provides instructions on how to setup and develop ESD python packages. Contact #esd-tech-help if you need any assistance.

Prerequisites

Before using Image Scanner, you need to install the following tools:

1. ds-deploy

Make sure you have the latest version installed.

You can install ds-deploy in two ways:

Option 1: Install using poetry or globally with pipx. See the ds-deploy installation guide for details.
Option 2: Clone the ds-deploy repository, run poetry install, and use poetry run ds-deploy image-scanner --help.

2. Trivy

Trivy is an external scanning tool required by Image Scanner. Install it based on your operating system:

macOS: brew install aquasecurity/trivy/trivy
Linux: curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin

For more installation options, visit the Trivy installation guide.

Note: The latest compatible Trivy version is 0.61.1. Verify your installation with trivy --version.

3. GCP Authentication

Authenticate with Google Cloud Platform:

gcloud auth login

ds-deploy image-scanner scan

The ds-deploy image-scanner scan requires:

Options:
  -n, --name TEXT             Cloud Run service name to scan
  -i, --image TEXT            Image URL to scan
  -f, --csv-file PATH         CSV file with image URLs (must have 'image' column)
  --stage [dev|staging|prod]  Stage for Cloud Run services
  -d, --reports-dir PATH      Directory to save reports. Defaults to home directory.

You can provide either a service name, a full image URL or a CSV file with image URLs to scan.

Usage Examples

Scan a specific Cloud Run service on prod stage

ds-deploy image-scanner scan --name my-service --stage prod

Scan multiple Cloud Run services

ds-deploy image-scanner scan --name service1 --name service2 --stage prod

Scan a specific Docker image

ds-deploy image-scanner scan --image registry.gitlab.com/project/image:latest

Scan multiple images

ds-deploy image-scanner scan --image image1:tag --image image2:tag

Scan images from a CSV file (must have 'image' column)

ds-deploy image-scanner scan --csv-file images.csv

Specify a custom directory for reports

ds-deploy image-scanner scan --name my-service --reports-dir /path/to/reports

Scan Reports

Understanding Scan Reports

Scan reports list "Common Vulnerabilities and Exposures" (CVEs) with: - Vulnerability ID: The unique CVE identifier - Package: The affected package - Fixed Version: The version where the vulnerability is fixed

Fixing Common Vulnerabilities

Using Base Images

Always use ESD base images: - Python 3.11: us-central1-docker.pkg.dev/ds-esd-shared/team-base/apps-py311:latest - Python 3.12: us-central1-docker.pkg.dev/ds-esd-shared/team-base/apps-py312:latest

# Example in Dockerfile
FROM us-central1-docker.pkg.dev/ds-esd-shared/team-base/apps-py311:latest AS base

Updating Dependencies

A common example is seeing an outdated package that is required by an internal or external dependency. If you find vulnerable packages:

Identify the parent dependency: poetry show vulnerable-package --why --tree
Update the parent dependency to the latest version in pyproject.toml.
Run poetry update --lock to regenerate dependencies.
Verify the fix with poetry show vulnerable-package.

Troubleshooting

These are common Docker errors that we ran into while building and scaning images:

ERROR: Failed to build installable wheels for some pyproject.toml based projects (psutil)

Solution: add --platform=linux/amd64 to docker build command.

ERROR: failed to resolve source metadata for us-central1-docker.pkg.dev/ds-esd-shared/team-base/apps-py312:latest: no match for platform in manifest: not found

Solution: Run docker pull us-central1-docker.pkg.dev/ds-esd-shared/team-base/apps-py312:latest --platform=linux/amd64.

FATAL ERROR (while running the scan)

...
FATAL   unable to find the specified image “image_name” in ["docker" "containerd" "podman" "remote"]: 5 errors occurred:
    * docker error: unable to inspect the image : Error response from daemon: No such image: 
    * containerd error: containerd socket not found: /run/containerd/containerd.sock
    * podman error: unable to initialize Podman client: no podman socket found: stat podman/podman.sock: no such file or directory
    * remote error: GET <image_name>: MANIFEST_UNKNOWN: Failed to fetch “<tag>”
    * remote error: GET <image_name>: MANIFEST_UNKNOWN: Failed to fetch “<tag>”

Solution: start an image docker container and run the scanner again.