Team software registries

The Custom Solutions team manages private Docker, Python and NPM registries to host images and packages developed by our team. Both registries are hosted on Google Cloud using Google Artifact Registry and require authentication.

To access these registries, you need to be a member of any of the following groups:

ds-esd-gcp-analyst - custom solutions team analysts
ds-gcp-developers - custom solutions technical teams
ds-cust-registry-users - for users at DS outside of the custom solutions team

If you aren't a member of one of these groups and would like access to the registry, post to #esd-tech-help or send an email ds-esd-gcp-admin@digital-science.com.

You will also need the Google Cloud command-line tool installed and configured. Follow these instructions for setting that up with DS GCP projects.

Docker registry

This are notes for setting up your personal workstation. Please see the []

You must have the Google Cloud SDK (gcloud) installed and authenticated on your workstation.

Please follow the customer team instructions from the DS wiki on setting up and authenticating gcloud.

Run the following command once to configure Docker to use our private registry:

gcloud auth configure-docker https://us-central1-docker.pkg.dev

docker pull us-central1-docker.pkg.dev/ds-esd-shared/projects/hello-world:latest

Python registry

We use uv as our primary Python dependency manager. Authentication with our Python registry uses keyring to manage credentials.

UV configuration (Recommended)

For uv, keyring authentication is the recommended approach.

Step 1: Install Keyring

Install keyring as a global tool with the GCP authentication plugin:

uv tool install keyring --with keyrings.google-artifactregistry-auth

Verify keyring is configured:

keyring --list-backends

You should see an entry with GooglePython in it.

Step 2: Configure uv Global Settings

Create a global uv configuration file to use keyring for authentication.

macOS/Linux: Create or edit ~/.config/uv/uv.toml

# Create the directory if it doesn't exist
mkdir -p ~/.config/uv

# Create/edit the config file
cat > ~/.config/uv/uv.toml << 'EOF'
keyring-provider = "subprocess"

[[index]]
name = "cust"
url = "https://oauth2accesstoken@us-central1-python.pkg.dev/ds-esd-shared/python/simple"
EOF

Windows: Create or edit %APPDATA%\uv\uv.toml

# In PowerShell:
New-Item -ItemType Directory -Force -Path "$env:APPDATA\uv"

# Create the config file
@"
keyring-provider = "subprocess"

[[index]]
name = "cust"
url = "https://oauth2accesstoken@us-central1-python.pkg.dev/ds-esd-shared/python/simple"
"@ | Out-File -FilePath "$env:APPDATA\uv\uv.toml" -Encoding utf8

Step 3: Verify Configuration

Test that you can access team packages:

uv pip install dimensions-api-client --index https://us-central1-python.pkg.dev/ds-esd-shared/python/simple --dry-run

This should complete without errors, showing it would install the package (without actually installing it).

The keyring will work with the gcloud CLI tool to authenticate your requests to the registry.

Publish packages

If you are developing a package and need to publish to the team registry, you can add the following to publish to this registry.

If you are publishing your packages via Github Actions, which is the default, you won't have to add this as the shared Action will publish your package to this registry by default.

[[tool.poetry.source]]
name = "cust-publish"
url = "https://us-central1-python.pkg.dev/ds-esd-shared/python-publish"

After making this change, you will want to run poetry.lock to update your lock file to read from this new registry.

If you are using uv to manage Python dependencies, add the following to your pyproject.toml:

[[tool.uv.index]]
name = "cust-publish"
url = "https://us-central1-python.pkg.dev/ds-esd-shared/python-publish"
explicit = true

The keyring will authenticate your requests automatically.

pip configuration

For projects using pip directly, create or update your pip configuration file:

Unix/macOS: ~/.config/pip/pip.conf (preferred) or ~/.pip/pip.conf (legacy)
Windows: %APPDATA%\pip\pip.ini

Install keyring first:

pip install keyring keyrings.google-artifactregistry-auth

Add the following to your pip configuration file:

[global]
extra-index-url = https://us-central1-python.pkg.dev/ds-esd-shared/python/simple/

Poetry configuration (Legacy)

For legacy projects still using Poetry, first add the keyring plugin:

poetry self add keyrings.google-artifactregistry-auth

To use the registry for installing packages, edit your pyproject.toml file to include our team registry:

[[tool.poetry.source]]
name = "cust"
url = "https://us-central1-python.pkg.dev/ds-esd-shared/python/simple"
priority = "supplemental"

The keyring will work with gcloud cli tool to authenticate your requests to the registry.

You can also add a new poetry source from the command line:

poetry source add --priority=supplemental cust https://us-central1-python.pkg.dev/ds-esd-shared/python/simple

Publishing Packages

For UV projects: Add the following to your pyproject.toml:

[[tool.uv.index]]
name = "cust-publish"
url = "https://us-central1-python.pkg.dev/ds-esd-shared/python-publish"
explicit = true

The keyring will authenticate your requests automatically.

For Poetry projects (Legacy): Add the following to your pyproject.toml:

[[tool.poetry.source]]
name = "cust-publish"
url = "https://us-central1-python.pkg.dev/ds-esd-shared/python-publish"

After making this change, run poetry lock to update your lock file.

If you are publishing your packages via GitHub Actions, which is the default, the shared Action will publish your package to this registry automatically.

Migrating from GitLab Registry

Existing projects may reference the GitLab package registry. After configuring to use the GCP Artifact Registry as described above, you should remove any references to the GitLab registry from your pyproject.toml file. Remove the following sections if they are present:

[[tool.poetry.source]]
name = "internal"
url = "https://gitlab.com/api/v4/projects/30546120/packages/pypi/simple"
priority = "supplemental"

[[tool.poetry.source]]
name = "internal-publish"
url = "https://gitlab.com/api/v4/projects/30546120/packages/pypi"
priority = "supplemental"

Poetry and Docker

UV and Docker

For Python projects that use Docker with UV, the registry token will be passed to the build process via an environment variable.

Example docker build command:

docker buildx build \
        --secret id=python-registry-token,env=UV_INDEX_CUST_PASSWORD \
        . \
        --load \
        --tag $(image):${tag}

Your Dockerfile will then need to mount this secret when installing dependencies:

RUN --mount=type=secret,id=python-registry-token \
    UV_INDEX_CUST_PASSWORD=$(cat /run/secrets/python-registry-token) \
    UV_INDEX_CUST_USERNAME='oauth2accesstoken' \
    uv sync --frozen

Poetry and Docker (Legacy)

For legacy Python projects that use Docker with Poetry, the registry token will have to passed to the build process. This will typically be done via an environment variable called POETRY_HTTP_BASIC_CUST_PASSWORD. You will want to set it before running a docker build command with, export POETRY_HTTP_BASIC_CUST_PASSWORD=$(gcloud auth print-access-token).

Docker will pass this credential as a Docker secret.

Example docker build command:

docker buildx build \
        --secret id=python-registry-token,env=POETRY_HTTP_BASIC_CUST_PASSWORD \
        . \
        --load \
        --tag $(image):${tag}

Your Dockerfile will then need to mount this secret when invoking poetry install, like the following:

RUN --mount=type=secret,id=python-registry-token \
    POETRY_HTTP_BASIC_CUST_PASSWORD=$(cat /run/secrets/python-registry-token) \
    POETRY_HTTP_BASIC_CUST_USERNAME='oauth2accesstoken' \
    poetry install --no-root

NPM registry

Some Dash packages also publish resources to an NPM registry. Our team NPM registry is https://us-central1-npm.pkg.dev/ds-esd-shared/npm.

Please follow GCP Artifact Registry documentation for obtaining an access token to authenticate with our registry.

For environments without gcloud tools

In some environments (e.g., NIH), you may not be able to use the gcloud CLI. In these cases, authenticate using a service account key provided by the Custom Solutions team. Request the key from #esd-tech-help.

Python projects (uv/pip)

Once you have the key, configure UV or pip to use it:

# For uv
UV_INDEX_CUST_USERNAME=_json_key_base64
UV_INDEX_CUST_PASSWORD=<key-value>

# For pip
PIP_EXTRA_INDEX_URL=https://_json_key_base64:<key-value>@us-central1-python.pkg.dev/ds-esd-shared/python/simple

To verify that the internal registry is configured properly, run the following:

# With uv
uv pip install dimensions-api-client

# With pip
pip install dimensions-api-client

Poetry projects (Legacy)

For legacy Poetry projects, configure credentials:

POETRY_HTTP_BASIC_CUST_USERNAME=_json_key_base64
POETRY_HTTP_BASIC_CUST_PASSWORD=<key-value>

Update your pyproject.toml file to include the following:

[[tool.poetry.source]]
name = "cust"
url = "https://us-central1-python.pkg.dev/ds-esd-shared/python/simple"
priority = "supplemental"

Then run:

poetry add dimensions-api-client

Docker authentication

To authenticate with this team registry obtain a service account key and save the key as a field on your machine, e.g. key.txt. Then run:

cat key.txt | docker login -u _json_key_base64 --password-stdin https://us-central1-docker.pkg.dev

To verify access, you can pull a sample image:

docker pull us-central1-docker.pkg.dev/ds-esd-shared/projects/hello-world:latest

If building images in these environments, see the UV and Docker or Poetry and Docker sections above for passing the registry token to Docker.

More information

For questions, contact #esd-tech-help.

Official Google documentation: