Skip to content

Package Development

This page provides instructions on how to setup and develop ESD python packages. Contact #esd-tech-help if you need any assistance.

Project Setup & Package Generation

Using the GitHub Template

The recommended way to create a new Python package is to use our GitHub template repository. This uses a hybrid approach combining GitHub templates with cookiecutter automation.

Steps:

  1. Navigate to the template repository:
    Go to cust-infrastructure-esd-python-package-template

  2. Create a new repository:
    Click "Use this template" → "Create a new repository"

  3. Choose a descriptive repository name (e.g., my-awesome-package)
  4. Add a description (this will become your project description)
  5. Choose visibility (typically private or internal)

  6. Click "Create repository"

  7. Run the template workflow:

  8. Go to the Actions tab in your new repository
  9. Click on "Setup Project from Template" workflow
  10. Click "Run workflow" button
  11. Configure your project (all fields are optional and will use defaults if not provided):
    • Project name: Defaults to your repository name
    • Project description: Defaults to repository description
    • Author name: Defaults to your GitHub username
    • Author email: Defaults to no-reply@dimensions.ai

  12. Click "Run workflow"

  13. Wait for generation:
    The workflow will take a few minutes to generate your project structure. You can watch the progress in the Actions tab.

  14. Clone and start developing: 

    $ git clone git@github.com:digital-science/your-repo-name.git
$ cd your-repo-name
$ uv sync

Package Layout

The package directory will look like this:

├── .github/
│   └── main.yml              # Move to .github/workflows/ when ready
├── .gitignore
├── .pre-commit-config.yaml
├── Makefile
├── README.md
├── mkdocs.yml                # Documentation configuration
├── pyproject.toml            # Package and dependency configuration
├── uv.lock                   # Lock file for dependencies
├── docs/
│   ├── index.md              # Main documentation page
│   ├── reference.md          # API reference
│   └── usage.md              # Usage examples
├── <package-name>/
│   ├── __init__.py
│   └── main.py              # Your package code
└── tests/
    ├── __init__.py
    └── test_base.py         # Test files

Developing your Package

Feel free to refer to our extensive technical team documentation for the Developer Setup. You can start with UV setup.

Testing

After adding source code to your package folder, make sure to add necessary unit tests in tests/ starting with test_. Feel free to use the test_base as a starting point.

You can then run your tests with: 

$ uv sync
$ make tests

Or run directly with uv: 

$ uv run pytest

Documentation

The template provides a standardized documentation format in Markdown. Run the docs server with: 

$ make docs_server

Refer to the ds-analysts-tools docs for detailed examples of the pages.

GitHub & Commits

By default the template has two configured branches, main and develop. We recommend you work in develop and then pull request to main.

While staging the changes and committing to the branch pay attention to the conventional commits format and versioning. You can read more about our commits here. The commitizen tool is setup for dev dependencies and is configured to fail when using incorrect commit messages.

Deployment

Whenever you are ready to deploy, you need to request a Github Runner for your repository by reaching out to #esd-tech-help and then move workflow files to activate an existing CI/CD: 

$ git clone git@github.com:digital-science/your-repo-name.git
$ cd your-repo-name
$ mv file-name.yml .github/workflows/
$ git add .github/workflows/
$ git commit -m "ci: activate CI/CD workflows"
$ git push

The existing CI/CD pipeline will pick up your changes pushed to main and deploy your package to our package registry.

Updating Python version and dependencies

Please keep your package updated to the latest Python version supported by the team to ensure its compatibility with our packages.

To change the Python version of your package follow the steps below. Let's say you want to switch to Python version 3.13: 1. Change the python version constraints to include the new version in the pyproject.toml file. 

# pyproject.toml
[project]
requires-python = ">=3.10,<3.14"
2. Update your virtual environment by running uv python install 3.13 and then uv sync to recreate the environment with the new Python version. 3. Run uv lock to update the lock file with the new dependencies. 4. Check with uv sync and uv run pytest if everything works correctly.