An Updated Guide to Setuptools and Pyproject.toml

In the past I’ve written multiple blog posts on how to create a Python package with Setuptools. Since then the recommended way of using Setuptools has shifted in the direction of using the pyproject.toml in favour of setup.py and setup.cfg. In this blog post I’ll provide a simple and concise guide to packaging a Python project with Setuptools.

Starting Point

To create a package, we need to have source code. In this guide, our starting point is the following directory structure:

project_directory/
├── src/  
│   └── example/             Python package with source code.
│       ├── __init__.py      Makes the folder a package.
│       └── source.py        An example module containing source code.
├── tests/  
│   └── test_source.py       A file containing tests for the code in source.py.
└── README.md                README with information about the project.

Inside the project directory, we have a src/ directory which contains the Python packages. In our case, the package directory is src/example/. This directory contains __init__.py, which makes Python treat it as a package, and source.py, which contains the actual source code. There can be additional python files (or ‘modules’) and subdirectories (‘submodules’, each with their own __init__.py file).

The project directory also includes a file named README.md. This file is commonly found in software repositories and provides important information about the project or repository. While not strictly necessary, we will assume it exists for the purpose of this guide.

Lastly, there is a tests/ directory. Although not mandatory, I think that any package worth sharing is also worth testing. Note that the tests are located outside of the source directory and will not be included in the package.

Typically, you will have more files and folders in the project directory, such as a .gitignore or a LICENSE. These are, however, not required.

Creating a Package

In order to package the project, we need to add a single file to the project directory: the pyproject.toml. This file will contain the metadata that defines the package. In our example it could look like this:

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "example"
description = "An Example Project"
version = "0.1.0"
readme = "README.md"
dependencies = ["PyYAML==6.0.1"]

The [build-system] table defines that our package will be built by Setuptools, and that this requires the setuptools package of at least version 61.

The [project] table sets the metadata of the package: the name, description, and version. It links the README.md file as a readme, and defines that PyYAML==6.0.1 is a dependency.

Note that the name matches the name of the source package directory: "example". This is no requirement, but it may be confusing if it is not. An example of a package where the package name and the directory do not match is Scikit-Learn: you install it using pip install scikit-learn, while you use it by importing from sklearn.

Also note that there are requirements on the allowed version strings. See docs on version specifiers for a complete specification.

Building a Package

The above example is all that you need to build a package. To do so, run:

python3 -m pip install build
python3 -m build

This will install the build package and uses it to build your package. It will create dist/ directory and create both a source distribution (sdist, named example-0.0.1.tar.gz) as well as a binary distribution (wheel, named example-0.0.1-py3-none-any.whl).

You can distribute those files and install your package from them, for example:

pip install dist/example-0.0.1-py3-none-any.whl

This will install the package from the wheel into the Python environment.

If you have created a package that you would like to share with the world, you can upload the distributions to PyPI. Instructions can be found here. You will have to come up with a more original name than "example", though!

Editable Install

Aside from building a package, and subsequently installing it, you can also install the package from source code into your Python environment. Running

pip install -e .

will make it available to Python. This makes it easy to develop with the code.

Advanced Configuration Options

The example pyproject.toml above is roughly the minimum that is required to create a package. There is, however, much more you can configure.

Metadata

First, you may want to specify more metadata regarding the project, such as authors and maintainers of the project:

[project]
authors = [
    {name = "Your Name", email = "your@email.address"},
    {name = "Your Co-Author", email = "their@email.address"},
]
maintainers = [ 
    {name = "Your Name", email = "your@email.address"}
]

To make your package easier to find on PyPI, you can add keywords:

[project]
keywords = ["some", "words", "that", "are", "applicable"]

Additionally, in order to further help search, add classifiers to categorize your release:

[project]
classifiers = [
    "Development Status :: 4 - Beta", 
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3",
]

A full list of classifiers can be found here.

If, on the other hand, you want to make sure your package is never uploaded to PyPI, add the Private :: Do Not Upload classifier. PyPI will always reject packages with this classifier.

If you have a non-standard license you can link the file like this, but for standard licenses providing the applicable classifier is sufficient:

[project]
license = {file = "LICENSE"}

Finally, you can provide a list of URLs in the project.urls section:

[project.urls]
Homepage = "https://your.project"
Documentation = "https://docs.your.project"
Repository = "http://github.com/you/your.project"

Dependencies and Requirements

The dependencies parameter is not required, but so common that I included it in the minimal example. It takes a list of strings, each representing a dependency:

[project]
dependencies = [
    "numpy",
    "pandas>=2.0",
    "PyYAML==6.0.1",
]

This defines that we require any version of numpy, version 2.0 or greater of pandas, and exactly version 6.0.1 of PyYAML. See the docs for all possibilities.

Aside from the dependencies key, there is more you can configure regarding the requirements and dependencies of your project. Perhaps most important: you can specify the required python version. For example:

[project]
requires-python = ">=3.8"

Any dependencies that are not strictly required, or only used in certain situations, can be added as optional requirements:

[project.optional-dependencies]
dev = ["pytest>=7.4.3", "ruff==0.1.8"]

This is commonly used to specify dependencies that are required for development, as in the example above. Installing the package like pip install "example[dev]" (or pip install -e ".[dev]" when doing an editable install) will trigger pip to also install the optional dependencies.

Other common use-cases are adding optional dependencies for a GUI or a data source. Of course you should make sure your package can also work without these – and does not fail with an ImportError if they are not present.

Scripts

In order to make part of your code executable as a CLI command, you can add a script. For example, if there is a function called main in example/source.py, this would make it available as your-cli-command:

[project.scripts]
your-cli-command = "example.source:main"

If you are using typer as your CLI framework and your source.py file contains the following code:

from typer import Typer

app = Typer()

@app.command()
def hello():
    print("Hello.")

@app.command()
def bye(name: str):
    print(f"Bye {name}")

You can configure the script as follows to enable running commands like your-cli-command bye rogier:

[project.scripts]
your-cli-command = "example.source:app"

Dynamic Versions

Hard-coding the version of your package in the pyproject.toml may not be ideal, as it requires you to update it manually. Updating the version in the file every time can be error-prone and time-consuming, especially in larger projects with frequent updates.

Additionally, if you want your package to have access to its own version, you will have to add a global variable with the version to a source package. This means you will have to manually keep those versions in sync, making the process even more elaborate.

Fortunately, setuptools provides options to dynamically fill the version. If you are using git (of course you are!), your easiest option is to use setuptools_scm. You will need to add it as a requirement to the build-system, and specify that you want the version to be dynamic:

[build-system]
requires = ["setuptools", "setuptools-scm"]

[project]
# version = "0.0.1"  # make sure there is no version parameter
dynamic = ["version"]

[tool.setuptools_scm]
version_file = "src/example/_version.py"

The [tool.setuptools_scm] section must exist, but may be empty, to enable setuptools_scm. The version_file parameter ensures that setuptools_scm writes a __version__ and a __version_tuple__ to a file inside your package, allowing the package to have access to its own version.

The version is determined from the latest tag and the number of commits since the latest tag, and a date is added when there are uncommitted changes in your tree. More specifics can be found here.

Package Data

In a typical Python package, you usually find code, either as (Python) source code or binary. However, there are cases where you may need to include data files in your package for use by the package itself. To achieve this, setuptools provides multiple options.

The easiest way is to ensure that these files are tracked by git and to add setuptools_scm as a build-system requirement (see above).

If you prefer more fine-grained control over which files to include, you can list them explicitly. For example, if you want to include all .txt files:

[tool.setuptools.package-data]
example = ["*.txt"]

This instructs setuptools to include all files with the .txt extension in your example package. You can then access them using importlib.resources.

Wrap-up

The contents below are a good starting point for building a Python project with Setuptools. For further reference, see Python Packaging User Guide – Writing your pyproject.toml.

[build-system]
requires = ["setuptools", "setuptools-scm"]
build-backend = "setuptools.build_meta"

[project]
name = "example"
dependencies = [
    "PyYAML==6.0.1"
]
requires-python = ">=3.8"
authors = [
    {name = "Your Name", email = "your@email.address"},
    {name = "Your Co-Author", email = "their@email.address"},
]
maintainers = [ 
    {name = "Your Name", email = "your@email.address"}
]
description = "An Example Project"
readme = "README.md"
license = {file = "LICENSE"}
keywords = ["some", "words", "that", "are", "applicable"]
classifiers = [
    "Development Status :: 4 - Beta", 
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3",
]
dynamic = ["version"]

[project.optional-dependencies]
dev = ["pytest>=7.4.3", "ruff==0.1.8"]

[project.scripts]
your-cli-command = "example.source:main"

[project.urls]
Homepage = "https://your.project"
Documentation = "https://docs.your.project"
Repository = "http://github.com/you/your.project"

[tool.setuptools_scm]
version_file = "src/example/_version.py"

After copying this, please ensure to review each line for any necessary modifications.

Photo by Brandable Box on Unsplash