Minimal pyproject.toml example

In this blogpost, I'll show you a minimal example of a pyproject file which uses flit in order to install it's dependencies. Already back in 2016, PEP 518 was created which aimed to fix the catch-22 of setup.py files (it depending on libraries which can only be defined in the setup.py file). It introduced the pyproject.toml file. On a personal note, I've tried to stick to setup.py files as long as possible (as I like the simplicity of it), but have now made the switch to pyproject.toml.

The setup.py example

The minimal setup.py we're going to convert.

import os
from setuptools import setup, find_packages

BUILD_ID = os.environ.get("BUILD_BUILDID", "0")

setup(
    name="titanic_ml",
    version="0.1" + "." + BUILD_ID,
    # Author details
    author="Niels Zeilemaker",
    author_email="nielszeilemaker@xebia.com",
    packages=find_packages("src"),
    package_dir={"": "src"},
    setup_requires=["pyspark[ml]", "sklearn", "pytest-runner"],
    tests_require=["pytest", "pytest-nunit", "pytest-cov"],
    extras_require={"develop": ["pre-commit", "bump2version"]},
)

I'll explain some of these lines in more detail. First, I get the BUILD_BUILDID. That's an Azure Devops specific environment variable, which is a reference to a specific pipeline run. I add that BUILDID to the version patch number.

Also, I'm a big fan of putting all code in a src folder. This is a slightly safer, and the general recommended way of structuring a Python project. Eg, see the pypa-sampleproject.

Then I define three different sets of requirements. setup_requires, which define the base dependencies of this project. tests_require, which define dependencies which need to be installed when running the tests (in this case when running python setup.py test). And finally, extras_require wherein I specify some development extras.

The pyproject.toml example

[project]
name = "titanic_ml"
description = "titanic_ml example package"
version = "0.1.0"
authors = [
    { name = "Niels Zeilemaker", email = "nielszeilemaker@xebia.com" }
]
dependencies = [
    "pyspark[ml]",
    "sklearn"
]

[project.optional-dependencies]
dev = [
    "tox",
    "pre-commit",
    "bump2version"
]

[build-system]
build-backend = "flit_core.buildapi"
requires = ["flit_core >=3.2,<4"]

I actually had quite some difficulties creating this file. This new pyproject.toml standard doesn't really feel like a standard just yet. At least not as defined in PEP 621. But in the end, the file itself is quite similar to the setup.py file i shared above.

Some observations, I cannot fetch the BUILD_BUILDID anymore, as this isn't a Python. But bump2version is a workable workaround. Additionally, I really wanted to make use of pip, and editable installations, eg pip install -e .. Using flit and pip 21.3.1, that is now possible.

Finally, python setup.py test is replaced by tox. That works, is a bit slower, but also much more feature rich (eg testing with different python versions).

I hope this is usefull, and good luck converting from setup.py to pyproject.tomls.