The Python Packaging Authority (PyPA) is in the process of deprecating the use of setup.py to build and install Python packages. This mechanism has been used for ament_python packages since their inception, and is currently the only build mechanism supported for building ament_python and plain Python packages in colcon.
The replacement build process, often referred to as “standards-based”, is defined by several PEPs including:
- PEP 517 - A build-system independent format for source trees
- PEP 518 - Specifying Minimum Build System Requirements for Python Projects
- PEP 621 - Storing project metadata in pyproject.toml
- PEP 660 - Editable installs for pyproject.toml based builds (wheel based)
Supporting this new workflow in colcon (and eventually ROS) has several challenges. For a deep-dive into how these PEPs work and their relationship to colcon, please refer to the Infrastructure Community Meeting presentation from this past May.
I’ve created a prototype colcon package which can be used to “opt in” to the new Python workflows to help us gain test coverage and confidence in them before attempting to make them the new default in colcon. Installation instructions are in the README for that project, but to summarize, you need only build the new colcon extension in a colcon workspace and source that workspace.
I’ll keep the issue list for that repository up-to-date as we discover bugs. Also note the list of idiosyncrasies I’ve identified already.
To verify that the prototype is working, use colcon list to inspect the packages in your workspace. The prototype will detect python packages as python.project, and ros.ament_python packages as ros.ament_python.project. If you see those package types in the output, the prototype is enabled. To disable the prototype, simply don’t source the workspace in which it was built.
Once you’ve enabled the prototype, you shouldn’t need to rebuild or clean an existing workspace - just continue using colcon as before. Explicitly unsupported, however, is interacting with a workspace which built packages using the prototype, without the prototype enabled (it’s a one-way trip). If you disable the prototype, you should wipe and rebuild any workspaces you built with it.
We’re specifically interested in diverse workflows and environments:
- Platforms (i.e. various Linux distributions, Windows, and macOS)
- Symlink and non-symlink builds
- Merged and isolated builds
- Clean builds and rebuilding to overwrite
- New and older dependency versions (Mainly the Python interpreter and setuptools)
- Build backends besides setuptools (i.e. flit, poetry, etc)
- Packaging features like entry points, scripts, and data files
I look forward to your feedback and testing, and I’m excited to take advantage of the new features we’ll unlock as we leave setuptools behind.
–scott