I’m not able to mentor this one, as I suck at python, but perhaps someone else could:
Adding jobserver support to colcon
Detailed description:
Currently its not possible to limit the amount of threads used by colcon exactly. E.g. one can specify the amounts of threads used per package (T) and the maximum number of packages build in parallel (P). Therefore colcon can spawn up to P*T threads.
On machines with little RAM this can leads to a system freeze, as all memory is consumed by the threads and the system starts swapping. E.g. This mostly happens for c++ packages with heavy template usage, were one thread can consume multiple GB of RAM.
As a workaround either T or P can be reduced, which then leads to longer build times, which is not desirable.
A better solution would be, to implement a jobserver like the one in catkin. A jobserver allows to control the total amount of threads used by all make child instances of colcon. Additional, one can monitor the RAM usage of the system, and dynamically limit the amount of threads used, if a low RAM threshold is hit.
For more information on jobservers see: POSIX Jobserver (GNU make)
The catkin jobserver implementation can be found here: catkin_tools/catkin_tools/execution/job_server.py at main · catkin/catkin_tools · GitHub