This is definitely something that has been considered at various times. And it’s been thought about recently by @cottsay who has been looking into generating rpms. As rpms have stricter linters.
The automation of separating dbg symbols was eventually resolved with it becoming part of the standard debian toolchain:
From that issue you can find links to various other discussions of adding -dev packages as far back as 2011. But on reflecting on it I think that the reason that we haven’t found a solution for created them is that it doesn’t make sense to “automatically” split packages. There are many heuristics to say when to put something into the dev version or the runtime component. But in reality there’s many corner cases that will never be met so the maintainer will basically need to list out resources to land in either regular or dev variants of the package. In the Debian world this is the role of the debian package maintainer. Each resource is placed into one or the other bucket. But more importantly the downstream package maintainer then has to add the right dependency on either the runtime or devel package. That downstream maintainer also splits their package into runtime and devel packages. But more importantly they also split the dependencies conditionally on the devel and runtime dependencies appropriately.
Now you can see that we actually have 4 different cases for one dependency. There’s both devel and runtime versions of both packages. And the dependencies declared on the downstream package have to be conditionally evaluated for the subset of the package that is being used to build or made available.
We have a rich language of dependencies with at least 5 different versions of dependencies for a package including build, build_export, buildtool, buildtool_export, exec. And that’s excluding the extra test and doc dependencies as well as groups etc. We would then have to include a way to declare associations of each of those dependencies to the runtime or devel versions of a package.
Looking at this from a little bit further away, the main reason for having foo and foo-dev packages is because historically foo was released as a tarball and then the debian maintainers would pick up the tarball, and select what went to each subpackage, and add the appropriate dependency declarations. Since then the process has not evolved much, though now often it’s pulled directly from the source repository. But in general the upstream maintainers are not thinking about the impacts of releasing their software and packaging.
By comparison in the ROS ecosystem core developers actively use packages as atomic units and leverage those packages even when developing, and they use and pay attention to the package dependencies. The efficiency gains from modular packaging that are usually gained by the debian maintainers work can be leveraged in the development environment too.
As such I think that it might behoove us not to think of this as a problem to be fixed at the release stage, but as an area of efficiency that we can gain by structuring our upstream packages similiarly. The benefits of separating the build and runtime dependencies helps overall with compile times. Dependency trees do not grow as quickly if they are not the union of build and runtime dependencies. Parallel build tools can build faster when these potentials are paid attention to in the development environment.
And I think that many of our common practices that the ROS community has developed such as recommending making message only packages gets a lot to this end. They are the higher level equivalent of headers and break apart the dependency trees by providing standard interfaces. And through encouraging small modular packages we get the ability to leverage the find grain dependency control.
Specifically separating the build and runtime dependencies is something that has been shown to be quite effective that we’re not actively doing. However, instead of trying to do that at release time. I’d suggest that we consider making that a best practice for the ROS maintainers to create full packages that separate concerns into runtime and build time dependencies. This will give the developer the ability to fully specify the dependencies for each and where content goes is also clearly obvious. We could even pick up the naming convention of debian to name the content designed for build time to end with -dev however that would then conflict with the Fedora/RPM convention to name things with -devel.
In ROS 2 we’ve leveraged this separation of dependency types to enable swapping out the runtime libraries and hide the underlying implementations yet building them all against the same headers by defining a clear interface specification. There’s always a matter of tradeoffs between making things more modular and keeping things as simple as possible.
So in conclusion I think what I’d like to suggest is that we already have support for all the bells and whistles needed to very clearly define the necessary dependencies. And that if we can identify places where separating the build and runtime dependencies would be valuable that we consider simply making separate packages and not try to find ways to automatically split a package and then find ways to refer to those partial packages in downstream packages. We’re generating packages, we have packages. Keeping it one to one is basically the simplest solution.
