If you really want to build each package in a separate container, you should consider using a functional package manager like GNU Guix. Each Guix package is built in its own isolated environment where only explicit inputs to the build function for that package are visible.
It should be relatively straightforward to write an importer from ROS packages to Guix packages.
Then you could do all sorts of cool things, like easily create a container for a ROS package with only the necessary dependencies included and nothing more to minimize container size. Or you could use it to verify the binaries provided by some server really correspond to the source code for safety or security reasons.
