I’m working on perhaps one of the greater paper cuts in the ROS/Ubuntu ecosystem:
Basically, if you try to upgrade to a new Ubuntu release while also having a ROS release installed, the upgrade can cryptically fail because, of course, ROS doesn’t support that, and the Ubuntu upgrader doesn’t know that ROS doesn’t support that. I’m adding that knowledge to the Ubuntu upgrader so we can halt the upgrade before screwing up anyone’s computer with a partial upgrade, and I could use a quick sanity check (cc @dirk-thomas and @jrivero).
What is the best way to detect if a ROS distro is installed? The most robust way I can think of is to look at individual package names. There are two kinds of packages in the ROS repositories: those that are part of a given ROS distro (e.g. ros_environment and the like) and those that are not (e.g. rosdistro, colcon, etc.). I suspect that, for the most part, the latter group of packages will survive an upgrade without issue. The problematic set is the former, those that are part of a given ROS distro.
In order to avoid hard-coding as much as possible, I think it would be ideal if I could check to see if a specific package is installed, one that everyone who has any component of a ROS distro installed will have. For both ROS 1 and 2, I believe that would be ros_environment, can anyone confirm? Does anyone have any better ideas for how to do this? Does anyone see anything else I might be missing before I continue?
You know that the install path for binaries is /opt/ros/<distro>/*. You’d have to ask someone at OR if that’s configurable, but to my knowledge that’s just were it goes. You could check for an /opt/ros directory existing and then checking for the <distro>s specifically installed (e.g. Noetic and/or Foxy).
For the case of production robots (assuming they don’t use docker or similar) it could be installed anywhere if doing custom packaging so I’m not sure that would hold up for all users.
Indeed, although if you build from source you might very well have stuff there and upgrading won’t cause issues for you (at least, not with the upgrade itself). I don’t want to halt an upgrade that won’t be problematic. The problems come from the debs, so the test needs to be a little more fine-tuned than “is there stuff in /opt/ros/?”
Indeed, folks using snaps or docker needn’t worry, that will continue to work fine.
That will probably be true of this new feature in general, honestly. That said, I think we can improve the current situation dramatically, despite it not being a silver bullet.
dpkg -s ros-noetic-<pkg> will give you info only if installed. Either using that or another dpkg tool, you could find if the base ROS binary is installed as a way to know if ROS is around, e.g. dpkg -s ros-<distro>-ros.
Yeah that’s essentially the idea I outlined in the OP, although note that ros-noetic-ros is not the “most base” dependency. It depends upon roslib, which then depends upon ros_environment (e.g. if we checked for ros-noetic-ros, it would be possible to have roslib installed and we’d miss it). ros_environment has no dependencies, which leads me to believe it is the root dependency. I just want to be sure I’m right, and make sure there are no other dependency trees I’m missing.
This might help:
ros-noetic-ros depends upon:
ros-noetic-catkin (does NOT depend on ros-noetic-ros-environment)
ros-noetic-mk (indirectly depends on ros-noetic-ros-environment)
ros-noetic-rosbash (indirectly depends on ros-noetic-ros-environment)
ros-noetic-rosboost-cfg (does NOT depend on ros-noetic-ros-environment)
ros-noetic-rosbuild (does NOT depend on ros-noetic-ros-environment)
ros-noetic-rosclean (does NOT depend on ros-noetic-ros-environment)
ros-noetic-roscreate (indirectly depends on ros-noetic-ros-environment)
ros-noetic-roslang (does NOT depend on ros-noetic-ros-environment)
ros-noetic-roslib (depends on ros-noetic-ros-environment)
ros-noetic-rosmake (does NOT depend on ros-noetic-ros-environment)
ros-noetic-rosunit (indirectly depends on ros-noetic-ros-environment)
I think I’m wrong. There are multiple roots, here. I’ll write a script tomorrow to find them.
Alright, using the above metapackage, I propose that having any of the following packages installed should result in a halted upgrade (using melodic just as an example):
ros-melodic-ros-environment
ros-melodic-catkin
ros-melodic-rosboost-cfg
ros-melodic-rosclean
Any of those can be installed independently of the others, but it appears that all other ROS components would depend on at least one of them.
This list sounds kind of fragile. I would guess that there are other ROS packages (which are build with plain CMake and not catkin) which have no other ROS package dependencies.
Why not check if any ros-<distro>-* packages is installed since that is what would cause the problem of not being available in a newer Ubuntu distro? Depending on the output of e.g.
That is actually the first thing I considered, but that would flag the ROS-related packages in upstream Debian (e.g. ros-base, ros-core, etc.) as being problematic, when in fact they’re not because they (and their dependencies) are available in the next Ubuntu release as well.
I could work around that by including the ROS distro name (e.g. check for ros-melodic-*), but then we’re in a situation where we have stuff hard-coded and need to update it every time a new release comes out. That’s why I was hoping that looking for patterns like ros-[^\-]+-ros-environment would work. You’re of course correct, it’s fragile, but it seems less fragile (and require less ongoing maintenance) than the alternatives we’ve discussed so far.
I think a truly ideal solution would be for the upgrader to grow support for running hooks that can be installed by other packages such that ROS can maintain its own “hey you might want to think about this for a sec” check and change it as necessary, but that would require ROS to actually have that hook maintained as a truly root dependency. That kind of updater feature also can’t be SRU’d, but it might be worth doing for future releases if that sounds interesting. Doesn’t change this discussion though, I’d like to fix this all the way back to Ubuntu 16.04.
Sorry the editor ate the trailing <distro> part Yes, that was what I thought.
I assume the place where you want to add this check doesn’t have room for “dynamic” behavior like getting distro names from a yaml file online?
Hopefully nobody release a package ros-my-ros-environment to upstream since then your logic would think that ROS distro specific package is installed while only upstream Ubuntu package is…
In ROS 2 bloom enforces that every Debian package has a dependency on ros_workspace. I don’t think it is feasible to add something like that for ROS 1 for all active distros.
So your manual maintained list of package names is probably the best you can get for ROS 1 atm.
I was hoping to avoid it, yeah. I would probably need some fallback behavior defined if I couldn’t hit the index anyway.
A fair point. Thankfully I can make sure that’s the case in released Ubuntu versions (ignoring the possibility to install such a package from a non-standard repo, of course). At that point I feel okay about the level of risk. And you know what, I think I can just make this a warning and ask if they’re sure they want to proceed rather than bailing out entirely. That way a false positive just results in one extra prompt instead of a terrible annoyance.
Hey, just the nugget I was looking for, thank you! And yeah, don’t worry about the active ROS 1 distros, I doubt the dependency tree I’ve already found will change much for those anyway.
No, those are only defined if a workspace is activated. It also doesn’t necessarily imply that Debian packages are installed at all, which is really the issue here.
Could you look at where the packages on the system came from (e.g. APT-Sources in apt show) and warn on packages that didn’t come from a repo known to support dist-upgrades?
Yeah that would be trivial, but there are tons of repos out there (google chrome, steam, millions of PPAs, etc.). I think the false positive rate of that test would be tremendously high.
We could use the regexp ros-[^\-]+-ros-environment together with checking that the origin repository is packages.ros.org to make the check a bit more robust.
Checking that origin discounts the use of mirrors or other build farms though. I’m not sure how widespread that is, but I at least know several folks who run that way.
Yes, that was what I thought.