Deprecate Ament Packages

General
21

Dear Katherine,

I am disappointed by your behavior as the community moderator. I see that your post targets me directly using my previous posts elsewhere while my criticism is totally about the technical state of ROS ecosystem and tooling. I do not say anything against people here and I do not accept that kind of behavior. Several other posters have stated their pain using similar words in this post. You also stated that you are getting such posts every year. Why do you think this is related to my personality and use my other posts elsewhere as a proof here? I find it very disrespectful.

I did my PhD on formal syntax and semantics of specification languages and therefore formal methods is my expertise. Of course I will write blog posts about the formal aspects of OpenScenario 2.0 standard linked in your post. As an experienced reviewer, I objectively state mathematical drawbacks in that proposal and would reject such proposal if it were a journal paper. This is my professional job. Build systems and tooling are not that mathematical yet general principles still hold. Simplicity, using less resources, the reduction of abstraction layers must be pursued in every possible way. Otherwise we see problems as in every complex software system and ROS is no exception. The particular goal is to eliminate every possible failure mode in the system (build system to runtime) so that we can use them reliably in safety-critical situations. Every complication in ROS ecosystem carries a potential risk to cause the loss of human lives and money. As a researcher working on the safety of real-time systems, it is my duty to point and criticize such problems. If you call it inflammatory, let it be
 But it is no joke.

Again I am really disappointed
 I expect a public apology here before we move forward.

Let’s back to the topic: For people asking a concrete proposal. It is clear in the title, my posts here, and in the posts Katherine shared. It’s the usual C++ way
 Simply deprecate ament, write pure cmake/setup.py, install at conventional locations. Let people learn general C++/python tools because this is regular C++ and Python. Look at Boost and QT libraries and others. Remember that ROS is a software library in essence.

I can do it myself in my projects. This is my intention. However, I recently realize that devs change conventional include directories in an LTS distro to solve a somewhat fringe problem even without mentioning in changelogs. Developers do not see it as a problem because of the over-reliance on ROS tooling. Then ROS claim their tooling is optional. Nope – ROS dictates it. Then bear the criticism please every year.

Other suggestions from previous posts were also good. Separate IDL generation. Separate topological sorting algorithm if you think that’s big deal. Separate package boilerplate generation.

Thank you.

22

I must agree with @doganulus here, @Katherine_Scott. I don’t find anything rude or angering in Dogan’s posts. They are disturbing for long-time ROS developers, yes, but not out of the moral lines of discussion. With the only exception - the straight dismissing of REPs and PRs to improve ROS. These things take time, Dogan, but they work if there is enough will and they are good.

I know this discussion hasn’t been too fruitful yet, as we haven’t seen any concrete proposal, but nobody is forced to read it, so I don’t see the reason why it would waste other developers’ time.

I think it is useful to discuss possible ways of improving the very building blocks ROS is built on. I think it’s apparent the build tool won’t change overnight, but as we’ve seen two or three times already that changing it is actually possible. So I’m really happy to see discussion, even yearly recurring, as the other technologies are developing and these discussions might actually (once!) bring a valid point.

Asking a newcomer to provide an extensive system using his proposed solution doesn’t seem reasonable to me. But I’d also weigh in to ask at least for a short example with e.g. 3 interdependent packages and their fully automatic build that retains the nice features of ROS packages like the ability to source or not source a particular workspace (so no installing to /usr/include). And no Docker, please, I hate it!!! :wink:

5 Likes
23

I am curious. Without the ament abstraction, how would this and this CMakeLists.txt look like?

They are the bare minimum examples of an interface and a publisher.

1 Like
24

If you want to be part of this community, you have to be a bit more constructive. Requesting an apology after you complained about how this community handles things gets you on a route of being ignored.

Complaining but not proposing a concrete solution (i.e. something that is proven to work) is not constructive.

As far as I can tell, ament is just a library of CMake macros. CMake is just a build-script generator (make, ninja, 
), and build-scripts just invoke compilers (GCC, clang, 
). And finally, colcon itself is just a meta-build tool to build a large set of heterogeneous packages using CMake, setup.py, meson, etc.

The stack of the build system is ament → CMake → make → GCC and you essentially want to get rid of ament. I think you can do this by just using the CMake functions called by ament. This will remove ament but gets you a much more verbose CMake file.

Now, if you continue to remove these abstractions in the stack, you will end up having to call gcc for all source files yourself. If you further remove colcon, you have to build every package manually in the correct order. This is probably not what you want. So where do we stop abstracting compilation?

If you propose an alternative to ament, I chime in with the others and suggest that you propose a concrete implementation or example that allows the community to evaluate the pro&cons compared to other solutions. You could take one of the core ROS2 packages (e.g. rclcpp) and give a working example of how the CMake file would look without using ament.

7 Likes
25

Dude, take it down a notch.

Criticism is always welcome, but without proposing an alternative to what you criticize, it’s very hard to take you seriously. Or at least, be more concrete about your criticism. I honestly can’t find what it is that you don’t like about ament, just vague statements about “the reduction of abstraction layers must be pursued in every possible way” (BTW, our entire discipline is about coming up with abstractions: languages, libraries, frameworks, etc.), something about ament_lint, and a comment about the location of the headers.

If you want this community to be on your side, you should at least attempt to understand how we work, there are no committees here. Learn about REPs, which is by no means a ROS invention, plenty of other open source communities have adopted the same approach (e.g. Python PEP, Rust RFCs), and make an attempt at understanding what you’re criticizing. How would you solve inter-package dependencies in a complex multilingual (Python, C++, C, Java, etc.) system? That’s exactly rosmake / catkin / ament’s raison d’ĂȘtre

And you’ve missed the context of Katherine’s comment, it’s not about people complaining about ament on a regular basis, it’s about newcomers to the ROS community who for some reason just want to change the foundations of this project without providing any justification. I’d even add that this usually happens when school is about to start (August/September) or during the winter break (January/February). In my over 10 years in this community, I’ve seen this happen so many times, and not only in the ROS community.

Anyway, there’s plenty to criticize about our packaging system for sure, my preference would be to integrate rosdep with conan, as it has solved some of the long-standing issues of rosdep (binary Windows support, multiple releases per package, etc.) and seems to be more and more widely used in the C++ world (it’s similar to Cargo, setuptools, etc. in many ways). But I appreciate and understand why our build system is the way it is and how it solves many complex problems that otherwise would make it almost impossible to build a decentralized open source robotics community that uses multiple languages for writing their software.

22 Likes
26

No, it’s not. This is a fundamental misunderstanding.

5 Likes
27

Why not give a little more explanation, instead of simply saying he is wrong? How not to misunderstand the role of ROS? I think this kind of reply will not make this discussion more educational or inspirational, but only add more fuel to the fire.

@peci1 and @esteve both made their points and I very much like to read more opinions like this. It’s helpful to know what issues others have faced and what ideas people have come up with, even if the ideas may be infeasible or incomplete at the time. Not discussing the issues or simply negating others’ ideas or existing solutions without explanation/new proposals are not helpful for the community to find a fix.

4 Likes
28

@doganulus, PM me if you’d like, and we can chat about ways to mold your ideas into an example and a more formal format, and perhaps bring them up at a TSC meeting.

4 Likes
29

You’re right. I’m sorry. I should have explained why it’s a misunderstanding.

ROS is not a software library. ROS is an ecosystem. It’s sometimes described as a meta-operating system.

An operating system is made up of a kernel and a huge number of packages that provide things like standard libraries for interfacing between the kernel and user-space applications, utilities for launching other utilities, file management utilities, documentation, graphical resources (e.g. icons and mouse cursors), and so on.

In the same way, ROS provides a huge number of packages that provide things like libraries for allowing nodes to talk to each other, those nodes themselves, libraries for use by other libraries or by nodes, files describing how to launch many nodes together as a single application, visualisation assets (CAD models, URDF files, etc.), debugging tools, introspection tools, simulators (note the plural), and so on. In other words, ROS is to robot applications what an operating system is to Microsoft Office (or KOffice, if you like).

All the tools that were developed to building and using ROS packages have to handle this massively broad range of things that people want to put in their packages. We try to follow existing conventions when they exist and provide what we need, but sometimes they don’t (or didn’t) exist, and sometimes they don’t provide what we need.

One example of something we need that the existing tools don’t/didn’t provide is a resource index. This is a tool that exists as a part of ament (if I recall correctly) and provides rapid lookup of whether certain resources exist and, if they do, what package they are in. Without this many ROS tools would be much slower to respond.

5 Likes
30

I like to think of ROS as a collection of user-space drivers and services for a kernel (Linux, Windows NT kernel, 
) which use IPC to exchange data, similar to how microkernel operating systems conceptually work.

1 Like
31

The important question here is how should non-ROS programs interface with ROS. I recently spent several months trying to get a non-ROS animated avatar system (actually it’s a quasi-ecosystem, not just a program) to interface with ROS-based robot controllers, and the result was a rather collosal cluster f@ck. We eventually did get it working but at no time during the project did I say anything close to ‘I’m so glad ROS has it’s own funky build system’.

Perhaps ROS really is an island but in case letting external parties visit from time to time, like often happens in the real world, would be beneficial, it would be good to define some docking and entry requirements at least. How about a way to interface with RMW without actually being on the island?

32

I think you’ve made a very good point. I’m a robotics engineer and mostly work on robotics projects. Even so, from my experience, if all packages for a project are created in the “ROS way”, then catkin/colcon generally works quite well, but things may not play out as nicely if you need to handle non-ROS components altogether. As to this point, colcon is already doing a much better job than catkin, but still, there are chances that you may get frustrated trying to figure out how ament/colcon stuff should live with standard C++/CMake stuff peacefully.

A few have mentioned conan. We’re also using conan for many of our standard C++ packages. I’m still not sure how to handle conan pkgs with ROS packages yet. Right now, the workaround we’re using is to generate the packages to .deb installation files and install them to the system for ROS packages to use.

1 Like
33

Not to stray too far from the general topic of the thread, but for those interested in Conan as part of the ROS2 dependency/packaging/build tool story, it is worth mentioning the exploratory(?) integration work that @alsora and other iRobot folks did:

I’m not sure whether this approach is still in active development, but it does touch on a number of the topics brought up in this discussion.

3 Likes
34

I would postulate that a way to help anyone bridge from ROS2 to anything else would be to provide a toolkit or library that talks (publishes/subscribes) to the ROS2 middleware layer as a ROS2 node could, but can otherwise be integrated with other custom code developed in C/C++/C#, Python, or some other popular language using any build tools, package managers, etc. that are applicable to the target platform and environment.

35

I bit offtopic, but it was mentioned here a couple of times that colcon and friends just use CMake in the background. You can build e.g. rmw_fastrtps_cpp just with CMake (cmake -B build && cmake --build build). It will even generate a standard cmake config-file (rmw_fastrtps_cppConfig.cmake) for you that you can use for any external CMake project. Of course, you have to install the dependencies yourself and then cannot rely on any of the “funky” tools like colcon or rosdep.

1 Like
36

Katherine’s firm answer is highly appreciated. Same issues in other open source forums as well. We have no time really to take care of a sudden brain ignitions with no background research.

38

Fork ros-infrastructure/rep, write REP, branch, commit, push, open PR.

1 Like
39

If anything, we’ve asked you to speak more. We’ve asked you to be more specific about the deficiencies you found in ament, some of us even proposed improvements given that you refuse to do so. Instead, you’ve written overall more words about Tony Hoare’s speech than about ament itself.

Please detail what exactly were the issues you faced, how you think the experience could be improved, and be respectful to the members of this community, nobody owes you anything and still here we are trying to understand how we can make ROS better for users like you.

8 Likes
41

Sure, ament hides the complexity of CMake and colcon hides the complexity of maintaining multiple build systems. Those are engineered tools for this purpose. If you do not like these abstractions, you can always go down the stack and even compile your cpp files manually if you wish.

The next time you take a plane, please forgo the pilot as an abstraction layer and just fly the aircraft yourself. Let’s see how safe that is. After all, the pilot should not hide the complexity of flying an aeroplane, right?

If you think that it is a big mistake that ROS, ament and colcon are eliminating the complexity of building robotic software, then why are you even here? You could use any combination of tools to make your robot go.

How many people would use an aircraft, if they have to fly it themselves? How many would use a PC or smartphone, if they would have to learn the command line? In engineering (and research) there is abstraction everywhere. But abstraction is needed in order to make technology more widely available. You are always free to look behind these abstractions.

No one tries to sell colcon as a tool to replace everyone else’s too. I don’t know any Mathematician that uses robots, btw. Everything in the ROS ecosystem is an engineered tool to solve typical problems that researchers in robotics are facing when managing large collections of software. You are free to use them or to use any other tool that solves your problems.

Now, you still haven’t defined your problem with ament properly. As far as I can tell, you don’t like ament’s abstraction of the C++ build process. You haven’t clarified why ament’s abstraction is a problem, but CMake’s abstraction is not.

You were asked a couple of times now to propose a concrete alternative solution. Unless you give a counter-example of how a typical robotic software stack with many heterogeneous packages and their dependencies can be built without the abstraction of ament, colcon and friends, I must assume that you are not really looking for a solution.

8 Likes
42

I still look out of the window before departure to see if the flaps are down (in the age of automated FMS checklists) knowing what happened at Wayne County Airport. Granted I can’t do much but still a reflex response.