I do understand why ament packages introduced at the first place but I do not see the reason why they need to continue. A roadmap to deprecate them from ROS community would be much needed and we instead use modern C++ and Python tooling directly without the thin layer of abstraction that ament packages provide.
Since the community here is already familiar with ament packages, it would be hard to see why I see this as a problem and write such a post here but, for good software design and achitecture, such unnecessary abstractions must be eliminated from the system and workflows as much as possible. Let people learn standard C++ and Python tooling instead of ROS tooling, which cannot cope with their pace apparently.
Agreed. These ament tools appear to be quite brittle as they seem to break every OS version and/or python major version. They also seem to be quite blackbox due to lack of documentation. The amount of times I have had to delve into some random ament cmake include to figure out what is actually going on behind the scenes is crazy.
I think you must only be talking about python because C++ as a language does not have any standard tooling that replaces ament. You could argue that cmake is a defacto standard for the build generator of C++ projects, and therefore, we should be using CPM instead of ament, but the use of CPM is in no way a standard for packaging. There are Conan, VCPkg, and others, but there is no ‘standard C++ tool’ for building/packaging in the way such a thing exists in the python ecosystem.
I think what he’s referring to is that so much of ament_cmake is just a wrapper for standard CMake practices. And that it is somewhat confusing for someone coming from a CMake background who is used to doing things manually.
E.g. Why should I use ament_export_targets and ament_export_dependencies? What are these doing that I can’t do myself as a CMake developer?
I suppose the answers are along the lines of:
It removes a lot of boilerplate
It provides a degree of consistency across ROS packages
It reduces the barrier to entry for new people
Some of the things it replaces are fairly complex and impractical/impossible to write manually
As someone who is certainly not an expert in CMake, I find it simultaneously confusing and helpful. On the one hand there are all these functions that “just work” and you have to know to use that are different to the other 99% of CMake projects out in the wild. On the other hand, having fumbled around to implement them manually in other projects, it’s very nice to have a system that “just works”.
As I mentioned in another thread we are transitioning some libraries from normal CMake to ROS/ament. Using macros to maintain both systems at once (so we can build on our existing non-ROS platforms) is a pain, but we generally feel that if we had the option to only use one or the other, we’d take the convenience of ament.
I haven’t worked with ament myself yet, but assuming it is built on the same idea as catkin, it should not be tied too much to ROS. You can install catkin via pip and use it for non-ROS projects pretty easily. Assuming ament is similar in this, you could also think about removing the pure CMake code and keeping only ament CMake code - and say that ament is a build dependency of your project in the same way CMake is.
The standard tooling for C++ is the C++ compiler. There is already a battle-tested way to search for headers/libs implemented in compilers, which we expect from package managers. Place your artifacts under /usr, /usr/local/, or /opt/prefix and we are good.
Other newish (standard) tooling for C++ is clang-format, clang-tidy, clangd. I really do not understand why we do need to wrap them too using ament_lint for example. In more general, we should recommend people use those tools directly instead of ament.
Adding an abstraction layer just for convenience is a big no-no in software design yet we love it…
Do you invoke your C++ compiler directly, or do you use a build system?
ament_lint provides cmake functions that allow you to add linters to ctest in an easy way. CMake is the defacto standard in C++ and CTest is how you run tests in that environment. There are several different kinds of linters wrapped by ament_lint and I find some of them more useful than others. If you are familiar with cmake and you go read the source of ament_lint you will be surprised to find the cmake functions there are actually really boring and straight-forward.
One thing I think you are forgetting is that, ignoring CPM, normal cmake does not have a straightforward way to build cmake packages that depend on each-other in an automated way.
As to placing build artifacts in system folders (and searching for them there). Here are a couple of objections to that:
Have you worked on multiple projects with conflicting dependency graphs? Should a user with that requirement be required to maintain multiple system configurations?
Have you worked in other languages (python, rust, android, etc)? If anything having a directory local dependency system for building self-contained projects is more modern than depending on searching paths in environment variables.
How, without the ament/colcon tooling do you propose to have workflows where a developer can vendor and contribute to upstream projects as part of their project?
Responding to this:
Adding an abstraction layer just for convenience is a big no-no in software design yet we love it…
Is your compiler not just a convenience abstraction of ELF? Using that must be a big no-no for you. I would say that software design is the process of creating and iterating on our tools/libraries until we can build more useful software. Robotics includes many disciplines and therefore has tooling you might not be familiar with. Can you give us an example of a large project that does this whole software thing correctly?
You can write colcon-compatible pure CMake and the ros2 pkg create tooling even creates some of the boilerplate to do that for you. I’ve used it in a few places where it seemed appropriate. While I don’t think the ament/colcon system is as unreasonable as you seem to, I can’t wait to drop C/C++/make/CMake/Python and just use Rust/Cargo.
Sorry but the requirements of ROS build is not that special than you imagine.
In the beginning I have said I understand why ROS tooling has emerged in this way. But version control, containers, and build systems have developed and improved since those days. In 2022, ROS tools still try to implement features useful in 2000s. My answer simply is to use docker. Clone it and build it in docker. Multiple times. In many different configurations. That’s how software development is done today.
Besides I wouldn’t compare the impact of compilers with macros for linters in any sentence. This piece actually shows why thin wrappers are detrimental for software engineers and developers. Preventing ROS developers to utilize the full power of compilers (or another sufficiently impactful abstractions) behind a few magic functions (rocker is another example). The convenience provided by those scripts ties one of your hands in the fight.
Nothing about how ament_cmake works prevent you from using ROS packages in that way, you can change the install directory of CMake to some system directory and build the graph yourself manually. Docker is a really obnoxious requirement for development. Many things that were once easy become really annoying if you require your users to build in Docker.
We’ve discussed the pros and cons of Catkin/Ament/Whatever-in-ROS a few times.
Personally I’m not against changing things, but repeating the same arguments a couple of times will likely not result in a community of people used to a certain workflow to just agree with you, drop whatever they’re doing and start doing things your-way / the better-way.
I realise this is dangerously close to “this is open-source, if you want to complain, fix it yourself”, but in this case, I believe the best demonstration of the benefits of any alternative would be an (example) implementation of what you propose.
Personally, I’m not in any fights. I just want my ROS package’s build scripts done and out of the way so I can get to the interesting parts: developing the actual functionality.
Build scripts are so low on my list of annoyances I’d rather other things would get attention first. But that’s my personal list, not the list-of-the-ROS2-community. I can imagine others in the community feel the same way, which would make recruiting developer resources for your idea perhaps difficult.
If you feel strongly about this, and have ideas for how things can be changed, write a REP, a design document, or show a (reference) implementation.
Hand-waving with claims of “compilers have done this for ages” and “Docker is the answer, just build” are not constructive.
Build complex software is a hard task. This is the fight or challenge I mention above, not implying I am fighting with you personally. Again I have said you may be familiar with all the tradition since the days of catkin but I am definitely not happy with the ROS tooling. I try to avoid it as much as possible. Yet I like the pub-sub messaging and API.
Regarding the REP and PRs: I knew the history of computer science. I have read the ACM speech of Hoare, the Emperor of New Clothes. I couldn’t convince the committee by any means by REPs or implementations. Committees are for incremental improvements. No committee will ever do this until it is too late. It must change from the bottom.
If you would like a change from the bottom up, I’d think it would go a lot easier if you could provide a concrete suggestion on your proposed improved workflow and then see if the community as a whole or a part of the community agrees with you and is willing to rally around your proposal to improve it and adopt it.
Without a concrete suggestion it’s hard to reason about the pros and cons of anything.
@doganulus do you have examples of your work to show how we could do better at this? I am happy to consider any alternatives to a point. I don’t really want to spend much time fighting with build tools and not working on making robotics better.
I totally understand your pain - I have spent more hours than I care to admit trying to maintain a mental grasp on which ament CMake macros I should/should not be using for various use-cases, what each actually does, when it is ok (or necessary) to mix standard CMake calls (like ament_target_dependencies() for some dependencies, but target_link_libraries() for others).
That said, I tend to agree with others here that there needs to be a concrete proposal for an alternate workflow that handles all of the use-cases that ament works to try and support for the majority of different types of users, veteran CMake users and ROS2-newcomers alike. While I would love to have one less abstraction layer, like you, it is often easier and less painful for developers like us to directly read and understand ament’s cmake implementation than it is for a new users/junior developers to figure out all of the CMake boilerplate they need to add dependencies to both core ROS2 dependencies and public packages.
I’ve tried to think of solutions to this myself, but ament always ends up feeling like the lesser-of-evils. My favored approach would be to support the use of the Conan C++ package management tool for everything dependency-related in ROS2, as it supports plain CMake approaches to specifying and exporting dependencies by supporting the creation/use of proper CMake packages and modules.
That said, Conan does not currently jive well with ROS/ROS2’s rosdep dependency management system, and it can also be a little painful to use with colcon. It is also yet another abstraction/tool that people must learn and understand, so you end up trading one problem for another.
I don’t see any major shifts happening here unless you or someone proposes and comprehensively demonstrates an actual alternative. Until then, like @tylerweaver jokes, I’ll await the day we can switch to rust.
@doganulus, I am a moderator for ROS Discourse. About once a year we have someone new to the community complain that they don’t understand / like our build tooling (Catkin, Colcon, or Ament) and that we should change them to some other sort of build tool.
It would be helpful for you to recognize a few things.
Our tools were often built prior to the existence of viable replacements. The ROS project is well over a decade old. This history has inertia and makes it difficult to change things quickly.
Our goal is to build tools that help those who may not be CS experts (e.g. electrical / mechanical engineers) have a solid foundation in robotics as well as supporting the power users. The standardization that ament provides avoids a lot of boilerplate code and simplifies the development for larger systems.
ROS is a big project, like a tanker ship, you can’t just turn the whole thing around on a dime.
When we do turn things around it requires the effort of a huge community to both build and test the changes and to roll them out to a giant community of downstream users who may not care about your particular issue.
Simply driving by and suggesting that we should change something for you simply because you don’t like our “level of abstraction” is not exactly a way to win friends and influence people.
We have our ways of changing the project. I am sorry you don’t like those either, but that’s how things get done in a community of this size.
I have already had multiple users claim you are being inflammatory and moreover, this seems to be a pattern of behavior. If you want to take a breath, propose a viable alternative to ament, gather sufficient resources to make that alternative happen, and follow our processes for changing ROS, by all means, make the proposal. Otherwise you are simply being inflammatory and angering a lot of your very busy colleagues. If that’s the case I am inclined to lock the thread.