I do understand why ament packages introduced at the first place but I do not see the reason why they need to continue. A roadmap to deprecate them from ROS community would be much needed and we instead use modern C++ and Python tooling directly without the thin layer of abstraction that ament packages provide.
Since the community here is already familiar with ament packages, it would be hard to see why I see this as a problem and write such a post here but, for good software design and achitecture, such unnecessary abstractions must be eliminated from the system and workflows as much as possible. Let people learn standard C++ and Python tooling instead of ROS tooling, which cannot cope with their pace apparently.
I agree, using any āspecialā build tools may make things somewhat easier for new, non-programmer ROS users, but it actually makes it VERY much harder to integrate ROS into other solutions!
Agreed. These ament tools appear to be quite brittle as they seem to break every OS version and/or python major version. They also seem to be quite blackbox due to lack of documentation. The amount of times I have had to delve into some random ament cmake include to figure out what is actually going on behind the scenes is crazy.
I think you must only be talking about python because C++ as a language does not have any standard tooling that replaces ament. You could argue that cmake is a defacto standard for the build generator of C++ projects, and therefore, we should be using CPM instead of ament, but the use of CPM is in no way a standard for packaging. There are Conan, VCPkg, and others, but there is no āstandard C++ toolā for building/packaging in the way such a thing exists in the python ecosystem.
I think what heās referring to is that so much of ament_cmake is just a wrapper for standard CMake practices. And that it is somewhat confusing for someone coming from a CMake background who is used to doing things manually.
E.g. Why should I use ament_export_targets and ament_export_dependencies? What are these doing that I canāt do myself as a CMake developer?
I suppose the answers are along the lines of:
It removes a lot of boilerplate
It provides a degree of consistency across ROS packages
It reduces the barrier to entry for new people
Some of the things it replaces are fairly complex and impractical/impossible to write manually
As someone who is certainly not an expert in CMake, I find it simultaneously confusing and helpful. On the one hand there are all these functions that ājust workā and you have to know to use that are different to the other 99% of CMake projects out in the wild. On the other hand, having fumbled around to implement them manually in other projects, itās very nice to have a system that ājust worksā.
As I mentioned in another thread we are transitioning some libraries from normal CMake to ROS/ament. Using macros to maintain both systems at once (so we can build on our existing non-ROS platforms) is a pain, but we generally feel that if we had the option to only use one or the other, weād take the convenience of ament.
I havenāt worked with ament myself yet, but assuming it is built on the same idea as catkin, it should not be tied too much to ROS. You can install catkin via pip and use it for non-ROS projects pretty easily. Assuming ament is similar in this, you could also think about removing the pure CMake code and keeping only ament CMake code - and say that ament is a build dependency of your project in the same way CMake is.
The standard tooling for C++ is the C++ compiler. There is already a battle-tested way to search for headers/libs implemented in compilers, which we expect from package managers. Place your artifacts under /usr, /usr/local/, or /opt/prefix and we are good.
Other newish (standard) tooling for C++ is clang-format, clang-tidy, clangd. I really do not understand why we do need to wrap them too using ament_lint for example. In more general, we should recommend people use those tools directly instead of ament.
Adding an abstraction layer just for convenience is a big no-no in software design yet we love itā¦
Do you invoke your C++ compiler directly, or do you use a build system?
ament_lint provides cmake functions that allow you to add linters to ctest in an easy way. CMake is the defacto standard in C++ and CTest is how you run tests in that environment. There are several different kinds of linters wrapped by ament_lint and I find some of them more useful than others. If you are familiar with cmake and you go read the source of ament_lint you will be surprised to find the cmake functions there are actually really boring and straight-forward.
One thing I think you are forgetting is that, ignoring CPM, normal cmake does not have a straightforward way to build cmake packages that depend on each-other in an automated way.
As to placing build artifacts in system folders (and searching for them there). Here are a couple of objections to that:
Have you worked on multiple projects with conflicting dependency graphs? Should a user with that requirement be required to maintain multiple system configurations?
Have you worked in other languages (python, rust, android, etc)? If anything having a directory local dependency system for building self-contained projects is more modern than depending on searching paths in environment variables.
How, without the ament/colcon tooling do you propose to have workflows where a developer can vendor and contribute to upstream projects as part of their project?
Responding to this:
Adding an abstraction layer just for convenience is a big no-no in software design yet we love itā¦
Is your compiler not just a convenience abstraction of ELF? Using that must be a big no-no for you. I would say that software design is the process of creating and iterating on our tools/libraries until we can build more useful software. Robotics includes many disciplines and therefore has tooling you might not be familiar with. Can you give us an example of a large project that does this whole software thing correctly?
You can write colcon-compatible pure CMake and the ros2 pkg create tooling even creates some of the boilerplate to do that for you. Iāve used it in a few places where it seemed appropriate. While I donāt think the ament/colcon system is as unreasonable as you seem to, I canāt wait to drop C/C++/make/CMake/Python and just use Rust/Cargo.
Sorry but the requirements of ROS build is not that special than you imagine.
In the beginning I have said I understand why ROS tooling has emerged in this way. But version control, containers, and build systems have developed and improved since those days. In 2022, ROS tools still try to implement features useful in 2000s. My answer simply is to use docker. Clone it and build it in docker. Multiple times. In many different configurations. Thatās how software development is done today.
Besides I wouldnāt compare the impact of compilers with macros for linters in any sentence. This piece actually shows why thin wrappers are detrimental for software engineers and developers. Preventing ROS developers to utilize the full power of compilers (or another sufficiently impactful abstractions) behind a few magic functions (rocker is another example). The convenience provided by those scripts ties one of your hands in the fight.
Nothing about how ament_cmake works prevent you from using ROS packages in that way, you can change the install directory of CMake to some system directory and build the graph yourself manually. Docker is a really obnoxious requirement for development. Many things that were once easy become really annoying if you require your users to build in Docker.
Weāve discussed the pros and cons of Catkin/Ament/Whatever-in-ROS a few times.
Personally Iām not against changing things, but repeating the same arguments a couple of times will likely not result in a community of people used to a certain workflow to just agree with you, drop whatever theyāre doing and start doing things your-way / the better-way.
I realise this is dangerously close to āthis is open-source, if you want to complain, fix it yourselfā, but in this case, I believe the best demonstration of the benefits of any alternative would be an (example) implementation of what you propose.
strong statements.
Personally, Iām not in any fights. I just want my ROS packageās build scripts done and out of the way so I can get to the interesting parts: developing the actual functionality.
Build scripts are so low on my list of annoyances Iād rather other things would get attention first. But thatās my personal list, not the list-of-the-ROS2-community. I can imagine others in the community feel the same way, which would make recruiting developer resources for your idea perhaps difficult.
If you feel strongly about this, and have ideas for how things can be changed, write a REP, a design document, or show a (reference) implementation.
Hand-waving with claims of ācompilers have done this for agesā and āDocker is the answer, just buildā are not constructive.
What @gavanderhoorn said, show us at least an example of a large ecosystem of packages you think does something better that we can learn from. Or write a REP or a PR?
Build complex software is a hard task. This is the fight or challenge I mention above, not implying I am fighting with you personally. Again I have said you may be familiar with all the tradition since the days of catkin but I am definitely not happy with the ROS tooling. I try to avoid it as much as possible. Yet I like the pub-sub messaging and API.
Regarding the REP and PRs: I knew the history of computer science. I have read the ACM speech of Hoare, the Emperor of New Clothes. I couldnāt convince the committee by any means by REPs or implementations. Committees are for incremental improvements. No committee will ever do this until it is too late. It must change from the bottom.
If you would like a change from the bottom up, Iād think it would go a lot easier if you could provide a concrete suggestion on your proposed improved workflow and then see if the community as a whole or a part of the community agrees with you and is willing to rally around your proposal to improve it and adopt it.
Without a concrete suggestion itās hard to reason about the pros and cons of anything.
My concrete solution to all the pain that is the current build/packaging/library ecosystem we find ourselves in is to adopt rust as quickly as possible.
Where did the committee come from? There is no committee. At least not one that was instated by anyone as far as Iām aware.
The bottom-up approach you advocate works best by showing a concrete implementation of your ideas. Or at least a prototype.
You havenāt posted anything more than your dislike of Catkin/Ament. Point us to your alternative. Or describe ā clearly ā how your alternative would work.
which is about as constructive as @doganulusās posts.
@doganulus do you have examples of your work to show how we could do better at this? I am happy to consider any alternatives to a point. I donāt really want to spend much time fighting with build tools and not working on making robotics better.
I totally understand your pain - I have spent more hours than I care to admit trying to maintain a mental grasp on which ament CMake macros I should/should not be using for various use-cases, what each actually does, when it is ok (or necessary) to mix standard CMake calls (like ament_target_dependencies() for some dependencies, but target_link_libraries() for others).
That said, I tend to agree with others here that there needs to be a concrete proposal for an alternate workflow that handles all of the use-cases that ament works to try and support for the majority of different types of users, veteran CMake users and ROS2-newcomers alike. While I would love to have one less abstraction layer, like you, it is often easier and less painful for developers like us to directly read and understand amentās cmake implementation than it is for a new users/junior developers to figure out all of the CMake boilerplate they need to add dependencies to both core ROS2 dependencies and public packages.
Iāve tried to think of solutions to this myself, but ament always ends up feeling like the lesser-of-evils. My favored approach would be to support the use of the Conan C++ package management tool for everything dependency-related in ROS2, as it supports plain CMake approaches to specifying and exporting dependencies by supporting the creation/use of proper CMake packages and modules.
That said, Conan does not currently jive well with ROS/ROS2ās rosdep dependency management system, and it can also be a little painful to use with colcon. It is also yet another abstraction/tool that people must learn and understand, so you end up trading one problem for another.
I donāt see any major shifts happening here unless you or someone proposes and comprehensively demonstrates an actual alternative. Until then, like @tylerweaver jokes, Iāll await the day we can switch to rust.
@doganulus, I am a moderator for ROS Discourse. About once a year we have someone new to the community complain that they donāt understand / like our build tooling (Catkin, Colcon, or Ament) and that we should change them to some other sort of build tool.
It would be helpful for you to recognize a few things.
Our tools were often built prior to the existence of viable replacements. The ROS project is well over a decade old. This history has inertia and makes it difficult to change things quickly.
Our goal is to build tools that help those who may not be CS experts (e.g. electrical / mechanical engineers) have a solid foundation in robotics as well as supporting the power users. The standardization that ament provides avoids a lot of boilerplate code and simplifies the development for larger systems.
ROS is a big project, like a tanker ship, you canāt just turn the whole thing around on a dime.
When we do turn things around it requires the effort of a huge community to both build and test the changes and to roll them out to a giant community of downstream users who may not care about your particular issue.
Simply driving by and suggesting that we should change something for you simply because you donāt like our ālevel of abstractionā is not exactly a way to win friends and influence people.
We have our ways of changing the project. I am sorry you donāt like those either, but thatās how things get done in a community of this size.
I have already had multiple users claim you are being inflammatory and moreover, this seems to be a pattern of behavior. If you want to take a breath, propose a viable alternative to ament, gather sufficient resources to make that alternative happen, and follow our processes for changing ROS, by all means, make the proposal. Otherwise you are simply being inflammatory and angering a lot of your very busy colleagues. If thatās the case I am inclined to lock the thread.