Another example contribution was the fact that searching gdb on the ROS tutorials yielded zero results, and I want to use GDB all the time. printf debugging is horrible with a two hour cross-compile build. Thus, I added docs on debugging ROS tests with gdb just last week
Can you point us towards the tutorial? I’m honestly frustrated with throwing debug printf’s everywhere around my code.
An interesting thing I’d like to share for people who use debug printf’s extensively, I recently came across a macro called RCLCPP_INFO_EXPRESSION, which prints an info message if a given expression is true. This comes in handy as you can have a parameter called verbose or debug, and then use this everywhere for debug prints.
I’m not sure about this new tutorial - but nav2 has some tutorials on profiling and getting backtraces (especially in a large, multi-node system like nav2):
Additionally, some issues is doing similar things in VSCode because ROS requires running a setup script (install/setup.bash), but VSCode tasks only support environment variable arguments, not running a whole environment setup script.
The best recommendation I have until VSCode and ros2 launch are more GDB/ROS friendly is use terminal GDB, which you can learn a bunch from Tridge and Henry here:
I’m happy to discuss/help more, but feel it’s a bit off-topic, so feel free to follow up privately or tag me another thread or Github issue. Debugging is important, and the barrier to entry is quite high still when using ROS 2.
I fully sympathise, having been part of an organisation trying to support (an) OSS project(s) while the appearance of much more resources being available made it very difficult to engage community members and find sufficient contributors.
What would perhaps help in this particular case is to clarify the current structure of the ROS 2 project: who is responsible for what, who are their employers, who is paying for what, which parts are now the sole responsibility of the OSS community, etc.
The number of times I’ve heard “yeah, but Google is now backing ROS 2, so Z” is almost innumerable.
For Z, choose one of:
“ROS 2 is a commercial product now” (!)
“resources are less of a problem” / “they have google-resources now” (whatever that means)
“software standards and software quality will now quickly become much higher”
“there is much more money compared to when it was a non-profit” (!)
“it’s much more serious now as a project”
“a company is now behind it” (?)
etc, etc
Clearly, it’s far from obvious what happened the past couple years after Intrinsic “acquired the for-profit part of OR”.
Many (all?) Intrinsic contributors still seem to use their old accounts (here, on Github, etc) and many package manifests still place them at openrobotics.org or one of the other old (non-profit) domains. Many of the (core) repositories are also still fully maintained by “old” accounts, who are the only ones with push or higher permissions. Getting a PR merged basically means interacting with “a core developer of ROS”, which are still the same accounts as always.
It’s apparently also far from clear what the future of the project is. It may be clear to some, but “the larger community” does not appear to understand it.
And I feel this in part is responsible for a lack of contributions: instead of it being clear ROS 2 is no longer maintained by a single organisation which somehow managed to find sufficient funds to develop new functionality, to pick up issues identified and to keep working on it – from WillowGarage times to now with a fully not-for-profit two-man foundation with some core maintainers – there is this pervading feeling everything will work out because that’s how it always worked with ROS.
(And then we get to complain when something doesn’t get fixed or implemented, which can only mean someone is purposefully ignoring the community for some reason … /s)
Have their been any updates from either Intrinsic or OSRF as to what will happen after the first two years where nothing will change for ROS, or who are now expected to do what? I don’t believe there have been, but I’m also not paying as much attention as I used to.
As a project, ROS got “bought by Google” (not what happened), then the people who controlled everything for so long communicated “nothing will change” (of course, also not true eventually) and now we’re surprised many still look to OSRF to organise everything, find resources for everything and keep the project running despite everything?
That was already a problem before the transition, and it is much more so now.
Edit: just to clarify: I’m genuinely worried here. If anything in this post comes across as sarcastic (with the exception of the /s there), is not assuming good faith or in general seems harsh, that was not my intent. I’m blaming my Dutch-ness.
As much as I want to agree with you on the systematic issues, I think we need to solve them one at a time. If you find other systematic issues, can you please bring it up on threads so that we could actively focus on solving them ? I think we’ve had enough conversation about the “existence” of a broader problem but I think the key to the solving the problem is to address them “specifically”
This helps to clarify specific problems I have raised in the past, such as the organization of community representative elections and the task of forum moderation. It’s not professional that those basic governing functions that require neutrality are performed by people employed by commercial companies. Intrinsic is not Willow Garage or the OSRC. This hampers the credibility of the Foundation as a standalone entity.
Still, I always address my questions and comments to the Foundation and people who are appointed to certain positions by the Foundation. Despite the misrepresentation of my comments here, the Foundation has a very important function to me. Together with effective governance, I believe even the existing resources would be already sufficient to drive the community from the stagnation which I have observed since I joined. Nay-saying and dismissal are what welcome any newcomers. Suppressing criticism on forums slowly kills innovation. I am not shy about fighting back, but not everyone likes that. I understand that people can burn out and get unmotivated, but then people in certain positions step back if they really care about the community.
Beyond that, the Foundation primarily must have a mission to organize and coordinate activities with member companies and the community. Unfortunately, I haven’t seen any vision statement from the Foundation since the last year, so I can only speculate. The Foundation must remain neutral and aim for the excellence of the ROS ecosystem and robotics, supporting research and education. Being open-source is not an excuse for low quality. A can-do attitude is really a game-changer. The courage to destroy what you did is very important. This is how to start again for the better.
Off-topic, as I agree with we need actions, not more words:
Click to expand
I’ve been involved in this community one way or another for the past 15+ years. I’ve also been involved in galvanising people to contribute more. I’ve tried so many things I don’t believe I would have anything to offer you now.
What you’re experiencing now, and what you’re trying to do is not new. I remember similar sentiments when WillowGarage stopped and O(S)R(F) took over (note: I’ll be eternally grateful to the ex-WillowGarage employees that did).
re: what can be done: I already pointed out something that could be done: be clear about who’s (going to be) responsible for what and be brutally honest about what will happen if/when people don’t pick up those things.
In my vey humble opinion, there is just too much money in robotics and AI, making the (perceived) cost of contributing so high (and the benefit of just taking from OSS so big) that it just doesn’t happen nearly as often as it should.
(note that the cost of contributing is also high because of long times to initial peer reviews, sometimes high number of iterations before PRs are accepted and misalignment between the goals of an OSS project and contributors)
Edit:
How would this work then? Are only academics or people with no affiliation whatsoever allowed to perform these tasks? Or should they be fully employed by whatever organisation it is they are governing?
Reality shows that is not feasible, at least not without substantial financial backing from other organisations.
Such support typically comes with strings attached.
Could you not start to move over a little to our side already to help address some of the long-standing issues you have identified a couple of times already?
I am involved with the ROS community because of Autoware. As you know, the automotive field requires much higher safety and software engineering levels. That’s why I am attending the newly formed production of WG according to my concerns and expertise. I would be happy if you read and give your feedback here: ROS Package Collections · Issue #11 · bpwilcox/ros2-production-working-group · GitHub
I’m about to head off on a vacation so I can’t reply in depth right now, but thank you very much for your specific and well articulated questions and concerns @gavanderhoorn . Problem statements like this are a welcome contrast to what I’ve seen in the past, and far less condescending. Furthermore, I appreciate that you’re not trying to dictate the particular form of the solution, nor are you making blanket and borderline offensive statements about the quality of work and motivation of those who are working on such things.
A response will be forthcoming, thank you for your trust in the meantime.
I hope you come up with a board proposal or decision to move the Foundation and community forward under the new governance. I think I had my points clear, which are not my personal agenda but well-known practices to ensure well-functioning governance. The separation of community positions from personas is one of the key elements in any solution. This should not be a family business, at least not anymore.
Repeating again, I am talking about the responsibilities and qualities of board members, TSC members, forum moderators, and community representatives. Although I constantly get personal attacks, veiled threats, and mobbing on the forum, I carefully keep my comments on the governance and technical issues only. If you manage to separate positions and personas, you will not feel them offensive. Not at all.
Is this actually the case? I feel like there is more support for ROS 2 than at any time since the demise of Willow Garage:
I count roughly 20 members of the TSC - each of which has pledged one FTE to ROS 2 - that’s on par with the size of the software team at Willow Garage and more than we had for most of the in between years. The downside of course is there are now 20 different employers setting goals rather than a single one.
We have multiple active working groups that are actually implementing and improving things in ROS 2. High level applications like Nav2 and MoveIt2 have significant development going on.
ROSCon was huge this past year - even the trade show floor was packed with vendors (and the cost of those booths has gone up significantly from the earlier days of ROS - so this is a much bigger financial commitment from industry to show up to such an event).
Documentation seems to be the big discussion right now - but it has always been a struggling point for ROSrobotics - it’s not flashy, it’s pretty thankless to contribute towards, and it is actually a huge effort. Back at the beginning of ROS, Willow Garage had 3 major milestones - milestone 2 was getting a PR2 to autonomously navigate and self recharge for a 26.2 mile marathon using ROS - milestone 3 was basically to write documentation for ROS! Guess which was harder to get people to contribute towards?
How are the TSC and Working Groups not accomplishing this? Honest question here - because from the latest TSC minutes I see:
A coordinated effort on rmw_zenoh_cpp involving several companies that are members of the TSC (and this effort was started based on the result of a TSC-organized survey of ROS users last year or so).
A coordinated effort on 24.04 setup for Jazzy.
Active working groups for Nav2, MoveIt2, and ros2_control (there may be others, but these are the ones I’m most familiar with). WGs have monthly-ish meetings and have numerous contributions going on. They aren’t centrally managed/directed by the foundation, but isn’t that kind of the idea of a big distributed software ecosystem?
Honest questions get honest answers. First, it is necessary to distinguish two classes of activities in any large-scale project.
The first class includes feature-development activities. These activities bring new features to the existing systems. I call them existential, as one successful attempt makes you successful. Everyone likes them. These tasks are easily parallelizable and suitable for open-source (bazaar) style development. Nav2, MoveIt, and ros2_control development belong to this class. No problem with those…
The second class is about infrastructure, maintenance, testing, documentation, etc… I call them universal tasks, as a single failure makes you fail. You do not get much credit for doing them and get the worst criticism if you cannot succeed. You cannot leave them to the community, as failures are costly. This is where the Foundation cannot delegate its responsibility to the community. This is where the most problems lie and I tell that to the Foundation whenever I see such an attempt:
Instead, the Foundation should seek sustainable solutions by organizing/taxing their member companies. Shifting the burden/blame to the community is not nice. This is why foundations exist.
What exactly is “the community” in your mind? I don’t understand how the foundation should “not delegate to the community” but “organize/tax their member companies”? Member companies are (a major) part of the community.
Also, is that first link (the Infrastructure Project Committee) supposed to be an example of a “problem” - because I see that link as a fine example of the foundation doing exactly what they should be doing - the committee includes the CTO and two employees of OSRF along with representatives from multiple companies/projects within the ecosystem announcing a more formal structure for what used to be done ad-hoc and inviting additional contributions.
Just some recognition on this effort. Sometime back in 2016 you mentioned a need for experts to respond to users on ROS Answers and that started my habit of reviewing questions I thought I could give quick answers to (over my morning coffee) which continues to this day. If you hadn’t given the nudge, I wouldn’t be still doing it today in 2024 with a larger mandate to nudge people in the right direction wrt Nav2 as well.
I think you did/do a great job and the attempts are worth the effort
Community (academics, students, volunteers, open-source enthusiasts, hobbyists) and member companies are separate. Yet perhaps it is clearer in the eligibility condition of community representatives, which explicitly excludes the employees of TSC companies.
I find relying on the volunteer effort problematic on the infrastructure. And I do not see how the existing proposal can attract more companies where they should drive those efforts collectively, as I do not think Intrinsic is a charity.
I’m sorry, but that’s just not true. The TSC charter notes that:
They represent the “broader” community who are unable to meet the higher threshold of “1 FTE” per member organization - but the other member organizations are still part of the ROS community. There is no us-vs-them - everyone is a single community - some may contribute more than others, but we either succeed or fail together in supporting ROS.
This never works. It is not about us-vs-them. It is about the separation of roles, responsibilities, and capabilities. Companies can do certain things better; individuals do other things better. The separation of roles and responsibilities is healthy.
The whole field of Law is based on this separation.
@mikeferguson I think the main point @doganulus wants to make is that the OSRF needs a sustainable source of funding, in order to tackle the ‘second class’ tasks, which are not or not efficiently done if left to the community.
Where this funding should come from is not really relevant in the context of this discussion. While @doganulus proposes to tax TSC member companies, others (including me) can have another opinion. But I do agree that the OSRF needs funding.
